Are the Chinese infecting hardware? Someone is lying

Bloomberg has reported that a company, Supre Micro, Inc., has had their hardware hacked, maybe with the knowledge or encouragement of the Chinese government. Impacted customers reportedly include Apple Computer and Amazon, who may have had their data centers compromised. Apple, Amazon, and Super Micro Inc have all issued strong denials.

The attack as described involves a tiny chip being surreptitiously inserted on the board of one of Super Micro Inc’s suppliers. According to the report, the chip could insert code that would allow for malware to be installed. We’ll come back to how to address that attack at a later date.

While this attack is at least feasible in theory, and while it is possible for vendors to keep a secret, and indeed it has enraged many people in the past that a bunch of vendors have kept secrets for quite a while, here we have a report where we have denials all around, and yet we have a somewhat detailed description of the attack. There are only three possibilities:

The reporters and their sources are accurate; in which case there is a MASSIVE conspiracy that includes Apple and Amazon, not to mention government officials.
The reporters are wrong, and have been fed corroborated yet false information by government sources.
The reporters are fabricating a story.

An existence proof – one board – would suffice to show that (1) is true. Proving (2) would be quite difficult without recorded conversations of confidential sources. (3) is also difficult to prove.

Let’s hope the reporters are fabricating the story, because the alternatives are far worse. If the reporters are accurate, we either have vendors standing on their heads or government sources feeding media a pack of lies. Furthermore, although China has broken into the computers of adversaries in the past, it would be particularly bad for false accusations to circulate that could later be used to discredit or tarnish those that are true.

More to come.

Where a bad review really makes for poor security

Releasing unstable software harms cybersecurity for everyone, not just those who install the product.

Most consumers do not take the time to upgrade their devices simply because vendors want them to: there has to be something in it for me. Apple, on the other hand, has been an exception. Studies have repeatedly shown that Apple users do regularly upgrade their phones. Just one month after release, their latest version was installed on 52% of their devices. By comparison, summing all Android releases from 2015 to present gets you that same number, with the latest releases coming in around 20% of the total.

This becomes a Big Deal when we start talking about vulnerabilities, and zero-day exploits. If there is a bug in your device and it is running an older version of the code, and you do not update, then that device can be used to attack you or someone else. This is something that Microsoft learned the hard way in the last decade when it snuck in extra software in a security update, losing trust and confidence and willingness of their users.

In his review, Gordon Kelly has told his Forbes readers not to upgrade to the latest Apple iOS release precisely because it may be too risky, that the release itself was rushed. When considering release timing, any vendor always has to balance stability and testing against other feature availability and security. Apple may well have gotten the balance wrong this time. The review in and of itself harms cybersecurity, not because the reviewer is wrong, but because the result will be that fewer people will have corrected whatever vulnerabilities exist in the release (as of this writing information about what is fixed hasn’t been disclosed). Moreover, such reviews reinforce a bad behavior- to delay upgrading. I call it a bad behavior because it puts others at risk.

This isn’t something that can be fixed with a magic wand. We certainly cannot fault Mr. Kelly for publishing his analysis and recommendations. If we wait for perfect security, we will never see another feature release. On the other hand, if things get too rushed, we see such bad reviews. Perhaps this argues that O/S vendors like Apple and Google should continue to provide security-only releases that overlap their major releases, at least until they are stable, which is what other vendors such as Microsoft and Cisco do. It costs money and people to support multiple releases, but it might be the right thing to do for the billions of devices that are each and every one a point of attack.

Ain’t No Perfect. That’s why we need network protection.

If Apple can blow it, so too can the rest of us. That’s why a layered defensive approach is necessary.

When we talk about secure platforms, there is one name that has always risen to the top: Apple. Apple’s business model for iOS has been repeatedly demonstrated to provide superior security results over its competitors. In fact, Apple’s security model is so good that governments feel threatened enough by it that we have had repeated calls for some form of back door into their phones and tablets. CEO Tim Cook has repeatedly taken the stage to argue for such strong protection, and indeed I personally have friends who I know take this stuff so seriously that they lose sleep over some of the design choices that are made.

And yet this last week, we learned of a vulnerability that was as easy to exploit as to type “root” twice in order to gain privileged access.

Wait. What?

Ain’t no perfect.

If the best and the brightest of the industry can occasionally have a flub like this, what about the rest of us? I recently installed a single sign-on package from Ping Identity, a company whose job it is to provide secure access. This simple application that generates cryptographically generated sequences of numbers to be used as passwords is over 70 megabytes, and includes a complex Java runtime environment (JRE). How many bugs remain hidden in those hundreds of thousands of lines of code?

Now enter the Internet of Things, where manufacturers of devices that have not traditionally been connected to the network have not been expert at security for decades. What sort of problems lurk in each and every one of those devices?

It is simply not possible to assure perfect security, and because computers are designed by imperfect humans, all these devices are imperfect. Even devices that we believe are secure today will have vulnerabilities exposed in the future. This is one of the reasons why the network needs to play a role.

The network stands between you and attackers, even when devices have vulnerabilities. The network is best in a position to protect your devices when it knows what sort of access a device needs to operate properly. That’s your washing machine. But even for your laptop, where you might want to access whatever you want to access, whenever you want to access it, through whatever system you wish to use, informing the network makes it possible to stop all communications that you don’t want. To be sure, endpoint manufacturers should not rely solely on network protection. Devices should be built with as much protection as is practicable and affordable. The network provides an additional layer of protection.

Endpoint manufacturers thus far have not done a good job in making use of the network for protection. That requires a serious rethink, and Apple is the posture child as to why. They are the best and the brightest, and they got it wrong this time.

The Road(s) To Singapore

I travelled by air to Singapore. But what if one wanted to go from Europe by car?

Path between Wetzikon and Singapore Recently a number of us trundled off to Singapore to attend the 100th Internet Engineering Task Force meeting, during which we shared our ideas on how to improve the Internet. But precisely how did we all get there? Why, by plane of course! In the case of yours truly, I went from Switzerland by way of Bangalore, India. These are long flights: the short haul from Bangalore was four hours and twenty minutes. The non-stop return flight was just over twelve hours, thanks to favorable winds.

But what if you wanted to drive? After all, instead of flying from San Francisco to Las Vegas, I drove; and I very much enjoyed the scenery. What would it take to get all the way to Singapore by car? Is it even possible? A little check on the map shows that it should theoretically be possible to travel the distance by land, with the occasional bridge crossing here and there. How would one even begin to plan such a trip? Well, for me it would be with everyone’s favorite navigation tool: Google Maps. We start there.

Google Maps Singapore

There’s that inviting “Directions” button. If I just click it, I’m hoping that it will show me a few alternative routes, and a driving time. Of course it will indicate the tolls and the fact that we are crossing borders.

No route to host

Unfortunately, the invitation was quickly rescinded.

What’s the problem? Well, like a good computer scientist I began to bisect the route to see if I could determine where Google thought there was no route. I figured, ok let’s see if I can get to India from Switzerland. I got the same answer.

But when I asked if I could get to Lahore, things began to improve. That would be an eighty six hour route, covering 7,734 kilometers. There’s only one problem: it would take me straight through the heart of Iran, and I very much doubt I could get a transit visa for this purpose. But now at least we have a route to Lahore. A little dragging and dropping in Google Maps shows that with a mere six hour detour, one can go over the Black and Caspian seas, instead of under them, as such:

Wetzikon to Lahore, up and over

Well, very good! We’ve gotten ourselves half way there. To do so, we travel through Germany, Czechia, Poland, The Ukraine, Russia, Kazakhstan, Uzbekistan, Afghanistan, and finally into Pakistan. Right about now, Iran is beginning to sound pretty good, by the way. An airplane more so. Consider this little factoid: the route takes us through Eastern Ukraine which right now is not exactly being friendly with the rest of the country.

It turns out that one can in fact cross the Pakistan/India border with a car at Lahore if one has all the right paperwork. One enters by the city of Amristar. Now let’s see if we can get from Amristar to Singapore. Surely enough, one can!

Amristar to Singapore

That’s another 105 hours or 6,404 kilometers. One travels across India, avoiding both Bangladesh and Bhutan. While it is probably possible to drive into Bangladesh, Bhutan is virtually impossible to enter without serious amounts of paperwork. Of course, this whole trip would require serious amounts of paperwork, but Bhutan would require its own stack. One can do this because the Indian states of Assam and Manipur juts quite far to the east.

For those keeping score, this route is just under 14,000 kilometers, and would take, if driven straight, ignoring traffic (hah!), about 200 hours. That would be about 25 days, if one limits one’s driving to eight hours per day. The route changes based on which citizenship one holds, to be sure. Many countries would require visas, and car permits. One challenge that one has to consider is that this is the most direct route, according to Google Maps. That doesn’t mean it’s the easiest route. For one thing, many of the directions themselves, are written out by Google in the local script. For the Ukraine, that means Cyrillic. For Myanmar, that’s Burmese. Of course, this says nothing of the languages themselves, nor whether anyone would accept Mastercard. Hotels? There may be inns along the route. Google is pretty good at spotting these and (perhaps more importantly) gas stations.

Having performed the exercise, I think it would be fun to do parts of this route. In particular, traveling in north-western India and into Myanmar seems interesting. I wonder what Hertz would say. Apart from the collision damage waver, and all other insurance, I’m pretty sure I’d want a very simple vehicle that could be easily repaired and could handle varying qualities of gasoline. An old Range Rover with an extra tank might be a good deal. Probably not the trip to take a Tesla.

To play around with this route, have a look at the Google map. Be sure to expand out the directions. Note the occasional U-Turn one is required to make.

Some final geographical points: this trip, while long, roughly follows the great circle route, and so it’s fairly optimal, from a distance standpoint. It is also probably the farthest south one can travel from Europe or Asia without taking a ferry. Assuming one can travel it at all. With ferries, it may be possible to get as far as Timur, but I haven’t checked that.

The role of the CISO and the Equifax Breach

CISOs don’t eliminate risk- they help companies manage it. Equifax made poor choices as a company. The CISO was ineffective.

I do not know Susan Mauldin, the now-former Chief Security Officer of Equifax, nor can I even tell you what her job was. That is because the role of Chief Information Security Officer (CISO) remains ill-defined: each company implements the role in different ways and has different expectations. It may well be that this person did not have the authority to implement policies that would have prevented the breach that revealed records of over 143 million US consumers.

What I can say is this:

The only way you can entirely secure a computer is to destroy it and melt down its components beyond the point that any recovery tool can glean information. Otherwise, there is always some security risk. You might be able to sufficiently secure a system such that the risk is so low as to be almost negligible, but to do that usually requires more resources than it will cost to mitigate a breach.

The goal of a CISO is to reduce the expected loss of a security breach to a level acceptable to the management. Expected loss has many components. It can include direct financial losses, losses in sales, reputation loss (and thereby future sales losses), stolen IPR, thus impacting product differentiation, and liability associated with stolen customer and partner information. In a world where information is worth its weight in gold, holding any information secret means that there is a risk it will be revealed. The decisions of a CISO or her management do not amount to loss due to a single event, but may be recurring losses, either due to expenses to mitigate risk or due to losses from breaches.

Equifax’s business is information about consumers. That means that they must retain the information necessary to report their findings to their customers, such as banks or employers who are assessing the trustworthiness of an individual. That can be a lot of information, such as credit card, mortgage, and utility payment histories. Equifax is a big fat target for information thieves, much the same way the US Office of Personnel Management is (they were breached in 2014).

It has been reported that the information thieves in this case made use of a vulnerability in Apache Struts that had been announced in March. Equifax stated that they detected anomalous behavior on the 29th of July. That left a period of roughly four months of exposure. In the grand scheme of things, this is not a long long time for an exposure. However, because the value of information that was at risk was actually quite high, and because the vulnerability in question was exploitable on the open Internet, there should have been a process in place to rapidly close the bug. There exist any number of patch management tools that spot open source software updates, and alert the customer.

Should Susan Mauldin have known all of this? Yes. Did she? I don’t know. Did she have the authority to effect change? I don’t know, but to be sure she was ineffective because the necessary processes were not in place. Will this sort of failure happen again? You can bet on it, but when and how much the loss will be is where CISOs make their money.