Eliot Lear's Ramblings

Ain’t No Perfect. That’s why we need network protection.

If Apple can blow it, so too can the rest of us. That’s why a layered defensive approach is necessary.

When we talk about secure platforms, there is one name that has always risen to the top: Apple. Apple’s business model for iOS has been repeatedly demonstrated to provide superior security results over its competitors. In fact, Apple’s security model is so good that governments feel threatened enough by it that we have had repeated calls for some form of back door into their phones and tablets. CEO Tim Cook has repeatedly taken the stage to argue for such strong protection, and indeed I personally have friends who I know take this stuff so seriously that they lose sleep over some of the design choices that are made.

And yet this last week, we learned of a vulnerability that was as easy to exploit as to type “root” twice in order to gain privileged access.

Wait. What?

Ain’t no perfect.

If the best and the brightest of the industry can occasionally have a flub like this, what about the rest of us? I recently installed a single sign-on package from Ping Identity, a company whose job it is to provide secure access. This simple application that generates cryptographically generated sequences of numbers to be used as passwords is over 70 megabytes, and includes a complex Java runtime environment (JRE). How many bugs remain hidden in those hundreds of thousands of lines of code?

Now enter the Internet of Things, where manufacturers of devices that have not traditionally been connected to the network have not been expert at security for decades. What sort of problems lurk in each and every one of those devices?

It is simply not possible to assure perfect security, and because computers are designed by imperfect humans, all these devices are imperfect. Even devices that we believe are secure today will have vulnerabilities exposed in the future. This is one of the reasons why the network needs to play a role.

The network stands between you and attackers, even when devices have vulnerabilities. The network is best in a position to protect your devices when it knows what sort of access a device needs to operate properly. That’s your washing machine. But even for your laptop, where you might want to access whatever you want to access, whenever you want to access it, through whatever system you wish to use, informing the network makes it possible to stop all communications that you don’t want. To be sure, endpoint manufacturers should not rely solely on network protection. Devices should be built with as much protection as is practicable and affordable. The network provides an additional layer of protection.

Endpoint manufacturers thus far have not done a good job in making use of the network for protection. That requires a serious rethink, and Apple is the posture child as to why. They are the best and the brightest, and they got it wrong this time.

The Road(s) To Singapore

I travelled by air to Singapore. But what if one wanted to go from Europe by car?

Path between Wetzikon and Singapore Recently a number of us trundled off to Singapore to attend the 100th Internet Engineering Task Force meeting, during which we shared our ideas on how to improve the Internet. But precisely how did we all get there? Why, by plane of course! In the case of yours truly, I went from Switzerland by way of Bangalore, India. These are long flights: the short haul from Bangalore was four hours and twenty minutes. The non-stop return flight was just over twelve hours, thanks to favorable winds.

But what if you wanted to drive? After all, instead of flying from San Francisco to Las Vegas, I drove; and I very much enjoyed the scenery. What would it take to get all the way to Singapore by car? Is it even possible? A little check on the map shows that it should theoretically be possible to travel the distance by land, with the occasional bridge crossing here and there. How would one even begin to plan such a trip? Well, for me it would be with everyone’s favorite navigation tool: Google Maps. We start there.

Google Maps Singapore

There’s that inviting “Directions” button. If I just click it, I’m hoping that it will show me a few alternative routes, and a driving time. Of course it will indicate the tolls and the fact that we are crossing borders.

No route to host

Unfortunately, the invitation was quickly rescinded.

What’s the problem? Well, like a good computer scientist I began to bisect the route to see if I could determine where Google thought there was no route. I figured, ok let’s see if I can get to India from Switzerland. I got the same answer.

But when I asked if I could get to Lahore, things began to improve. That would be an eighty six hour route, covering 7,734 kilometers. There’s only one problem: it would take me straight through the heart of Iran, and I very much doubt I could get a transit visa for this purpose. But now at least we have a route to Lahore. A little dragging and dropping in Google Maps shows that with a mere six hour detour, one can go over the Black and Caspian seas, instead of under them, as such:

Wetzikon to Lahore, up and over

Well, very good! We’ve gotten ourselves half way there. To do so, we travel through Germany, Czechia, Poland, The Ukraine, Russia, Kazakhstan, Uzbekistan, Afghanistan, and finally into Pakistan. Right about now, Iran is beginning to sound pretty good, by the way. An airplane more so. Consider this little factoid: the route takes us through Eastern Ukraine which right now is not exactly being friendly with the rest of the country.

It turns out that one can in fact cross the Pakistan/India border with a car at Lahore if one has all the right paperwork. One enters by the city of Amristar. Now let’s see if we can get from Amristar to Singapore. Surely enough, one can!

Amristar to Singapore

That’s another 105 hours or 6,404 kilometers. One travels across India, avoiding both Bangladesh and Bhutan. While it is probably possible to drive into Bangladesh, Bhutan is virtually impossible to enter without serious amounts of paperwork. Of course, this whole trip would require serious amounts of paperwork, but Bhutan would require its own stack. One can do this because the Indian states of Assam and Manipur juts quite far to the east.

For those keeping score, this route is just under 14,000 kilometers, and would take, if driven straight, ignoring traffic (hah!), about 200 hours. That would be about 25 days, if one limits one’s driving to eight hours per day. The route changes based on which citizenship one holds, to be sure. Many countries would require visas, and car permits. One challenge that one has to consider is that this is the most direct route, according to Google Maps. That doesn’t mean it’s the easiest route. For one thing, many of the directions themselves, are written out by Google in the local script. For the Ukraine, that means Cyrillic. For Myanmar, that’s Burmese. Of course, this says nothing of the languages themselves, nor whether anyone would accept Mastercard. Hotels? There may be inns along the route. Google is pretty good at spotting these and (perhaps more importantly) gas stations.

Having performed the exercise, I think it would be fun to do parts of this route. In particular, traveling in north-western India and into Myanmar seems interesting. I wonder what Hertz would say. Apart from the collision damage waver, and all other insurance, I’m pretty sure I’d want a very simple vehicle that could be easily repaired and could handle varying qualities of gasoline. An old Range Rover with an extra tank might be a good deal. Probably not the trip to take a Tesla.

To play around with this route, have a look at the Google map. Be sure to expand out the directions. Note the occasional U-Turn one is required to make.

Some final geographical points: this trip, while long, roughly follows the great circle route, and so it’s fairly optimal, from a distance standpoint. It is also probably the farthest south one can travel from Europe or Asia without taking a ferry. Assuming one can travel it at all. With ferries, it may be possible to get as far as Timur, but I haven’t checked that.

The role of the CISO and the Equifax Breach

CISOs don’t eliminate risk- they help companies manage it. Equifax made poor choices as a company. The CISO was ineffective.

I do not know Susan Mauldin, the now-former Chief Security Officer of Equifax, nor can I even tell you what her job was. That is because the role of Chief Information Security Officer (CISO) remains ill-defined: each company implements the role in different ways and has different expectations. It may well be that this person did not have the authority to implement policies that would have prevented the breach that revealed records of over 143 million US consumers.

What I can say is this:

The only way you can entirely secure a computer is to destroy it and melt down its components beyond the point that any recovery tool can glean information. Otherwise, there is always some security risk. You might be able to sufficiently secure a system such that the risk is so low as to be almost negligible, but to do that usually requires more resources than it will cost to mitigate a breach.

The goal of a CISO is to reduce the expected loss of a security breach to a level acceptable to the management. Expected loss has many components. It can include direct financial losses, losses in sales, reputation loss (and thereby future sales losses), stolen IPR, thus impacting product differentiation, and liability associated with stolen customer and partner information. In a world where information is worth its weight in gold, holding any information secret means that there is a risk it will be revealed. The decisions of a CISO or her management do not amount to loss due to a single event, but may be recurring losses, either due to expenses to mitigate risk or due to losses from breaches.

Equifax’s business is information about consumers. That means that they must retain the information necessary to report their findings to their customers, such as banks or employers who are assessing the trustworthiness of an individual. That can be a lot of information, such as credit card, mortgage, and utility payment histories. Equifax is a big fat target for information thieves, much the same way the US Office of Personnel Management is (they were breached in 2014).

It has been reported that the information thieves in this case made use of a vulnerability in Apache Struts that had been announced in March. Equifax stated that they detected anomalous behavior on the 29th of July. That left a period of roughly four months of exposure. In the grand scheme of things, this is not a long long time for an exposure. However, because the value of information that was at risk was actually quite high, and because the vulnerability in question was exploitable on the open Internet, there should have been a process in place to rapidly close the bug. There exist any number of patch management tools that spot open source software updates, and alert the customer.

Should Susan Mauldin have known all of this? Yes. Did she? I don’t know. Did she have the authority to effect change? I don’t know, but to be sure she was ineffective because the necessary processes were not in place. Will this sort of failure happen again? You can bet on it, but when and how much the loss will be is where CISOs make their money.

Secret sauce and sentencing? Say it isn’t so!

Could you spend a long time in prison due to a software bug and not have the right to examine the software? Possibly.

One of the things that we in technology understand is that we make mistakes, a truth we don’t like to admit to customers. What happens, however, when a mistake can lead to tragic consequences?

Yesterday’s New York Times reports about a case that the U.S. Supreme Court may soon hear, involving a man who received a six year jail sentence, in part due to a computer program. The software known as Compas was supposedly developed by Northpointe Inc. (although a search seems to redirect to a Equivant) to provide a risk assessment of a person’s reentry into society. Such a data-driven analysis is vaguely reminiscent of the movie, Minority Report. In this case, the defendant Eric L. Loomis was not allowed to examine the software that assessed that he was a significant risk to the community, even though at least one analysis showed that the software may be programmed with some form of racial bias. The company argues that the algorithm used to make the sentencing recommendation is proprietary, and so should not be subject to review, and that if they release their algorithm to scrutiny they will essentially be giving away their business model, and they may have a point. Patents on such technology may be flimsy, and they eventually do come to a halt. To protect themselves, they make use of another legal tool, the trade secret, which has no fixed term of protection.

One can’t say that a mistake is being made in the case of Mr. Loomis, nor can one authoritatively state that the program is formally correct. The Wisconsin Supreme Court argued creatively that much like college admissions, so long as the software is one input combined with others, the software can be used. Is it, therefore, any different from a potentially flawed witness giving evidence? The question here is whether those who wrote the software can be cross-examined, to what extent they may be questioned, and whether the software itself can be examined. Mr. Loomis argues that to deny his legal team access to the source is a violation of his 14^th Amendment right to due process.

We know from recent experience that blind trust in technology, and more precisely, those who create and maintain it, can lead to bad outcomes. Take for instance the over 20,000 people whose convictions were overturned because a chemist falsified hair analysis results, or other examples where the FBI Crime Lab just flat got it wrong. Even Brad D. Schimel, the Wisconsin attorney general, conceded before the appeals court that, “The use of risk assessments by sentencing courts is a novel issue, which needs time for further percolation.” But what about Mr. Loomis and those who may suffer tainted results if there is a software problem?

While the Supreme Court could rule soon on the matter, they will only have very limited avenues, such as permitting or prohibiting its use. Congress may need to get involved in order to provide other alternatives. One possibility would be to provide the company some new intellectual property protection, such as an extended patent with additional means of enforcement (e.g., higher penalties against infringement or lower thresholds for discovery) in exchange for releasing the source. Even if they do, one question would be whether or not defendants could then game the system so as to score better on sentencing. How great a risk that is we can’t know without knowing what the inputs to the algorithm are.

It is probably not sufficient for the defendant and his legal teams to have access to the source, precisely because more research is needed in this field to validate the models that software like Compas uses. That can’t happen unless researchers have that access.

Addressing the Department Gap in IoT Security

People in departments outside of IT aren’t paid to understand IT security. In the world of IoT, we need to make it easy for those people to do the right thing.

So, Mr. IT professional, you suffer from your colleagues at work connecting all sorts of crap to your network that you’ve never heard of? You’re not alone. As more and more devices hit the network, the ability to maintain control can often prove challenging. Here are your choices for dealing with miscreant devices:

Prohibit them and enforce the prohibition by firing anyone who attaches an unauthorized device.
Allow them and suffer.
Prohibit them but not enforce the prohibition.
Provide an onboarding and approval process.

A bunch of companies I work with generally aim for 1 and end up with 3. A bunch of administrators recognize the situation and fit into 2. Everyone I talk to wants to find a way to scale 4, but nobody has, as of yet. What does 4 involve? Today, it means an IT person researching a given device, determining what networking requirements it has, creating firewall rules, and some associated policies, and establishing an approval mechanism for a device to connect.

This problem is exacerbated by the fact that many different enterprise departments have wide and varied needs, and the network stands as critical to many of them. Furthermore, very few of those departments reports through the chief information officer, and chief information security officers often lack the attention their concerns receive.

I would claim that the problem is that incentives are not well aligned, were people in other departments even aware of the IT person’s concerns in the first place, and often they are not. The person responsible for providing vending machines just wants to get the vending machines hooked up, while the person in charge of facilities just wants the lights to come on and the temperature to be correct.

What we know from hard experience is that the best way to address this sort of misalignment is to make it easy for everyone to do the right thing. What, then, is the right thing?

Prerequisites

It has been important pretty much forever for enterprises to be able to maintain an inventory of devices that connect to their networks. This can be tied into the DHCP infrastructure or to the device authentication infrastructure. Many such systems exist, the simplest of which is Active Directory. Some are passive and snoop the network. The key point is simply this: you can’t authorize a system if you can’t remember it. In order to remember it, the device itself needs to have some sort of unique identifier. In the simplest case, this is a MAC address.

Ask device manufacturers to help

Manufacturers need to make your life easier by providing you a description what the device’s communication requirements are. The best way to do this is with Manufacturer Usage Descriptions (MUD). When MUD is used, your network management system can retrieve a recommendation from the manufacturer, and then you can approve, modify, or refuse a policy. By doing this, you don’t have to go searching all over random web sites.

Have a simple and accessible user interface for people to use

Once in place you now have a nice system that encourages the right thing to happen, without other departments having to do anything other than to identify the devices they want to connect. That could be as simple as a picture of a QR code or otherwise entering a serial #. The easier we can make it for people who know nothing about networking, the better all our lives will be.