Can the Internet Get “Walled”?

What’s the Suez Canal of the Internet?

The Ever Given blocking the Suez Canal — Ever Given

Over the last few days we bore witness to a minor economic disaster, thanks to the Ever Given having firmly planted itself into both walls of the Suez Canal. The Financial Times gives a very good overview of the factors that to this mishap. In that article, Brendan Greeley describes how the Ever Given got “walled” more so than just grounded, because it implanted itself into the canal walls.

For those of us whose life is about providing resilient services, one has to ask: where was the failure? Mr. Greeley goes into some depth about how the sheer height (beam), weight, and width of the ship, the shape of the canal, the water forces and wind all contributed to this mishap. He also pointed out that the economics favor larger vessels. This is an externality- there is no chance that the owners will ever pay for the amount of damage the blocked canal has caused, which is estimated to have been up to $10 billion. Syria was reportedly rationing fuel because of the blockage, and fuel prices across the globe ticked up. Several ships rerouted to go around the horn of Africa, risking hijackings.

The other far bigger failure here is that there is but one canal through which upon which large portions of the world economy depends. One big anything doesn’t make for good resilience. That canal could fail again. Knowing this, Iran has offered to create an alternate shipping lane, adding at least a bit of redundancy into the system. Ultimately, manufacturers throughout the supply chain can re-evaluate how to manage this sort of delivery delay. Should new lanes be formed? Should more production be closer to the end consumer? A new canal would surely cost tens of billions of dollars, and may offer only limited resilience. After all, why wouldn’t the same failure happen in both canals? In all likelihood it won’t be this precise “walling”, the hope being that canal operators and pilots will update their procedures to limit the risk.

We Internet geeks understand this class of problem in great detail, in many dimensions. A major benefit of cloud computing is to spread load across multiple CPUs in multiple locations, so that no single failure would cause disruption.

Taken individually and impacting individual customers, it’s a sure bet that cloud services are far more reliable than people rolling their own, just as it is safer to use a container vessel than trying to carry one’s products across in a dingy. However, the flip side of that coin is the impact those services have when they fail. Some examples:

When	What	Impact
2016	Mirai BOTNET / DYN attack	Twitter, other services out for a day
2020	GMail, YouTube, Google Docs	Services disrupted for an hour
2020	Amazon Web Services East Coast Data Center	Large numbers of application services failed
2020	Cloudflare DNS outage	Client resolvers failed for 27 minutes
2021	Microsoft Teams and Office 365	Services to their customers unavailable for four hours

Can an Internet-wide failure happen? Where’s that “Internet canal” bottleneck? I wrote about that for Cisco not long ago. It could very well be cloud-based DNS resolvers, such as Cloudflare’s 1.1.1.1. What we know is that these services can fail because they have done so in the past. Last year, MIT sage Dan Geer looked at market concentration effects on cybersecurity risk, which opens up a bigger question. This time, The Ever Given failed without any malice. Geer’s major point is that there is an asymmetric attack on large targets, like popular cloud services. The same perhaps can be said about the Suez Canal.

Note that large cloud services are not the only aggregate risk we face. Geer’s earlier work looked at software monocultures. When a large number of systems all use the same software, a single attack can affect all, or at least a great many, of them. This is just another example of a Suez Canal.

The economic drivers are always toward economies of scale, whether that’s a large cloud service or a single supplier, but at the often hidden price of aggregate resiliency. The cost generally amounts to an externality because of the size and scope of the service as well as the impact of an outage on others are not understood until an event happens. Having not considered it a week ago, some producers are considering this question today.

Courtesy of Copernicus Sentinel data 2021, https://commons.wikimedia.org/w/index.php?curid=102251045

The Challenges of CISOs

Are CISOs investing enough in protection? Do they have good visibility to threats?

Long ago there used to be a bar on Haight St. called Aub Persian Zam Zam, run by a cranky guy named Bruno. Bruno who hated everyone, and he preferred only to serve martinis. If you walked in before 7:00pm, he told you that table service started at 8:00pm. And if you walked in after 7:00pm, table service stopped at 6:00pm. As a customer, I felt a little like a Chief Information Security Officer (CISO).

CISOs constantly face a challenge with their boards: how much to invest in security. If you haven’t been hacked, then you are accused of spending too much on protection (and might be out of a job); and if you have, then you spent too little (and might be out of a job). But CISOs have to operate in the here and now. They don’t get to have the luxury of hindsight. What CISOs need is an appropriate level of investment to secure their charges and situational awareness to make good decisions.

Much is being made of the lax security that Solar Winds had. As Bruce Schneier pointed out in the New York Times, they had been hacked not just once, but several times. There was the attack on the company and then there was the attack on their customers. The attack on the customers involved the use of a DNS-based command and control (C&C) network, very stealthily crafted code, and the potential for an infected system to probe whatever was available to it at government and industrial installations across the globe. This may have been particularly damaging in the case of Solar Winds because the legitimate software could have stood in a privileged point within an enterprise, requiring access to lots of other core infrastructure. The Russians picked a really juicy target. They were, if you will, an incident waiting to happen, and happen it did. Solar Winds was detectable, but it required an appropriate investment in not only tooling but back-end expert services to provide situational awareness.

Not every target is quite so juicy. Most hackers hit web servers or laptops with various viruses. The soft underbelly of cybersecurity, however, are the control systems, who themselves have access to other infrastructure, as was demonstrated this past month, when a hacker attempted to poison a Florida city with lye. Assuming they have one, the Oldsmar CISO might have some explaining to do. How might that person do so, especially when it is the very system meant to protect the others? It starts by knowing how one compares to one’s peers in terms of expenditures. It’s possible to both under- and overspend.

Optimal investment models for cybersecurity has been an ongoing area of research. The seminal Gordon-Loeb Model demonstrates a point of optimality and a point of diminishing returns for risk mitigation. The model doesn’t given you the shape of either curves. That was the next area of research.

For one, some things are easy to do, and some are hard; but the easy things are often not the right things to do. Low level cybersecurity professionals sometimes make the wrong choices, being risk seeking for big ticket items like device policy management, two-factor authentication, training, and auditing; while being risk adverse to matters that are within their control. Back in 2015, Armin Sarabi, Parinaz Naghizadeh, Yang Liu, and Mingyan Liu set out to answer this question. The table below liberally borrowed from their paper shows a risk analysis of different sectors.

Sarabi et al, *Prioritizing Security Spending: A Quantitative Analysis of Risk Distributions for Different Business Profiles*, Workshop on the Economics of Information Security, 2015.

What this says is that based on reports received, configuration errors were a substantial risk factor pretty much everywhere but accommodation and food services, but they suffered because employees share credentials. It was a limited survey, and surely the model has changed since then. In the intervening time, cloud computing has become far more prevalent, and we have seen numerous state actors take on a much bigger, and nastier, role. It’s useful, however, is for a CISO to have situational awareness of what sorts of common risks are being encountered, and to have some notion as to what best practices are to counter those risks, so that whatever a firm spends is effective.

Expenditures alone don’t guarantee against break-ins. Knowing one’s suppliers and their practices is also critical. Knowing that Verkada had sloppy practices would have both deterred some from using their cameras, and in turn encouraged that provider to clean up their act. Again, situational awareness matters.

Gordon Loeb Diagram by By Luca Rainieri – Own work, CC BY-SA 4.0

Pas Parler?

Will the real Internet government please stand up?

This weekend, Google, Apple, and Amazon all took steps to remove the right wing conspiracy web site Parler from their services, steps that will cripple the social media site for some some period of time. In many ways, Parler had it coming to them. Amazon in particular alleged that Parler refused to take prompt action to remove abusive content that violated their terms of service.

In response, my right wing friends have gone nearly indiscriminately crazy, complaining that their 1st Amendment rights have been violated. Let’s review that amendment of the U.S. Constitution:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.
Amendment I of the U.S. Constitution.

In other words, Congress cannot stop someone from speaking. But these companies are not Congress, nor an arm of the U.S. government. We could, however, say that they are a form of government, in as much as these companies, along with a small number of other ones, such as TikTok control societal discourse. What rules would govern them if they decided that moveon.org was also not to their liking? Could these services exclude content that criticizes them?

Parler is a relative newcomer. Much in the same way that Fox News has lost its conservative gleam to NewsMax, Facebook and Twitter lost their gleam when they started applying editorial control to posts. They did this because they gauged societal harm against whatever short term revenue they were collecting from the likes of Donald Trump. There was seemingly no reason they had to, at least in the United States. U.S. Law says this:

No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.
47 USC § 230

Meddle with this rule at your peril. If we shift the burden of policing to online services, social media sites as we know them will cease to be, GMail and Yahoo! mail would be imperiled, and Amazon could no longer offer customer reviews. If there is a middle ground to be found, then scale factors must be considered. Any middle ground may well increase the risks of starting up new services. If the price of entry for a new Facebook or Twitter competitor is fancy artificial intelligence systems and patents, then we may have done ourselves no service in the long run.

There are other consequences to Apple and Google removing Parler from their respective phone and tablet stores: I saw one conversation in which someone was describing to her friends how to turn off automatic software updates. Software updates are the means by which developers correct vulnerabilities they have created. By disabling those updates, people leave themselves vulnerable to attack.

Today Parler is losing its voice, arguably for very deserved reasons. Tomorrow, some other site might lose its access. Will those reasons be just as good and who will decide?

Enabling the Crazy: Republicans and the right wing need to be held accountable for their dishonesty

Until Republicans police their own and use honest arguments, they cannot be trusted.

Over the last forty years, Republicans have developed some very bad habits. False equivalencies, out and out lying (such as claiming that Saddam had weapons of mass destruction), hypocritical stances (such as denying Merrick Garland a hearing for a position on the Supreme Court), accepting of lies for political expediency, and accusing the opposition of the very sins they commit are only some of the tactics that Republican base has grown to accept through networks such as Fox News and numerous zany online hysteria sites.

After a Berkman study found study (among many others) demonstrated that right wing media does not hold itself to account, and that Facebook is far more right wing than Twitter, Facebook began to take steps to address false reports. This led to criticism of social networks by President Trump, and a business opportunity for other purveyors of falsehoods. That is because Republicans can no longer survive without the falsehoods. A few people have noticed this danger, one of them being the founder of the founding online editor of the Washington Examiner and NewsBusters, Matthew Sheffield. In his own words:

I eventually realized that most people who run right-dominated media outlets see it as their DUTY to be unfair and to favor Republicans because doing so would some how counteract perceived liberal bias.
While I was enmeshed in the conservative media tradition, I viewed lefty media thinkers like @jayrosen_nyu as arguing that journalism was supposed to be liberally biased. I was wrong. I realized later that I didn’t understand that journalism is supposed to portray reality.

Mr. Sheffield wasn’t the only one to notice this problem. Last week Charles Koch put it quite simply: “Boy, did we screw up!” He did so by funding the Tea Party that fed on propaganda, not journalism.

The news echo chamber in which right wing people live live is not a threat to democracy, but is a threat to good, sound government, and to the health and welfare of the public. Donald Trump’s lies about the Corona Virus are merely the logical conclusion of a concerted attack that began with Rush Limbaugh in the 1980s, intensified with Fox News, and has led to such hate sites as InfoWars. Although Senator McConnell is smart enough not to peddle conspiracies, he has no problem profiting from them. He therefore does nothing to correct them, putting party before country. And so what do we get? Failure to recognize the risks of climate change, improper preparation for what became a pandemic that has led to economic destruction and chaos, and breeding of bigotry and violence.

Moral equivalences to justify some of Donald Trump’s more egregious acts seem to be a favorite. Previously we heard how, because cages were built under the Obama administration to house children for periods of up to 72 hours when they had no other place to go, it was therefore okay for the Trump administration to target migrant families for separation.

The latest claim from my friends on the right, led by people such as Representative Steve Scalise, has been that Donald Trump has the same right to use the courts to make whatever challenges he will, just as Al Gore did when he lost to George Bush in 2001. It is true that Al Gore did use the courts to attempt require a recount in certain districts. In that case, Gore and his legal team made very specific claims, and had evidence to advance them. They were these:

(1) The rejection of 215 net votes for Gore identified in a manual count by the Palm Beach Canvassing Board as reflecting the clear intent of the voters;
(2) The rejection of 168 net votes for Gore, identified in the partial recount by the Miami-Dade County Canvassing Board.
(3) The receipt and certification after Thanksgiving of the election night returns from Nassau County, instead of the statutorily mandated machine recount tabulation, in violation of section 102.14, Florida Statutes, resulting in an additional 51 net votes for Bush.
(4) The rejection of an additional 3300 votes in Palm Beach County, most of which Democrat observers identified as votes for Gore but which were not included in the Canvassing Board’s certified results; and
(5) The refusal to review approximately 9000 Miami-Dade ballots, which the counting machine registered as non-votes and which have never been manually reviewed.
Gore v. Harris, Florida Supreme Court, SC00-2431, December, 2000.

Each of these claims were specific, backed up by facts that were not in dispute, and raised substantial questions of procedure, as demonstrated by the ruling of the Florida Supreme Court. The point here is not to re-litigate an election that took place 20 years ago, but rather to highlight how Mr. Gore used the courts in good faith, even if he lost in the end.

In this case, the Trump campaign has put forward wild claims that have not been backed by evidence or law. While President Trump and his cronies have whined about fraud to the press and to the American people, when they have gone to court, their tune has been very different.

In Pennsylvania, Arizona, Michigan, and Nevada, when asked if they were arguing that fraud had occurred, the answer was “No”. In Michigan or Wisconsin, where the best they could come up with were what they called statistical irregularities. In 2016, those same “irregularities” occurred in districts Trump won, and yet there was no complaint at the time. That’s because there was no actual evidence or testimony of any wrong doing. Mr. Trump is a sore loser. He has thrown spaghetti at a wall to see what would stick. He and his clown, Rudolph Giuliani, have wasted the time of the courts, and have attempted to undermine the very institutions he swore to uphold. In computing we would call this a Denial of Service (DoS) attack, in which the resources of courts and opponents are wasted by suits filed in bad faith. In Williamsport, Pennsylvania, US District Judge Matthew W. Brann, a lifelong Republican, castigated Mr. Giuliani for his buffoonish arguments, writing:

… this Court has been presented with strained legal arguments without merit and speculative accusations, unpled in the operative complaint and unsupported by evidence.

In Georgia, Trump and his fellow loons have attacked the Republican Secretary of State for having followed the law and performed his duties. One of his loons, Russell J. Ramsland Jr., filed an affidavit that raised concerns about alleged incidents that didn’t even take place in the state. They have attacked various government officials in Nevada as well.

In the meantime, Republican senators and governors largely sat silent, allowing this travesty to continue. With one exception, the only people who dared to stand up to Mr. Trump were those who were retiring, or those who had just been reelected. That required no courage or leadership whatsoever. These people have been swayed by Trump’s media madness, worrying that they might not get re-elected, or thinking they might develop a political advantage when Biden is sworn in. Winning, not their oaths to their country, is all that matters to them.

Some will say that Democrats are no better, that Democrats obstructed Mr. Trump from the moment he came into office. Maybe there is some truth to this, but at least in the Democrat’s case, they used legal means and the stark truth to support their positions. Not so with the Right Republicans in Congress, and enablers such as Senator Collins.

There can be no reconciliation, nor can their be trust in their role in government, until Republicans police their own, starting with taking responsibility for not having stood up to a liar and bully, who harmed a great many people.

How to repair the damage done to the Supreme Court

For years, a primary goal of Republicans has been to reshape the judiciary. They have done so using every tool at their disposal, but one of those tools was not honesty. Not even considering President Obama’s nominee Merrick Garland on the basis that such nominations aren’t entertained during a presidential election year, and then ignoring that logic to confirm Amy Coney Barrett as a justice was a demonstration of both hypocrisy and Might Makes Right. With Republicans having precipitated this crisis; should the Democrats take control of the presidency, the House, and the Senate; as it appears they will; they would be perfectly justified in making use of that might to correct such an abuse.

What could they do? At a bare minimum, since Merrick Garland isn’t on the court and Neil Gorsuch is, the Congress could pass a law, increasing the size of the court to eleven. President Biden could then appoint two more people to re-establish a balance.

Others have argued for even more radical changes, including among others term limits and selection criteria by party; all of which are likely to be constitutionally problematic. Whatever power the Democrats will have next year, they will almost certainly not have the power to enact amendments. Indeed even to go so far as to increase the size of the court they will almost assuredly have to do away with the filibuster rule in the Senate.

Stepping Away From the Brink

If the Democrats take any of these actions, they will be perpetuating the use of the Supreme Court as an ideological football. There is one group of people who can stop this from happening: the court itself, specifically conservative justices. This can happen in one of a few ways, but the most obvious one would be for one or more conservative justices to retire. At a spry 72, Clarence Thomas may feel that his best years are in front of him, and that the Democrats wouldn’t dare tinker with the court’s composition. Justice Alito is 70, and may feel the same way. They should think again. While Joe Biden has indicated that he doesn’t want to get into such structural changes, he hasn’t rule them out.

Another alternative would be a clear pledge from conservative justices to maintain the status quo of their own accord. This would be a bitter pill to swallow, because it requires that one sublimate deeply held principles for the good of the institution. Indeed, one might ask, if one were to do this, why not retire from the court?

Even one conservative stepping down would be patriotic. It would allow the Senate to re-establish a comity that has been absent under Mitch McConnell, allowing things to get back to normal. It would also allow some rational discussion of what if any court reform would be necessary, such that it could take place in a bipartisan spirit. Whoever does this would be establishing a legacy that would likely far outlast any decision on the court.