I do not blog as much as I used to, but I found something disturbing in a Washington Post article that was asking whether Kamala Harris will have policies friendly to Silicon Valley.
I have two problems with this article:
It would lead the reader to think the Biden administration has had no tech policy. That’s not true.
It seems to imply that the tech companies have done their part in helping society. They have not.
Let’s start with the first.
The administration has pushed hard in an area no other has: cybersecurity. President Biden issued executive orders to improve the posture of IoT devices and software, by requiring transparency of vulnerabilities for all products sold to the federal government. DHS, NIST, FCC, and other branches of the government have worked hard throughout the last three years to get our infrastructure on a better footing. 1990s gave us Section 230. The 2000s gave us an Internet governance model that the Obama administration fought to retain. The Biden administration is fighting for the safety of our infrastructure. And that is something that all sectors of society, especially the tech industry, need, as they can’t sell products that are viewed as unsafe.
This brings me to my second point. The question should not be whether Harris has the Valley’s back, but whether Tech has our backs. Social media companies have been all but mute about adversaries like Russia, China, Iran, and North Korea taking advantage of offerings to harm our interests. They’ve profited over strife and discord in a way never seen, some of which has been sown by those same countries. At the same time, we all now heavily rely on the Internet infrastructure, as Crowdstrike painfully demonstrated. And yet the Valley is still playing the same hand they had in the 2000s: “We got this.”
The game has changed. The only real question is whether tech companies will accept that and work with the next administration and Congress to find new and innovative approaches that work for society.
Here’s a tool to create standard SBOMs from Ubuntu distribution information.
As I’ve previously mentioned on this blog, software bills of materials (SBOMs) are software ingredient lists similar to what you would find on a can of soup. The purpose of these lists is to determine if something bad is in the mix, so that administrators can figure out where their risks are. This is why President Biden’s Executive Order from last May specifically called them out.
Here now is a tool that I’ve just posted to pypi called apt2sbom. This tool is specific to Ubuntu. Similar tools can be built for other distributions. This tool will take the already existing information on a Ubuntu system and collect it into one of the standard formats, such as SPDX or CycloneDX.
% pip3 install apt2sbom
[...]
% apt2sbom -h
usage: apt2sbom [-h] (-j | -y | -c) [-p]
generate SPDX file from APT inventory
optional arguments:
-h, --help show this help message and exit
-j, --json Generate JSON SPDX output
-y, --yaml Generate YAML SPDX output
-c, --cyclonedx Generate CycloneDX JSON output
-p, --pip Include PIP files
The resulting file is then suitable for import into tooling that can spot vulnerabilities in particular versions of software.
The package is a little on the early side. There might still be a few bugs here or there. If you find one, just post it to the source repository as an issue.
Would this be considered a complete SBOM? Probably not, because there may be software installed on a system that is not part of either the apt or python distributions. However, it’s fairly easy to add additional elements into these files, particular the JSON ones.
Think you can keep everything up to date with the latest security fixes? Think again.
One common piece of advice you will hear from cybersecurity experts is that you should always keep your software up-to-date, so that vulnerabilities can be corrected. We like to believe that consumers are the biggest offenders in terms of keeping old software around. After all, Grandpa doesn’t always know how to upgrade his iPhone.
This isn’t so easy for professionals. Let’s take a web site. Your average web site is composed of numerous servers, each running all manner of code, from the operating system to standard applications like Apache to web support code such as Python, PHP, and back end services, like MongoDB. ALL of it has to be updated.
What does the inventory look like? This has been the work of a group of people who have created something known as software bills of materials or SBOMs. One could think of an SBOM as the ingredient list on the back of a can of soup, only there’s software inside.
So how does one generate the inventory? And this is where things can get a bit interesting. On any reasonable UNIX machine there exists a package manager. In the case of Ubuntu, that would be apt, which is built on top of Debian’s dpkg. One can simply type “apt list –installed” to get the installed packages on a system, right?
WRONG.
Of course that will get you software that apt has installed, but if your site runs with python, then you might need to get a software list by using Python’s package manager, pip. In fact, “pip freeze” provides this information, and that will get you most, but likely not all, python packages. Now repeat for node.js’s npm, and others (assuming you can find all of them).
Now you might want to update those packages. That was the problem statement, after all. This should be simple enough, right? Well, let’s start with apt. It generally is simple enough, at least to start with: one simply runs, “apt upgrade -y” and software package upgrades happen. Of course, you have to test all of your code after this, as apt won’t do that for you. Your CI/CD process is hopefully doing this for you.
Next we go to Python. pip’s upgrade command requires a list of packages. Conveniently, we have one that we froze from above. However, it turns out that not all python packages are managed by pip; and it knows this, and pip will happily blow chunks when you try to upgrade a package it does own that requires an upgrade for something it doesn’t own. In the Ubuntu world, there are a few good examples of this, Cairo and MongoDB to name a few. Instead, these packages are managed by apt. Well that’s all good, since we can use the apt upgrade process, right?
WRONG.
Often times these packages are installed have or meet dependencies of their won that might go unmet as part of an upgrade.
Alpine Linux has applied an interesting solution to all of this: provide common python packages in their own ‘apk’ package manager. By using apk, you assuredly will not get the latest version of a package, but you will get consistent upgrade behavior. But this presents its own set of problems:
What you are looking at is a publication hierarchy. Now we have software in multiple distributions, deriving from a repository, probably from different branches. It’s possible that there are even multiple repositories for the same code. If we are lucky, the repo owner is doing the integration to push updates to Ubuntu, Pypi, and Alpine. But we are almost never going to be that lucky, if for no other reason, a lack of authorization for all of them. And so we’ll end up with a mix of push and pull. It’s the pull that causes the version skew, and this is just a single distribution.
Now, I’ve picked on Python in this post, but pip and pypi actually do a pretty reasonable job of managing what packages it installed, so long as there aren’t dependencies to stuff it didn’t install.
So what does this mean in terms of managing your product or package?
I’m not ready to make any sweeping recommendations about what to do. One option is to just deal, and upgrade when you can. The problem is that your velocity will be slower than if you pull all source from their repos directly. On the other hand, doing the latter is a whole lot of work, requiring a whole lot of expertise when builds blow up.
This is a good example of where OSS needs to mature a bit more.
Chicken Noodle Soup by By Willis Lam – Campbell’s Chicken Noodle Soup, CC BY-SA 2.0.
Private companies are making public policy, with no societal agreement on what powers governments should and should not have to address cybercrime.
A few of us have been having a rather public discussion about who should be policing the Internet and how. This began with someone saying that he had a good conversation with a mature law enforcement official who was not himself troubled by data encryption in the context of Child Sexual Abuse Material (CSAM) on the Internet.
I have no doubt about the professionalism of the officer or his colleagues. It is dogma in our community that child online protection is a crutch upon which policy makers and senior members of the law enforcement agencies rest, and we certainly have seen grandstanding by those who say, “protect the children”. But that doesn’t mean there isn’t a problem.
Perhaps in that same time frame you may have seen this report by Michael Keller and Gabriel Dance in the New York Times. That would be 45 million images, 12 million reports of which were at the time passing through FB messenger. Those were the numbers in 2019, and they were exploding then. In some cases these images were hiding in plain sight. Is 45 million a large number? Who gets to say?
Law enforcement will use the tools they have.
We have also seen people object to June’s massive sting operation that led to the bust of hundreds of people, disrupting a drug gang network. At the same time, leading legal scholars have highlighted that the sixth amendment of the US Constitution (amongst others) has been gutted with regard to electronic evidence, because the courts in America have said that private entities cannot be compelled to produce their source or methods, even when those entities are used by law enforcement. In one case, a conviction stood, even though the police contracted the software and then couldn’t produce it.
By my score, then, many don’t like the tools law enforcement doesn’t have, and many don’t like the tools law enforcement does have. Seems like the basis for a healthy dialog.
Friend and colleague John Levine pointed out that people aren’t having dialog but are talking past each other, and concluding the other side is being unreasonable because of “some fundamental incompatible assumptions”. You can read his entire commentary here.
I agree, and it may well be due to some fundamental incompatible assumptions, as John described. I have said in the past that engineers make lousy politicians and politicians make lousy engineers. Put in a less pejorative form, the generalization of that statement is that people are expert in their own disciplines, and inexpert elsewhere. We have seen politicians playing the role of doctors too, and they don’t do a good job there either; but the US is in a mess because most doctors aren’t political animals. And don’t get me started on engineers, given the recent string of legislation around encryption in places like Australia and the UK.
John added:
It’s not like we haven’t tried to explain this, but the people who believe in the wiretap model believe in it very strongly, leading them to tell us to nerd harder until we make it work their way, which of course we cannot.
This relates to a concern that I have heard, that some politicians want the issue and not the solution. That may well be true. But in the meantime, FaceBook and Google have indeed found ways to reduce CSAM on their platforms; and it seems to me that Apple has come up with an innovative approach to do the same, while still encrypting communications and data at rest. They have all “nerded harder”, trying to strike a balance between the individual’s privacy and other hazards such as CSAM (amongst other problems). Good for them!
Is there a risk with the Apple approach? Potentially, but it is not as John described, that we are one disaffected clerk away from catastrophe. What I think we heard from at least some corners wasn’t that, but rather a slippery slope argument in which Apple’s willingness to prevent CSAM might be exploited to limit political speech; and (2) that the approach will be gotten around through double encryption.
I have some sympathy for both arguments, but even if we add the catastrophe theory back into the mix, the fundamental question I asked some time ago remains: who gets to judge all of these risks and decide? The tech companies? A government? Multiple governments? Citizens? Consumers?
The other question is whether some standard (a’la the 6th Amendment) should be in play prior to anyone giving up any information. To that I would only say that government exists as a compact, and that foundational documents such as the Constitution must serve the practical needs of society, and that includes both law enforcement and preventing governmental abuse. If the compact of the 18th century can’t be held, what does a compact of the 21st century look like?
Yet more research and yet more dialogue is required.
Encryption makes the Internet possible, but there are some controversial and other downright stupid uses for which we all pay.
Imagine someone creating or supporting a technology that consumes vast amounts of energy only to produce nothing of intrinsic value and being proud of that of that fact. Such is the mentality of Bitcoin supporters. As the Financial Times reported several days ago, Bitcoin mining, the process by which this electronic fools’ gold is “discovered”, takes up as much power as a small country. And for what?
Cambridge University Bitcoin Electricity Consumption Index
The euro, yen, and dollar are all tied to the fortunes and monetary policies of societies as represented by various governments. Those currencies are all governed by rules of their societies. Bitcoin is an attempt to strip away those controls. Some simply see cryptocurrencies as a means to disrupt the existing banking system, in order to nab a bit of the financial sector’s revenue. If so, right now they’re not succeeding.
In fact nothing about cryptocurrency is succeeding, while people waste a tremendous amount of resources. Bitcoin has been an empty speculative commodity and a vehicle for criminals to receive ransoms and other fees, as happened recently when the Colonial Pipeline paid a massive $4.4 million to DarkSide, a gang of cyber criminals.
What makes this currency attractive to hackers is that otherwise intelligent people purchase and promote the pseudo-currency. Elon Musk’s abrupt entrance and exit (that some might call Pump and Dump), demonstrates how fleeting that value may be.
Bitcoin is nothing more than an expression of what some would call crypto-governance, a belief that somehow technology is above it all and somehow is its own intrinsic benefit to some vague society. I call it cryptophilia: an unnatural and irrational love of all things cryptography, in an attempt to defend against some government, somewhere.
Cryptography As a Societal Benefit
Let’s be clear: Without encryption there could be no Internet. That’s because it would simply be too easy for criminals to steal information. And as is discussed below, we have no shortage of criminals. Today, thanks to efforts by people like letencrypt.org, the majority of traffic on the Internet is encrypted, and by and large this is a good thing.
This journey took decades, and it is by no means complete.
Some see encryption as a means by those in societies who lack basic freedoms as a means to express themselves. The argument goes that in free societies, governments are not meant to police our speech or our associations, and so they should have no problem with the fact that we choose to do so out of their ear shot, the implication being that governments themselves are the greatest threat to people.
Distilling Harm and Benefit
Bitcoin is an egregious example of how this can go very wrong. A more complicated case to study is the Tor network, which obscures endpoints through a mechanism known as onion routing. The proponents of Tor claim that it protects privacy and enables human rights. Critics find that Tor is used for illicit activity. Both may be right.
Back in 2016, Matthew Prince, the CEO of Cloudflare reported that, “Based on data across the CloudFlare network, 94% of requests that we see across the Tor network are per se malicious.” He went on to highly that a large portion of spam originated in some way from the Tor network.
One recent study by Eric Jardine and colleagues has shown that some 6.7% of all ToR requests are likely malicious activity. The study also asserts that so-called “free” countries are bearing the brunt of the cost of Tor, both in terms of infrastructure and crime. The Center for Strategic Studies quantifies the cost at $945 billion, annually, with the losses having accelerated by 50% over two years. The Tor network is key enabling technology for the criminals who are driving those costs, as the Colonial Pipeline attack so dramatically demonstrated.
Torflow visualization of the Tor network (2016)
Each dot on the diagram above demonstrates a waste of resources, as packets make traversals to mask their source. Each packet may be routed and rerouted numerous times. What’s interesting to note is how dark Asia, Africa, and South America were.
While things have improved somewhat since 2016, bandwidth in many of these regions still comes at a premium. This is consistent with Jardine’s study. Miscreants such as DarkSide are in those dots, but so too are those who are seeking anonymity for what you might think are legitimate reasons.
One might think that individuals have not been prosecuted for using encrypted technologies, but governments have been successful in infiltrating some parts of the so-called dark web. A recent takedown of a child porn ring followed a large drug bust last year by breaking into Tor network sites is enlightening. First, one wonders how many other criminal enterprises haven’t been discovered. As important, if governments we like can do this, so can others. The European Commission recently funded several rounds of research into distributed trust models. Governance was barely a topic.
Other Forms of Cryptophilia: Oblivious HTTP
A new proposal known as Oblivious HTTP has appeared at the IETF that would have proxies forward encrypted requests to web servers, with the idea of obscuring traceable information about the requestor.
Oblivious HTTP, from draft-thomson-http-oblivious-01
This will work with simple requests a’la DNS over HTTP, but as the authors note, there are several challenges. The first is that HTTP header information, which would be lost as part of this transaction, actually facilitates the smooth use o the web. This is particularly true with those evil cookies about which we hear so much. Thus any sort of session information would have to be re-created in the encrypted web content, or worse, in the URL itself.
Next, there is a key discovery problem: if one is encrypting end-to-end, one needs to have the correct key for the other end. If one allows for the possibility of receiving such information using non-oblivious methods to the desired web site, then it is possible to obscure the traffic in the future. But then an interloper may know at least that the site was visited once.
The other challenge is that there is no point of obscuring the information if the proxy itself cannot be trusted, and it doesn’t run for free: someone has to pay its bills. This brings us back to Jardine, and who is paying for all of this.
Does encryption actually improve freedom?
Perhaps the best measure of whether encryption has improved freedoms can be found in the place with the biggest barrier to those freedoms on the Internet: China. China is one of the least free countries in the world, according to Freedom House.
Freedom to use encryption: not all countries are assessed.
Paradoxically, one might answer the question that freedom and encryption seem to go hand in glove, at least to a certain point. However, the causal effects seem to indicate that encryption is an outgrowth of freedom, and not the other way around. China blocks the use of Tor, as it does many sites through its Great Firewall, and there has been no lasting documented example that demonstrates that tools such as Tor have had a lasting positive impact.
On the other hand, to demonstrate how complex the situation is, and why Jardine’s (and everyone else’s) work is so speculative, it’s not like dissidents and marginalized people are going to stand up for a survey, and say, “Yes, here I am, and I’m subverting my own government’s policies.”
Oppression as a Service (OaaS)
Cryptophiliacs believe that they can ultimately beat out, or at least stay ahead of the authorities, whereas China has shown its great firewall to be fully capable of adapting to new technologies over time. China and others might also employ another tactic: persisting meta-information for long periods of time, until flaws in privacy-enhancing technology can be found.
This gives rise to a nefarious opportunity: Oppression as a Service. Just as good companies will often test out new technology in their own environments, and then sell it to others, so too could a country with a lot of experience at blocking or monitoring traffic. The price they charge might well depend on their aims. If profit is pure motive, some countries might balk at the price. But if ideology is the aim, common interest could be found.
For China, this could be a mere extension of its Belt and Road initiative. Cryptography does not stop oppression. But it may – paradoxically – stop some communication, as our current several Internets continue to fragment into the multiple Internets that former Google CEO Eric Schmidt raised in 2018 thought he was predicting (he was really observing).
Could the individual seeking to have a private conversation with a relative or partner fly under the radar of all of this state mechanism? Perhaps for now. VPN services for visitors to China thrive; but those same services are generally not available to Chinese residents, and the risks of being caught using them may far outweigh the benefits.
Re-establishing Trust: A Government Role?
In the meantime, cyber-losses continue to mount. Like any other technology, the genie is out of the bottle with encryption. But should services that make use of it be encouraged? When does its measurable utility become more a fetish?
By relying on cryptography we may be letting ourselves and others off the hook for their poor behavior. When a technical approach to enable free speech and privacy exists, who says to a miscreant country, “Don’t abuse your citizens”? At what point do we say that, regardless, and at what point do democracies not only take responsibility for their own governments’ bad behavior, but also press totalitarian regimes to protect their citizens?
The answer may lie in the trust models that underpin cryptography. It is not enough to encrypt traffic. If you do so, but don’t know who you are dealing with on the other end, all you have done is limited your exposure to that other end. But trusting that other end requires common norms to be set and enforced. Will you buy your medicines from just anyone? And if you do and they turn out to be poisons, what is your redress? You have none if you cannot establish rules of the Internet road. In other words, governance.
Maybe It’s On Us
Absent the sort of very intrusive government regulation that China imposes, the one argument that cryptophiliacs have in their pocket that may be difficult for anyone to surmount is the idea that, with the right tools, the individual gets to decide this issue, and not any form of collective. That’s no form of governance. At that point we had better all be cryptophiliacs.
We as individuals have a responsibility to decide the impact of our decisions. If buying a bitcoin is going to encourage more waste and prop up criminals, maybe we had best not. That’s the easy call. The hard call is how we support human rights while at the same time being able to stop attacks on our infrastructure, where people can die as a result, but for different reasons.
Editorial note: I had initially misspelled cryptophilia. Thanks to Elizabeth Zwicky for pointing out this mistake.
