ASI Safety Lab

Is This Denial?

In the last few weeks, I have brushed up my knowledge about cryptography, computer security, and security engineering. I read a couple of books cover to cover, even peer-reviewed articles.

25 years ago, I was actually looking up to cryptographers and security experts who have had a very skeptical and conservative mindset. Bruce Schneier called them (and himself) professionally paranoid — truly professional skeptics. I really liked that.

But now, I am skeptical about them: has their skepticism make them also blind to possible threats lurking behind the horizon? How were they dealing with the threat from Quantum Computation (QC) for the last 25 years? What is their view on Artificial Superintelligence (ASI)? Or what do they think about Reverse Code Engineering (RCE) as a thread for keys?

I was really curious to see where and happened in the last 25 years in Cryptography. A refresher and having a current overview of what cryptographers think is important is always good — but I extended it also to “Security Engineering (3rd Ed.)” (2020, Ross J. Anderson). There were some new things I was learning, but mainly in security engineering. But quite honestly, it was shockingly little that was new in cryptography. Ok, SSL is now called TLS. Oh, and TLS version 1.3 got rid of some baggage from the past.

The only new chapter that I saw in my 2020 cryptography book was the admission that Quantum Computation (QC) could be a threat for all major algorithms used in Public/Private Key Encryption. Well, I knew that 25 years ago. So, what’s their solution? Well, not so fast. QC is not there yet. But at least most cryptographers would amid that they can’t ignore QC any longer.

In the 90s I actually studied the Shor algorithms, the first quantum computational algorithm claiming to factor large numbers into primes. Of course, we can’t know when it will be used in a first prototype device or in off-the-shelve products, but I would have considered that as something that must be mitigated as a risk much sooner. Fast forward to 2021: there is still no accepted algorithm capable to deal with that long-known threat, but NIST is in the process of defining one.
BTW: PKI is exposed to the threat from QC because the public key is being published. So, not keeping the public key secret is an accepted problem, because there is a chance that we can calculate the private key from it. … hmm, interesting … but more about that later.

Of course, I found nothing about ASI in a good deal of Cryptography 2020 books that I’ve checked out and nothing in the 1,200+ pages of the above-mentioned “Security Engineering, 3rd Ed” (2020), by Ross J. Anderson. Well, there is remarkably little, or should I (better) say nothing about Reverse Engineering from other top InfoSec researchers. I checked out computer security researchers like Mathias Payer’s “Software Security: Principles, Policies, and Protection” (2021): but nothing.

Reverse Code Engineering

There is a threat known to everyone in computer security: RCE (Reverse Code Engineering) … What does this threat even mean? In simple terms — applied to cryptography: Attackers are stealing the key by modifying the software. With a stolen key, no security, and safety. RCE can literally pull the rug under Cryptography and Computer Security. Why do we hear so little about these threats? Are these threats not real? Or is this threat too inconsequential to be even mentioned?

Is RCE new? That question is a funny one: in the 90ies, I remember reading the famous +ORC “how to crack” lessons from the Old Red Cracker (abbreviated +ORC). RCE changed, OS changed, tools changed — and then I saw recently an interesting patent US 10,747,518 (2020) “Method and System for Modifying Machine Instructions within Compiled Software” from And then there is even a book “Practical Reverse Engineering X86, X64, ARM, Windows Kernel, Reversing Tools, and Obfuscation” (2014) by Bruce Dang, Alexandre Gazet, Elias Bachaalany, S√©bastien Josse.

Is RCE illegal? Well, I am not a lawyer; but yes, computer crimes are illegal. However, are attackers known for following the law? I thought we assume that they don’t care.

RCE is very hard to defend against — someone I was talking to, said: it is close to impossible to make a computer and its content safe, in particular, once you gain Sysadmin privileges. So, what can really be done against RCE?

  1. Access Management: Don’t give an attacker sufficient access rights (means sysadmin rights) to do attacks on other software.
  2. Obfuscating machine code: making it deliberately more difficult to understand the software — at least for humans.
  3. Temper proofing software: Changing the software so that at runtime internal tools or subroutines can detect and possibly block computer attacks to continue, which is called RASP (Runtime Application Self-Protection)
  4. Crypto-Hardware: Using separate and dedicated hardware to protect keys and the encryption process.

Access Control is about the environment in which the encryption/decryption should run. Normally, software is not allowed to call or modify other applications. But there is an exception: sysadmins or superusers can do almost anything. Therefore, the goal of an attacker is to gain that role. The OS is trying to make that as difficult as possible, but the reality is the OS can only make it more difficult but never prevent this because the OS can’t decide if the developer wants that as a feature or if it is a bug. Usually, a human must identify it as a bug manually, and then it can be fixed by the developer. But access control is something that is truly outside the control of the security engineer implementing cryptography. In other words, we must take it as given that the encryption and decryption software is being attacked with sysadmin right, on the same level as we must accept that attackers listening to encrypted content on networks. Cryptographer and computer security experts must accept that every attack software operates with sysadmin rights as a simple fact.

It is not that crypto tool developers can’t do anything about an attacker trying to steal the keys. There are existing software components, called obfuscators, that are changing the machine code so that humans and tools used by attackers have a hard time understanding the inner working of an algorithm. Obfuscators are even inserting new subroutines and making simple operations extra-complicated without changing the actual result of the calculation. But there are a few simple facts that even obfuscators can’t ignore: there are a few calculations in subroutines busier than others — BTW: this can also be called a side-channel attack, a term that is being “reserved” for other types of attacks but is equally applicable here. One can estimate how much time a suspected underlying algorithm would take that contains and uses the key or essential elements of the key. Once the essential subroutine is located the key can be revealed. And that is only one of the many tricks RCE practitioners have on their leaves. And yes, attackers can use any combination of VM’s, profilers, disassemblers, and statistical tools (to save them time). It is really hard to imagine, that attackers could really be thrown off by obfuscation.

Obfuscation is a transformation of the original machine code into some other form of less readable/understandable machine code which is preserving the results that the original software is delivering. There is no reason to assume that we can’t develop a De-Obfuscator that transforms any machine code input into a form that can be much better understood than the input, potentially even better than the result provided by a compiler. I would guess that this could become a textbook application for reinforcement learning. Who knows, maybe there are tools using machine learning for improving the speed or code transparency already developed — if that is the case, please send me a message, I really would like to know. However, I can make an educated guess: some 3 letter agencies have already these tools besides other cool commercial tools like WindowSCOPE or the more basic Interactive Disassembler IDA to name only two.

Temper-proofing software is using obfuscators to hide the subroutines responsible for detecting and blocking computer attacks to continue. There are multiple RASP solutions on the market, and they all claim that they are making it so much harder to reverse engineer software, but I am not aware of any product that can really claim that it can’t be hacked. Again, I believe that it is only a matter of time that machine learning can be used to simplify the machine code and systematically remove all internal temper proofing methods automatically.

Once the above-mentioned defenses: Access Control, Obfuscation, and RASP are neutralized, there is no security for the keys anymore, doesn’t matter if we talk about PKI, digital signatures, or TLS. The only protection left is that content is encrypted when transmitted using session keys. And these session keys are deleted after they were used. But wait a minute, the session keys have been encrypted using the public key of the server. So without the private key from the server, the session key can’t be revealed from the key exchange message on the network. This is shifting the focus on the attack on encryption to servers — which could be advantageous if the attacker is only eavesdropping on exchanged messages.

For servers, there are different SSL/TLS solutions available, from an SSL box dedicated to encrypting/decrypting internet traffic at scale to Crypto cards designed to hold all keys in an independent and separate hardware component (like IBM’s CEX6S (4768) PCIe Cryptographic Coprocessor (HSM)). It really seems to be pretty hard to get the private keys out of this system (multiple hard- and software-based tamper-proofing measures). HMS even generates the keys so that there is no way to steal the private key from the hardware. But these cards have one disadvantage: anyone on the computer (including hackers) could use the card with their own software (this is called the “API problem”) which then could create serious problems for the owner of that system. Because of tight organizational measures around the physical machines, the most likely attack would come via the network, actually through the firewall. But many companies think that they have tightened up their network security so that they don’t believe that an intrusion can happen. I am curious if that argument holds water when ASI arises.

Heavy Software-Bias in Cryptography

I was talking with several computer security and TLS experts, not with top-level yet, but I get their explanation/excuse: RCE is not their problem. That problem needs to be solved by someone else. This silo mentality is not new or even untypical. And in a complex world, why should people bother about problems that can’t be solved. Well, there is some truth in it: Intel and other CPU manufacturer are using security rings, TEE (Trusted Execution Environment), and TPM (Trusted Platform Module) to get a lot of security and cryptography safely done. I don’t dare to have an opinion on how good this is — all I know: it is really complex, and that means to me, I don’t trust it.

There is a dominant trend in Cryptography and Computer Security: security must be solved with software. If there is no solution using software, then the excuse kicks in: let’s stop worrying, we have done our best. Full Stop! Why? Because all what hardware-based security brings is inflexibility and many promises it can’t keep, see the above-mentioned API problem. Nevertheless, reducing the problem of security largely to software measures is a group-think and their assumptions don’t need to be true necessarily.

Many in IT worry: Software is already complex and intransparent; at least we can do something. Hardware is so much worse: billions of transistors on the CPU. CPUs have an instruction set that goes into thousands of different instructions — and only a few thousand engineers have working knowledge on that level. Comparing that with software, in particular, open-source software — which can be checked out by any of the millions of developers.

From a security point of view, the problem remains: hardware must be taken with almost blind faith — we can’t check out the real internals of CPUs: they are incomprehensibly complex and no single analyst (not even a team of analysts) can understand and call it single-handedly secure. And then there is the suspicion that some companies are including hidden features/backdoors as demanded by their government (e.g. Huawei and China). Focusing on software as the basis of security is understandable, but it is still the wrong answer. If we can’t trust or fix the hardware foundation then we should think outside the box and look for other and better solutions (with much less complexity). We don’t even need to replace the complex security. I believe it is sufficient to have one simple additional security (low-level) feature that could be enough (see below).

As with other academic endeavors, Computer Security is largely a group effort. Components are isolated into modules and analyzed. We trust but verify (and peer review) the findings of other researchers. Security is being analyzed module, by module — with the intention: reduction of complexity. Is this approach working: yes and no. My concern is that we are currently too slow and not good enough when we have soon much better adversarial AI tools (usable by criminals) or when we experience a relentless ASI. Also, how good do we really understand the cumulative effects of multiple weaknesses in different components that we would identify as weaknesses if we are seeing them isolated? Or does this kind of effect doesn’t exist? If someone is finding that kind of weakness, then it is most likely ASI.

Additionally, security is currently analyzed as if the product/solution is immutable — including its software. I am not advocating for continuous recertification. But there is a systemic problem: If the product with its software delivers what it supposed to deliver; i.e. has no bugs, then: a job well done. No further reason to check for “problems”. But we know that there are reverse engineers who try to screw things up and try to cause a problem. Unfair? How can anyone expect that things are secure if hackers/crackers are changing the software? Irrelevant? I really don’t want to be the messenger of bad news, but that is what criminals are doing when they steal or extort. … and that is what we might have to expect in the future when ASI is around.

This problem is of course known to the security community. Their silence on this topic I find extremely problematic. It is one thing, knowing you can’t solve a problem and another thing, deliberately concealing or withholding warnings about known threats and/or attacks on the security infrastructure of TLS or PKI.

I can understand why professional skeptics have a hard time accepting that we may have to deal with ASI soon — because: maybe ASI is only a plot from Hollywood (a kind of fringe science and who would seriously ask the Pentagon if they have a defense plan against aliens – ridiculous) or we are lucky and ASI is no problem. But even if AI/ASI doesn’t have any intention on its own, AI as a tool can be used to automate an attack on computer systems and used to steal keys instead of using brute force in its key attacks. But can crypto practitioners really argue: stealing is like cheating and below the level of sophistication that one would expect in an attacker — the mindset is: it’s not up to us to make assumptions on what an attacker is doing. Limiting the considered threats to the technology that one is able to deal with is disingenuous.

It is often argued that attackers belong to different categories of skill sets, education, and resources. Definitely, someone smarter, better educated, and having better tools has a much higher probability to succeed. But with software, the heavy intellectual lifting has usually been done by someone else. Then, including or applying an exploit doesn’t require a genius.

Let’s extrapolate a few trends into the future: more software complexity, easier applicable machine learning/ reinforcement learning, and AI used in the development of new exploits. Sooner or later, N.S.A. Ph.D. level software security can be broken by a high-school kid. You don’t believe me? Well, let’s ask software companies like Microsoft about the level of sophistication that is required to bypass the access control measures for their more premium products like Office, etc. Or will there be an organized campaign to keep this kind of technology out of the hand of “amateurs”?

I read in almost every book on cryptography and computer security: complexity is the enemy of security and each and every computer security expert I was talking with (again anyone top-level yet) gave me the same answer: we must make key extraction more complex: more obfuscating, more RASP, faster updates due to 0-day vulnerabilities. Here we are — I am skeptical about complex solutions for security. Instead, I have a simple conclusion: We can’t protect keys, therefore we have no security. Software alone can’t solve security. Let’s call our current state of affairs what it is: The Illusion of Security.

If we want to survive the rise of ASI, then security and safety must deal with ASI and with whatever it could throw at us. Therefore, I would not call a product safe when the software can be hacked — It must withstand ASI: If there is any reason to distrust the software (e.g. modified in an unauthorized manner), then some additional mechanism must immediately put the product/hardware in safe mode. How many products follow this simple common-sense rule? Can this be done? Yes, it can; but we need to discuss this in a different post. Does this prevent software updates? No, it doesn’t (when done correctly); and this would need to be discussed also in a different post.

However, related to encryption/security/communication, I would like you to ask yourself the following question: would you really make a private or confidential call in a room full of quiet strangers? No? What if all these strangers got a headset from an event organizer and these headsets produce some distraction? Would you then dare to make a call in that room? Personally, I won’t — I would leave the room — or applied to our devices: Sure, we could “clear the room” on an IT device and remove all suspicious software from RAM — but that’s not done; because we trust the claims that other can’t be a problem. And then, where do I draw the line: what is safe and what is not trustworthy. That means, clearing the device before critical operations is probably not a doable solution.

Is There a Way Out? Can We Make Encryption safe?

I wrote a blog on Trustworthy Computing: “Safety based on First Principles” in which I discussed the use of Harvard Architecture for Security.

I am guided by First Principle Thinking. I want to create a situation in which an attacker must (almost) break the laws of nature when he really wants to reach his goal to breach the security of a system or create circumstances that will make the breach visible and obvious to us.

For achieving trustworthy encryption/communication, I believe we can make the exchange of messages safe and protected so that even an ASI would fail in breaking into it. Because every hardware can most likely successfully be probed physically by an attacker, the security that I would propose transcends that probing scenario. If we would additionally assume that the probed hardware would survive and the user would not even detect that his hardware was attacked, then the goal would be that any usage of the information (i.e., the keys) gained in that attack would be detected soon after that attack. So in the worst case, we would have a signal that would change/replace the compromised keys automatically but existing credentials, relationship, or connection information could be recovered and no further damage would occur. The compromised keys would be marked accordingly and could be used as honeypots.

The idea is to protect all keys (private and public) in a special hardware storage unit that is called the Key Safe (KS). Associated to the Key Safe is a Key Generator that generates Private/Public Key pairs and symmetric session keys. The keys within the Key Safe can’t be read out or used outside the KS in cleartext. The use of keys happens exclusively within the Encryption, Decryption Unit (EDU) directly associated with the KS. Only encrypted keys are exchanged with other hardware i.e. with another KS/EDU components. Under no circumstances can the KS/EDU give any of its keys or credentials to an outside system that is not being identified as a bonafide KS/EDU — in particular no keys, not even a public or session key is being sent to a software instance as every software is considered potentially compromised.

The problem with allowing a public key from the key pair generated in the KS/EDU to be published e.g. within the PKI is that we can’t know if the system requesting a key is really KS/EDU hardware or only pretending. If any key: private, public or session key would show up outside a Key Safe in cleartext then this would indicate a successful breach of security of the KS/EDU in which valuable key information was stolen. Keys are identified via their hashcode.

I will post more details related to the Key Safe idea in a separate post related to item 3 in my post “Standards in ASI Safety”. So, I want to leave here only to few highlights:

  1. When manufactured, the KS/EDU receives an ID; it generates within a component-specific private/public key pair that it uses when it has no other key within its Key Safe. For every single key and key pair, we have associated hashcodes that are used to identify keys (in case any was stolen) or it identifies the system on with the key-pair is being used or to which the public key belongs.
  2. The component-specific private/public key pairs are stored — other keys are stored when required
  3. Hashcode generated
  4. Digital signature created and confirmed