Protecting AI against cryptanalytic attacks 

Security researchers have developed the first functional defence mechanism capable of protecting against ‘cryptanalytic’ attacks used to ‘steal’ the model parameters that define how an AI system works.


Image by poppet07 | Freepik

“AI systems are valuable intellectual property, and cryptanalytic parameter extraction attacks are the most efficient, effective, and accurate way to ‘steal’ that intellectual property,” said Ashley Kurian, first author of a paper on the work and a PhD student at North Carolina State University, US. “Until now, there has been no way to defend against those attacks. Our technique effectively protects against these attacks.”

“Cryptanalytic attacks are already happening, and they’re becoming more frequent and more efficient,” said Aydin Aysu, corresponding author of the paper and an associate professor of electrical and computer engineering at North Carolina State University, US. “We need to implement defence mechanisms now, because implementing them after an AI model’s parameters have been extracted is too late.”

The issue is the cryptanalytic parameter extraction attacks. Parameters are the essential information used to describe an AI model. Essentially, parameters are how AI systems perform tasks. Cryptanalytic parameter extraction attacks are a purely mathematical way of determining what a given AI model’s parameters are, allowing a third party to recreate the AI system.

“In a cryptanalytic attack, someone submits inputs and looks at outputs,” Aysu added. “They then use a mathematical function to determine what the parameters are. So far, these attacks have only worked against a type of AI model called a neural network. However, many, if not most, commercial AI systems are neural networks, including large language models such as ChatGPT.”

So, how do you defend against a mathematical attack? The new defence mechanism relies on a key insight the researchers had regarding cryptanalytic parameter extraction attacks. While analysing these attacks, the researchers identified a core principle that every attack relied on. To understand what they learnt, you have to understand the basic architecture of a neural network.

The fundamental building block of a neural network model is called a ‘neurone’. Neurones are arranged in layers and are used in sequence to assess and respond to input data. Once the data has been processed by the neurones in the first layer, the outputs of that layer are passed to a second layer. This process continues until the data has been processed by the entire system, at which point the system determines how to respond to the input data.

“What we observed is that cryptanalytic attacks focus on differences between neurones,” continued Kurian. “And the more different the neurones are, the more effective the attack is. Our defence mechanism relies on training a neural network model in a way that makes neurones in the same layer of the model similar to each other. You can do this only in the first layer or on multiple layers. And you could do it with all of the neurones in a layer, or only on a subset of neurones.”

“This approach creates a ‘barrier of similarity’ that makes it difficult for attacks to proceed,” stated Aysu. “The attack essentially doesn’t have a path forward. However, the model still functions normally in terms of its ability to perform its assigned tasks.”

In proof-of-concept testing, the researchers found that AI models which incorporated the defence mechanism had an accuracy change of less than one per cent.

“Sometimes a model that was retrained to incorporate the defence mechanism was slightly more accurate, sometimes slightly less accurate – but the overall change was minimal,” Kurian said.

“We also tested how well the defence mechanism worked,” added Kurian. “We focused on models that had their parameters extracted in less than four hours using cryptanalytic techniques. After retraining to incorporate the defence mechanism, we were unable to extract the parameters with cryptanalytic attacks that lasted for days.”

As part of this work, the researchers also developed a theoretical framework that can be used to quantify the success probability of cryptanalytic attacks.“This framework is useful because it allows us to estimate how robust a given AI model is against these attacks without running such attacks for days,” Aysu continued. “There is value in knowing how secure your system is – or isn’t.”

“We know this mechanism works, and we’re optimistic that people will use it to protect AI systems from these attacks,” said Kurian. “And we are open to working with industry partners who are interested in implementing the mechanism.”

“We also know that people trying to circumvent security measures will eventually find a way around them – hacking and security are engaged in a constant back and forth,” concluded Aysu. “We’re hopeful that there will be sources of funding moving forward that allow those of us working on new security efforts to keep pace.”

DOI: 10.48550/arXiv.2509.16546