How reliable is artificial intelligence, really? An interdisciplinary research team at TU Wien, has developed a method that allows for the exact calculation of how reliably a neural network operates within a defined input domain. In other words: It is now possible to mathematically guarantee that certain types of errors will not occur – a crucial step forward for the safe use of AI in sensitive applications.
From Theory to Safety
From smartphones to self-driving cars, AI systems have become an everyday part of our lives. But in applications where safety is critical, one central question arises: Can we guarantee that an AI system won’t make serious mistakes – even when its input varies slightly?
A team from TU Wien – Dr. Andrey Kofnov, Dr. Daniel Kapla, Prof. Efstathia Bura and Prof. Ezio Bartocci - bringing together experts from mathematics, statistics and computer science, has now found a way to analyze neural networks, the brains of AI systems, in such a way that the possible range of outputs can be exactly determined for a given input range – and specific errors can be ruled out with certainty.
Small Changes, Big Impact?
“Neural networks usually behave in a predictable way — they give the same output every time you feed in the same input” - says Dr. Andrey Kofnov. "But in the real world, inputs are often noisy or uncertain, and cannot always be described by a single, fixed value. This uncertainty in the input leads to uncertainty in the output.”
“Imagine a neural network that receives an image as input and is tasked with identifying the animal in it,” says Prof. Ezio Bartocci. “What happens if the image is slightly altered? A different camera, a bit more noise, changes in lighting – could that cause the AI to suddenly misclassify what it sees?”
"Understanding the full range of possible outputs helps in making better, safer decisions — especially in high-stakes areas like finance, healthcare, or engineering," adds Andrey Kofnov. "By computing the likelihood of possible outputs, we can answer important questions like: What’s the chance of an extreme outcome? How much risk is involved?”
These kinds of questions are difficult to answer using conventional testing. While many scenarios can be tried out, full coverage of all possible inputs is virtually impossible. There may always be rare edge cases that were not tested – and in which the system fails.
Mathematics in Multi-Dimensional Space
The solution developed at TU Wien uses a geometric approach: “The set of all possible inputs – for example, all possible images that could be fed into such an AI system – can be imagined as a space that is geometrically similar to our 3-dimensional world, but with an arbitrary number of dimensions,” explains Prof. Efstathia Bura. “We partition this multi-dimensional space into smaller subregions, each of which can be precisely analyzed to determine the outputs the neural network will produce for inputs from that region.”
This makes it possible to mathematically quantify the likelihood of a range of outputs – potentially ruling out erroneous results with 100% certainty.
The theory is not yet applicable to large-scale neural networks, such as Large Language Models. “An AI like ChatGPT is much too complex for this method. Analyzing it would require an unimaginable amount of computing power,” says Daniel Kapla. “But we have shown that at least for small neural networks, rigorous error quantification is possible.”
Interdisciplinary Research at SecInt
The method was developed as part of the SecInt, opens an external URL in a new window doctoral college at TU Wien, which fosters interdisciplinary collaboration in the field of IT security. Ethical issues and the societal impact of technology also play a central role in the program.
Prof. Ezio Bartocci and Prof. Efstathia Bura worked together with Dr. Andrey Kofnov (former PhD student and current postdoc) and Dr. Daniel Kapla (postdoc) to develop this new method, combining ideas from AI theory, statistics, and formal methods.
Original Publication
The research was peer-reviewed and accepted for presentation at ICML 2025, opens an external URL in a new window – one of the world’s leading conferences in machine learning. The event will take place from July 13 to 19, 2025, in Vancouver, Canada. A preprint of the paper is available., opens an external URL in a new window
Contact
Dr. Andrey Kofnov
Institute of Statistics and Mathematical Methods in Economics
TU Wien
+43 1 58801 10588
andrey.kofnov@tuwien.ac.at
Dr. Daniel Kapla
Institute of Statistics and Mathematical Methods in Economics
TU Wien
+43 1 58801 10584
daniel.kapla@tuwien.ac.at
Prof. Ezio Bartocci
Institute for Computer Engineering
TU Wien
+43 1 58801 18226
ezio.bartocci@tuwien.ac.at
Prof. Efstathia Bura
Institute of Statistics and Mathematical Methods in Economics
TU Wien
+43 1 58801 10580
efstathia.bura@tuwien.ac.at