Prof. Tim Oates Receives IARPA Contract under TrojAI Program

Prof. Tim Oates Receives IARPA funding to protect AI models from malicious actors

Professor Tim Oates, working with ARM Research, just received a contract from the Intelligence Advanced Research Projects Activity (IARPA) under the TrojAI program.  The goal of the work is to detect trojans hiding in deep neural networks (DNNs). 

DNNs are widely used in industry to solve problems like recognizing stop signs in self-driving cars and finding low and slow attacks on corporate intranets.  But there are ways of attacking DNNs during training to implant a trojan, an input pattern that can cause the network to do the wrong thing.  For example, one could poison the stop sign recognizer so that when a yellow sticky note is placed on the stop sign the network sees it as a speed limit sign. 

Professor Oates will explore ways of examing trained DNNs, without access to the data on which they were trained, to determine if they are hiding trojans.  His work will help protect systems that use DNNs from malicious actors.