Xinyu Xing
Northwestern University, IL·Computer Science·Early-career
Artificial intelligence agents that learn by trial and error can be fooled. Deep reinforcement learning, a method in which a software agent learns actions by collecting rewards, can be tricked by an attacker.
- Fields
Computer Science
Engineering
- Topics
deep reinforcement learning
adversarial attacks
adversarial policy
- Grants
- 1 known grant, through Sep 2027
What they work on
- Train adversarial agents that act in a target agent's environment to provoke undesired behavior, using explainable AI methods to guide where the attacks focus.
- Use the learned adversarial agents to expose weaknesses in deep reinforcement learning agents, then retrain those agents with adversarial training and evaluate them with a robustness check.
- Repair agents that still fail the adversary-resistance check by applying formal verification, which proves system properties mathematically, combined with program synthesis, which automatically generates code.
More about the research
A newer and more practical kind of attack does not touch the agent's sensors at all. Instead, a second, adversarial agent acts in the shared environment and provokes the target into bad behavior. These attacks are hard to defend against. Standard defenses built for other deep learning systems do not carry over well, and there are few tools for inspecting an agent to find its weak spots.
Why it matters
Reinforcement learning agents are used in settings where being manipulated can be costly. This work could give developers practical ways to test agents against adversarial opponents and harden them, which may improve the security of reinforcement learning applications. It also aims to advance AI security, explainable AI and verified AI.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Sheikh Rabiul Islam
University at Albany, NY·Computer Science
Develop seven AI4SecureCI learning modules that teach fair and explainable AI for securing cloud cyberinfrastructure, covering topics from malware detection to adversarial attacks, and integrate them into undergraduate and graduate courses.
Jun Zhuang
Boise State University, ID·Computer Science
Improve the resistance of quantum neural networks to adversarial attacks, meaning small deliberate input changes meant to fool them, using Bayesian label transition to infer more reliable labels.
Abdollah Homaifar
North Carolina A & T State University, NC·Engineering
Characterize combined trojan and adversarial attacks on deep learning-based UAV navigation, where a model resists ordinary attacks but turns malicious when a hidden trojan trigger is present.
The school
Northwestern University: 71 labs on PI Pages, $1.3B a year on research, 536 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.