Nikolai Matni
University of Pennsylvania, PA·Engineering
Robots and other systems that rely on machine learning increasingly operate in places their designers never fully anticipated. Keeping them safe is hard because there is uncertainty in the environment, in the system's goals, and in the learned components themselves.
- Fields
Engineering
Computer Science
Mathematics & Statistics
- Topics
safe learning-enabled systems
machine learning
robust control
- Grants
- 1 known grant, last ended Sep 2026
What they work on
- Develop safety-rich data augmentation and domain randomization techniques for training learning-enabled systems, and determine which types of data must be collected to guarantee end-to-end safety.
- Integrate these data methods into safety-aware robust learning, control and verification methods that make systems provably robust to known sources of uncertainty during offline design.
- Create online safety monitoring, uncertainty quantification and adaptation methods based on conformal prediction and active learning, so deployed systems can detect and learn about unknown unknowns while balancing safety risk against data collection.
More about the research
Some of these risks are "known unknowns" that can be planned for. Others are "unknown unknowns" that only show up once the system is running. This collaborative project, a partnership between the University of Pennsylvania and UC Berkeley, proposes a way to handle both.
Why it matters
Learning-enabled systems such as autonomous robots and vehicles must stay safe in environments that are uncertain and changing. This work aims to provide mathematical guarantees of end-to-end safety, plus methods to spot and learn from new, unexpected sources of risk during operation. These results could help engineers design and deploy safer autonomous systems.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Radoslav Ivanov
Rensselaer Polytechnic Institute, NY·Computer Science
Develop extrinsic neuro-symbolic calibration that adjusts the uncertainty of already-trained neural networks to match the assumptions of downstream state estimators and controllers, without retraining the networks.
Vijay Subramanian
University of Michigan, MI·Engineering
Develop robust reinforcement learning algorithms that use model-class knowledge, future predictions and off-policy methods to cope with mismatches and limited data from simulators or offline records.
Jaime Fernández Fisac
Princeton University, NJ·Engineering·Early-career
Develop a framework based on dynamic game theory in which a robot guarantees safety against the worst case within an Operational Design Domain (ODD), the conditions designers or regulators say it must handle, while modeling how agents gain information over time.
The school
University of Pennsylvania: 90 labs on PI Pages, $2.2B a year on research, 750 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.