Atri Rudra
University at Buffalo, NY·Computer Science
Deep learning systems now touch daily life, but the models behind them have grown enormous. Training and running them takes so much computation that frontier work is largely limited to a handful of large technology companies.
- Fields
Computer Science
Mathematics & Statistics
Engineering
- Topics
deep learning
structured matrices
matrix computations
- Grants
- 1 known grant, through Apr 2027
What they work on
- Design a family of structured matrices meeting all properties deep learning requires at once, including an efficient projection problem for mapping arbitrary matrices into the family during training.
- Develop learning algorithms for these structured matrices that are parallel and friendly to modern hardware constraints, so models match large dense networks at a fraction of the size and compute.
- Identify new applications that the resulting theory of structured matrices unlocks, and use the deep learning setting to pose new versions of classical problems in matrix computation.
- Train undergraduate researchers on hardware-aware matrix computation and integrate responsible computing into the undergraduate curriculum.
More about the research
This project asks whether models can keep the expressive power of very large networks while using a small fraction of the size and compute. The strategy is to replace the dense matrices at the heart of these models — the big tables of numbers that transform data at each layer — with structured matrices, which have internal patterns that allow much faster multiplication and compact storage, chosen to fit the constraints of real hardware such as parallel processors.
Why it matters
If large-model performance can be matched at a fraction of the computational cost, capable deep learning systems become buildable by researchers, smaller organizations and institutions without access to massive computing clusters, broadening who can work at the frontier of machine learning. The underlying matrix theory may also improve general-purpose numerical computation.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Michal Derezinski
University of Michigan, MI·Computer Science·Early-career
Reformulate optimal matrix sketching, the construction of a small randomized approximation of input data, using black-box sampling methods, so that sketch and sub-sample outputs come with reliability guarantees suitable for software.
Vivak Patel
University of Wisconsin-Madison, WI·Mathematics & Statistics
Map a modular theory of randomized linear algebra algorithms onto composable computational kernels so that new, application-tailored randomized solvers can be assembled rapidly rather than coded from scratch.
Stephen Becker
University of Colorado Boulder, CO·Mathematics & Statistics
Develop gradient-free optimization algorithms for objective functions whose matrix-valued gradients have low intrinsic dimensionality, such as sparsity, low rank, or sparsity-plus-low-rank, to break worst-case efficiency barriers.
The school
University at Buffalo: 32 labs on PI Pages, $542M a year on research, 316 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.