Rachee Singh
Cornell University, NY·Computer Science
Modern machine learning (ML) models have grown so large, often with trillions of parameters, that they no longer fit on a single computer. They have to be split across many accelerators, the specialized processors such as GPUs (graphics processing units) that do the heavy math.
- Fields
Computer Science
Engineering
- Topics
distributed machine learning
collective communication
accelerators
- Grants
- 2 known grants, through Jun 2028
What they work on
- Measure how communication speed and delay vary between accelerators in cloud data centers, accounting for proprietary interconnects and hidden network paths that current tools ignore.
- Automatically generate collective communication strategies, the coordinated data-exchange plans used in distributed ML, that are tailored to the measured links of a specific cloud deployment, within one data center or across several.
- Develop reconfigurable chip-to-chip optical interconnects using silicon photonics to link accelerators inside multi-accelerator servers, so that link bandwidth keeps pace with faster GPUs.
- Design custom accelerator layouts enabled by optical links that achieve bandwidth-optimal collective communication during distributed ML and limit the impact of accelerator failures.
More about the research
These chips constantly exchange data, and that traffic is now a major bottleneck. The lab's two projects attack this problem at two levels: the software that coordinates data exchange across cloud data centers, and the physical links that connect chips inside a single server.
Why it matters
Faster communication between accelerators could cut the time, energy and cost of training and running large ML models in cloud data centers. Cloud providers could benefit from better use of costly GPUs, and researchers in fields such as climate modeling and biomedical research could benefit from cheaper computing. Openly released tools and datasets could also make powerful ML more accessible for classroom teaching.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Divya Mahajan
Georgia Institute of Technology, GA·Computer Science·Early-career
Build a cross-platform framework that measures how thermal hotspots, frequency throttling, power spikes and energy limits restrict distributed AI training performance on GPUs and other accelerators.
Jiaqi Gu
Arizona State University Campus Immersion, AZ·Engineering·Early-career
Develop a learned electromagnetic solver, a machine-learning model of how light moves through photonic devices, and a circuit-level co-simulator for fast simulation of mixed electronic-photonic chips.
Daehyeok Kim
University of Texas at Austin, TX·Computer Science·Early-career
Create a programming model that describes robot manipulation tasks as cooperating control loops with explicit deadlines and recovery from robot and network failures.
The school
Cornell University: 37 labs on PI Pages, $1.6B a year on research, 537 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.