Douglas Thain
University of Notre Dame, IN·Computer Science
Modern science often runs large computations on shared computing clusters and clouds, and the software that organizes this work can become a bottleneck. These projects address two such bottlenecks.
- Fields
Computer Science
Engineering
- Topics
distributed computing
scientific workflows
computational notebooks
- Grants
- 2 known grants, through Jun 2028
What they work on
- Develop NBFlow, a toolkit that records notebook executions and converts them into workflows that schedule tasks concurrently on high performance clusters or clouds.
- Create techniques to capture provenance, the record of how results were produced, from notebook-based workflows, and apply incremental computation so that only changed steps rerun.
- Deploy NBFlow with user communities in high energy physics and in geospatial and sustainability science through the I-GUIDE cyberinfrastructure.
- Build Pledge, an experimental system in which workflows declare their file paths, access modes and consistency needs up front as "consistency contracts" that the runtime enforces.
- Test whether workflow-level consistency management yields higher input/output bandwidth and faster end-to-end runtimes than strict global consistency on shared parallel file systems.
More about the research
One is the gap between interactive notebooks, where scientists explore data, and large-scale workflows that run across many machines. The other is the shared file system, which slows down when many tasks read and write data at once. Both aim to let scientists run data-intensive analyses faster and with less effort.
Why it matters
Scientists in fields such as high energy physics and geosciences could move from exploratory notebooks to large cluster computations with far less engineering effort. Faster, more predictable file access on clusters could shorten run times and reduce unexpected performance failures for researchers and system administrators running data-intensive workflows.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Roch Guerin
Washington University in St Louis, MO·Computer Science
Design methods to assign GEOS-Chem atmospheric columns on its cubed-sphere grid to processors so loads stay even, accounting for cost peaks at sunrise and sunset.
Xian-He Sun
Illinois Institute of Technology, IL·Computer Science
Engineer IOWarp, a modular and scalable data management platform for AI-infused scientific workflows, using NVMe SSDs, CXL memory devices and CPU-GPU co-design to reduce data access times on HPC systems.
Valerio Pascucci
University of Utah, UT·Computer Science
Build a testbed for a National Science Data Fabric that connects storage, networking and computing resources across supercomputing centers, campus clusters, commercial cloud and individual labs.
The school
University of Notre Dame: 19 labs on PI Pages, $346M a year on research, 277 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.