Shivam Patel
Shivam Patel

Shivam Patel

PhD Student, Electrical & Computer Engineering Carnegie Mellon University

shivamap [at] andrew [dot] cmu [dot] edu

I am a third-year PhD student in ECE at Carnegie Mellon University, fortunate to be advised by Prof. Gauri Joshi.

I was an AI research intern at Bosch AI in summer of 2026, where I worked on datacenter level inference optimization for LLMs. I did my undergrad in Electrical Engineering from IIT Bombay in 2024, advised by Prof. Vivek Borkar. I also spent a great summer at USC Viterbi with Prof. Meisam Razaviyayn in 2023.

Research Overview

My research focuses on making autoregressive inference more efficient and capable, through algorithmic and systems advances for solving complex tasks. In particular, I study how to improve the compute and memory efficiency of LLM systems, with an emphasis on effectively utilizing multiple models to push the Pareto frontier of performance and latency. Specifically I care about:

  1. LLM Ensembles: Combining multiple language models to improve overall performance.
  2. Efficient Inference: Reducing the compute, memory, latency, and cost of LLM systems.
  3. Inference Systems: Building better serving engines and scheduling policies for complex, multi-model and multi-turn workloads.

Before CMU, I worked on theoretical aspects of machine learning. I co-designed an asymptotic CVaR risk measure, and a method for improved statistical fairness in ML models.

I’m always excited to explore new ideas across a broad range of topics in LLMs. If you’re interested in collaborating or discussing potential research ideas, please feel free to reach out via email.

News

Aug 2026
A new work Beyond Accuracy and Cost tackles latency-aware LLM query routing for dynamic workloads.
Jul 2026
LOCUS presented at AdaptFM@ICML 2026 in Seoul, South Korea!
May 2026
Joined Bosch as an AI Research Intern in Pittsburgh. Working on large-scale inference optimization.
May 2026
ProxRouter presented at AISTATS 2026 in Tangier, Morocco!
Apr 2026
Awarded best PhD Qualifying Examination award by ECE, CMU (Spring 26).
Jan 2026
Two preprints out: LOCUS on generating capable low-dimensional model embeddings, and Federate the Router for decentralized and private LLM query router training.
Oct 2025
New Preprint: ProxRouter improves robustness of LLM query routers under true distribution shifts.
Sep 2025
Awarded the William Messner Endowed Fellowship by ECE, CMU.
Aug 2024
Began my PhD at Carnegie Mellon University, advised by Prof. Gauri Joshi!! Supported by the CMU CIT Dean's Fellowship.

Selected Publications All publications →

  1. arXiv 2026Beyond Accuracy and Cost: Latency-Aware LLM Query Routing for Dynamic Workloads

    Shivam Patel*, Akaash Parthasarathy*, Ankur Mallick, Gauri Joshi

    * Equal contribution

    Project page →arXiv

  2. AdaptFM @ ICML 2026LOCUS: Low-Dimensional Model Embeddings for Efficient Model Exploration, Comparison, and Selection

    Shivam Patel, William Cocke, Gauri Joshi

    Project page →arXiv

  3. arXiv 2026Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations

    Baris Askin*, Shivam Patel*, Anupam Nayak*, Andrea Vigano, Jiin Woo, Gauri Joshi, Carlee Joe-Wong

    * Equal contribution

    arXiv

  4. AISTATS 2026ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers

    Shivam Patel, Neharika Jali, Ankur Mallick, Gauri Joshi

    Project page →arXiv

  5. ICLR 2024f-FERM: A Scalable Framework for Robust Fair Empirical Risk Minimization

    Sina Baharlouei, Shivam Patel, Meisam Razaviyayn

    arXiv

Experience

Awards

Beyond research

I have recently become an endurance sports fan — get in touch if you are around Pittsburgh and up for a long run or a swim. I used to be an avid birdwatcher, and I played tabla and harmonium in an early life. I speak four languages.