Yiming Su
B.Sc. in Computer Science, The University of Chicago.
whoami
I'm a third-year CS PhD at The University of Illinois-Urbana Champaign, advised by Prof. Tianyin Xu.
I work on AI for System Reliability Engineering: I design system-side frameworks and tooling support to make agentic systems more reliable.
Research
LLM-powered coding agents have fundamentally changed engineering workflows in software development. However, on the System Reliability Engineering side, LLM agents have achieved little successes so far. System Reliability Engineering tasks, such as incident management and cluster maintenance, require a substantially different set of capabilities that current agents and agentic systems lack. Moreover, the nondeterministic behavior of LLMs can have catastrophic effects on production systems. My research focuses on building high-fidelity benchmarks to evaluate how prepared LLM agents are for System Reliability Engineering tasks, as well as developing agent frameworks that enable them to operate cloud clusters and distributed systems more reliably.
My research has culminated in SREGym, a live SRE benchmark with high-fidelity SRE incidents to evaluate SRE agent's incident diagnosis and mitigation capabilities, and Stratus, a multi-agent systems that enables autonomous SRE incident management through a transaction-like semantics, and previously, HotGPT, trying to understand the edges and limits of LLMs before time.
A short bio can be found here.
News
- JOB Starting research internship at Microsoft Research in Redmond!
- NEW Gold Reviewer status reported by Illinois CS News Notes! Note
- AWARD Gold Reviewer at ICML'26!
- NEW SREGym featured on Illinois CS News! Post
- NEW Check out SREGym, a high-fidelity benchmark and training ground for SRE agents!!
- AWARD Best Presentation at CSL Student Conference 2026 for SREGym demonstration!
- NEW ICLR '26: SysMoBench paper accepted! Evaluating AI on formally modeling complex real-world systems.
- NEW NeurIPS '25: STRATUS multi-agent system for autonomous reliability engineering accepted!
- NEW NSDI '26: Research on cloud application management reliability accepted!
- CRA Undergraduate Research Awards Honorable Mention in UChicago CS News
Publications
- Arxiv Preprint
SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios [pdf]- Jackson Clark*, Yiming Su*, Saad Mohammad Rafid Pial, Yifang Tian, Lily Gniedziejko, Hans-Arno Jacobsen, Yinfang Chen, Tianyin Xu (*: co-primary authors)
- CAIS '26
SREGym: A Live Training Ground for AI SRE Agents with High-Fidelity Failure Drills [pdf]- Jackson Clark, Yiming Su, Saad Mohammad Rafid Pial, Lily Gniedziejko, Tianyin Xu
- Arxiv Preprint
Neuro-Symbolic Verification on Instruction Following of LLMs [pdf]- Yiming Su, Kunzhao Xu, Yanjie Gao, Fan Yang, Cheng Li, Mao Yang, Tianyin Xu
- ICLR '26
SysMoBench: Evaluating AI on Formally Modeling Complex Real-World Systems [pdf]- Qian Cheng, Ruize Tang, Emilie Ma, Finn Hackett, Peiyang He, Yiming Su, Ivan Beschastnikh, Yu Huang, Xiaoxing Ma, Tianyin Xu
- NeurIPS '25
STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds. [pdf]- Yinfang Chen*, Jiaqi Pan*, Jackson Clark*, Yiming Su*, Noah Zheutlin, Bhavya Bhavya, Rohan Arora, Yu Deng, Saurabh Jha, Tianyin Xu (*: co-primary authors)
- NSDI '26
Who Watches the Watchers? On the Reliability of Softwarizing Cloud Application Management [pdf]- Jiawei Tyler Gu, Zhen Tang, Yiming Su, Bogdan A. Stoica, Xudong Sun, William X. Zheng, Yue Zhang, Akond Rahman, Chen Wang, and Tianyin Xu
- SOSP '24
If At First You Don’t Succeed, Try, Try, Again...? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems [pdf]- Bogdan Alexandru Stoica*, Utsav Sethi*, Yiming Su, Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath (*: co-primary authors)
- HotOS '23
HotGPT: How to Make Software Documentation More Useful with a Large Language Model? [pdf]- Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Awards
- Gold Reviewer, ICML'26
- Best Presentation, Security, Privacy, and Trustworthy AI, CSL Student Confernece 2026 -- SREGym Systems Live Demonstration (Feb. 24, 2026)
Talks
- Speaker, CSL Student Confernece 2026 -- SREGym Systems Live Demonstration (Feb. 24, 2026, Me presenting SREGym)
- Co-host, IBM-Illinois Discovery Accelerator Institute Annual Meeting -- Workshop Demo on SRE AI Agent Cloud Incident Mitigation on ITBench. (Apr. 25, 2025)
- Speaker, HotGPT: How to Make Software Documentation More Useful with a Large Language Model? -- HotOS '23. (Me presenting our work)
Services
- ICML '26 Reviewer
- Siebel Peer Mentor
- OSDI '25 Artifact Evaluation Reviewer
- ATC '25 Artifact Evaluation Reviewer
Mentoring
I'm extremely fortunate to be able to mentor and work with these students.
Random things
Misc. images I collected during PhDMisc. quotes
Number of times this page got falsely marked as deceptive website
| dotfiles: git@github.com:yimingsu01/dotfiles.git |
Linkin Park - Rock am Ring 2001 Opening
Unless specifically noted, I do not own any of the images presented on this site. All rights go to their respective owner.
My past life... here