Yiming Su
B.Sc. in Computer Science, The University of Chicago.
whoami
I'm a third-year CS PhD at The University of Illinois-Urbana Champaign, advised by Prof. Tianyin Xu.
I work on agentic systems reliability: I design system-side frameworks and tooling support to reduce or eliminate the impacts of unreliable agent behaviors.
Research
LLM-powered agents are inherently nondeterministic in their behaviors, preventing us from applying them in systems where guarantees are desired (e.g., cluster management systems). My research focuses on providing reliability and gurantees on the AI agents. The efforts include using formal verification on the agent's behaviors to provide guarantees (WIP), building benchmarks to evaluate the agent's reliability, providing systems-side frameworks that prevent any harmful behaviors, or providing tooling support for the agent at runtime to mitigate them.
My research has culminated in SREGym, a live SRE benchmark with high-fidelity SRE incidents to evaluate SRE agent's incident diagnosis and mitigation capabilities, NSVIF, an agent harness that improves trustworthiness of agent output through formal methods, Stratus, a multi-agent systems that enables autonomous SRE incident management through a transaction-like semantics, and previously, HotGPT, trying to understand the edges and limits of LLMs before time.
Before working on agents, I focused on distributed systems reliability. My research focused on the semantics challenge of managing traditional distributed systems (e.g., Apache Cassandra) on cloud-native platforms (e.g., Kubernetes). Large-scale distributed software have complicated management semantics that are hard to capture in management programs (termed "operators"). We conducted an effort to understand and detect such semantics bug in operator programs, which is accepted into NSDI '26. We found 86 bugs (53 confirmed and 28 fixed) in popular operators of distributed systems.
A short bio can be found here.
News
- JOB Starting research internship at Microsoft Research in Redmond!
- NEW Gold Reviewer status reported by Illinois CS News Notes! Note
- AWARD Gold Reviewer at ICML'26!
- NEW SREGym featured on Illinois CS News! Post
- NEW Check out SREGym, a high-fidelity benchmark and training ground for SRE agents!!
- AWARD Best Presentation at CSL Student Conference 2026 for SREGym demonstration!
- NEW ICLR '26: SysMoBench paper accepted! Evaluating AI on formally modeling complex real-world systems.
- NEW NeurIPS '25: STRATUS multi-agent system for autonomous reliability engineering accepted!
- NEW NSDI '26: Research on cloud application management reliability accepted!
- CRA Undergraduate Research Awards Honorable Mention in UChicago CS News
Publications
- Arxiv Preprint
SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios [pdf]- Jackson Clark*, Yiming Su*, Saad Mohammad Rafid Pial, Yifang Tian, Lily Gniedziejko, Hans-Arno Jacobsen, Yinfang Chen, Tianyin Xu (*: co-primary authors)
- CAIS '26
SREGym: A Live Training Ground for AI SRE Agents with High-Fidelity Failure Drills [pdf]- Jackson Clark, Yiming Su, Saad Mohammad Rafid Pial, Lily Gniedziejko, Tianyin Xu
- Arxiv Preprint
Neuro-Symbolic Verification on Instruction Following of LLMs [pdf]- Yiming Su, Kunzhao Xu, Yanjie Gao, Fan Yang, Cheng Li, Mao Yang, Tianyin Xu
- ICLR '26
SysMoBench: Evaluating AI on Formally Modeling Complex Real-World Systems [pdf]- Qian Cheng, Ruize Tang, Emilie Ma, Finn Hackett, Peiyang He, Yiming Su, Ivan Beschastnikh, Yu Huang, Xiaoxing Ma, Tianyin Xu
- NeurIPS '25
STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds. [pdf]- Yinfang Chen*, Jiaqi Pan*, Jackson Clark*, Yiming Su*, Noah Zheutlin, Bhavya Bhavya, Rohan Arora, Yu Deng, Saurabh Jha, Tianyin Xu (*: co-primary authors)
- NSDI '26
Who Watches the Watchers? On the Reliability of Softwarizing Cloud Application Management [pdf]- Jiawei Tyler Gu, Zhen Tang, Yiming Su, Bogdan A. Stoica, Xudong Sun, William X. Zheng, Yue Zhang, Akond Rahman, Chen Wang, and Tianyin Xu
- SOSP '24
If At First You Don’t Succeed, Try, Try, Again...? Insights and LLM-informed Tooling for Detecting Retry Bugs in Software Systems [pdf]- Bogdan Alexandru Stoica*, Utsav Sethi*, Yiming Su, Cyrus Zhou, Shan Lu, Jonathan Mace, Madan Musuvathi, Suman Nath (*: co-primary authors)
- HotOS '23
HotGPT: How to Make Software Documentation More Useful with a Large Language Model? [pdf]- Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, Suman Nath
Awards
- Gold Reviewer, ICML'26
- Best Presentation, Security, Privacy, and Trustworthy AI, CSL Student Confernece 2026 -- SREGym Systems Live Demonstration (Feb. 24, 2026)
Talks
- Speaker, CSL Student Confernece 2026 -- SREGym Systems Live Demonstration (Feb. 24, 2026, Me presenting SREGym)
- Co-host, IBM-Illinois Discovery Accelerator Institute Annual Meeting -- Workshop Demo on SRE AI Agent Cloud Incident Mitigation on ITBench. (Apr. 25, 2025)
- Speaker, HotGPT: How to Make Software Documentation More Useful with a Large Language Model? -- HotOS '23. (Me presenting our work)
Services
- ICML '26 Reviewer
- Siebel Peer Mentor
- OSDI '25 Artifact Evaluation Reviewer
- ATC '25 Artifact Evaluation Reviewer
Mentoring
I'm extremely fortunate to be able to mentor and work with these students.
Random things
Misc. images I collected during PhDMisc. quotes
Number of times this page got falsely marked as deceptive website
| dotfiles: git@github.com:yimingsu01/dotfiles.git |
Linkin Park - Rock am Ring 2001 Opening
Unless specifically noted, I do not own any of the images presented on this site. All rights go to their respective owner.
My past life... here