Xiteng Yao

PhD Candidate, Electrical and Computer Engineering, Boston University

prof_pic.jpg

I am a PhD candidate in Computer Engineering at Boston University, advised by Prof. Martin Herbordt in the CAAD Lab (Computer Architecture and Automated Design), expecting to graduate in May 2027.

My research is on systems for large language models: where computation, state, and coordination should live in distributed LLM training and inference. I work across the stack, from programmable switches and SmartNIC datapaths, through collective communication and RDMA, to GPU performance modeling and LLM serving runtimes. I also build FPGA accelerators for secure machine learning, and ask how to measure what really helps LLM agents that optimize code.

I have interned at Arm as an AI Research Scientist Intern, building an attention–FFN disaggregated LLM inference runtime that, on models of up to 2.4T parameters across 36 GB200 GPUs, reached up to 67% higher throughput than vLLM, and at Amazon Web Services as an Applied Scientist Intern, where I built KernelSight-LM, a kernel-level LLM inference simulator. More in experience and the CV.

Research interests: LLM training and inference systems · SmartNIC/FPGA acceleration · programmable and stateful switches · collective communication · GPU performance modeling · AI for performance engineering · privacy-preserving computation.

news

Sep 2026 Three papers at IEEE HPEC 2026: ACBench, PoolStateLens, and RL-guided GPU compiler optimization on Blackwell. Research
Jun 2026 KernelSight-LM, a kernel-level LLM inference simulator from my AWS internship, is on arXiv. Research page
Jun 2026 Presented SmartFuse-LLM at HEART 2026: fusing computation into collectives projects 50.5% lower per-boundary latency. Research page
May 2026 Started as an AI Research Scientist Intern at Arm in Seattle, working on attention–FFN disaggregated LLM inference. Experience
Sep 2025 Our paper Accelerating Multi-Party Computation Using Heterogeneous Systems received the Outstanding Student Paper Award at IEEE HPEC 2025. Research page
Aug 2025 Started as an Applied Scientist Intern at Amazon Web Services in Santa Clara. Experience

selected publications

  1. ACBench: A Benchmark Suite for Auditing Retry and Profiling Feedback in LLM-Driven Code Optimization
    Xiteng Yao*, Ziyuan Chu*, Feng Tai*, Bangjie Xue, Yigong Hu, and Martin Herbordt
    In 2026 IEEE High Performance Extreme Computing Conference (HPEC), 2026
  2. PoolStateLens: Measurement-Grounded Evaluation of an On-Switch Memory Pool for LLM Training
    Xiteng Yao and Martin Herbordt
    In 2026 IEEE High Performance Extreme Computing Conference (HPEC), 2026
  3. SmartFuse-LLM: Fusing Computation into Collectives for Efficient Tensor-Parallel LLM Systems
    Xiteng Yao, Bangjie Xue, Anthony Skjellum, and Martin Herbordt
    In Proceedings of the 16th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies (HEART), 2026
  4. arXiv
    KernelSight-LM: A Kernel-Level LLM Inference Simulator
    KernelSight-LM: A Kernel-Level LLM Inference Simulator
    Xiteng Yao, Taeho Kim, Hengzhi Pei, Xinle Liu, Kyle Ulrich, Leonard Lausen, Ashish Khetan, Xiang Song, George Karypis, and Martin Herbordt
    arXiv preprint arXiv:2606.28565, 2026
  5. Accelerating Multi-Party Computation Using Heterogeneous Systems
    Outstanding Student Paper Award
    Xiteng Yao, Shining Yang, Mayank Varia, and Martin Herbordt
    In 2025 IEEE High Performance Extreme Computing Conference (HPEC), 2025
  6. ICS
    SmartNIC-GPU-CPU Heterogeneous System for Large Machine Learning Model with Software-Hardware Codesign
    SmartNIC-GPU-CPU Heterogeneous System for Large Machine Learning Model with Software-Hardware Codesign
    Anqi Guo, Yuchen Hao, Xiteng Yao, Shining Yang, Jianyu Huang, Tong Geng, and Martin Herbordt
    In Proceedings of the 39th ACM International Conference on Supercomputing (ICS), 2025