News & updates
Research, teaching, and product announcements.
FrontierCS 2, a benchmark I co-authored, is out on arXiv. It tests whether AI agents can do computer science R&D: each of its 48 tasks removes a paper's key contribution from its released code, and an agent passes only by beating the original on hidden workloads. Watch the one-minute video or see the publication entry.
I started Boxi Learn, which takes the stage-based approach of Boxi Math to other Kâ12 subjects. It has 109 hands-on lessons across nine subjects so far, from Chinese and English to physics, history, and computing, in English and Chinese. See Education equity on the home page.
We released CXBench, a benchmark built with Genesys that scores LLM agents in regulated customer-experience workflows on both the business outcome and the required process. Across seven models, 80.1% of outcome-passing runs failed a process check. The paper is now on arXiv; see the research summary.
I started Boxi Math, a collection of free Kâ12 math videos in English and Chinese that show why math works, organized by stage rather than school year. The videos and text are CC BY 4.0, and the tools behind them are open source at animind-ai. See Education equity on the home page.
OneAI, which I co-founded, now supports olympiad training in mathematics, physics, chemistry, biology, and informatics at The Affiliated High School of South China Normal University and Guangzhou University Affiliated Middle School. See how it works.
OneAI is now also used at Zhixin School and Daoyuan School, both affiliated with the Chinese University of Hong Kong, Shenzhen, bringing it to four schools in Guangzhou and Shenzhen. See where it is used.
Four new preprints: on RAG hallucination detection, conversational agent testing, code specification alignment, and causal root-cause analysis. All are on the publication list.
Two papers accepted at ICML 2026, one in the main track and one in the position-paper track: "SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark" and "Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility".
Our paper "UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench" was accepted at ACL 2025.
Our extended abstract "DSPy Guardrails: Building Safe LLM Applications via Self-Refining Language Model Pipelines" was accepted at the Compound AI Systems Workshop, Data + AI Summit 2024 (San Francisco, June 13, 2024).
Our paper "Testing Graph Database Systems via Equivalent Query Rewriting" was accepted at ICSE 2024.
We introduced Retromorphic Testing, a general black-box approach to the test oracle problem. Inspired by inverse functions, it builds a dual-program architecture to test the target software. Read the paper.
Our paper "Deep Learning or Classical Machine Learning? An Empirical Study on Log-Based Anomaly Detection" was accepted at ICSE 2024.
Our paper "Automated Testing and Improvement of Named Entity Recognition Systems" was accepted at ESEC/FSE 2023.
Our paper "ROME: Testing Image Captioning Systems via Recursive Object Melting" was accepted at ISSTA 2023.