Research · products · teaching
I am Boxi Yu, a software engineering researcher at Lero. I study how to test and evaluate coding agents and AI systems so that people can decide when to trust them, then turn those lessons into tools people can use and courses people can learn from.
Strengthening tests and examining the reliability of benchmark results.
Using relations between programs and executions to detect inconsistencies.
Metamorphic transformations and benchmarks reveal errors in vision, language, and agent systems.
LightAD, OpenRCA 2.0, and CLEANet examine model efficiency, causal diagnosis, and robustness to contaminated data.
Research · first-authored
Coverage- and mutation-driven test augmentation rejected 19.71% of previously passing patches in our evaluation of 30 agents on SWE-Bench Verified.
Generated unit tests exposed 345 erroneous patches that had originally passed SWE-Bench evaluation, showing how test quality changes conclusions about coding agents.
A dual-program approach to constructing test oracles: transform an input, map the output back to the input domain, and check the resulting relation.
A benchmark that scores business outcomes and process compliance separately for LLM agents in regulated customer-experience workflows, built with Genesys.
An empirical study that compares classical and deep learning approaches on accuracy and computational cost across five log datasets.
Courses · interactive lecture notes
For seven courses, I build or co-build one interactive page per lecture: from AI literacy for everyone, through a first Python program, discrete mathematics, data structures, linear algebra, and machine learning, to building software with coding agents. Every program and interactive check has been run, and every output shown is genuine.
Open course in Chinese and English · Designer: Boxi Yu · 中文版
Eight shared modules on using AI, checking what it says, and knowing when not to use it, with tracks for K–12, university, working adults, seniors, and teachers. The core module, C4, teaches learners to catch AI’s mistakes.
CUHK–Shenzhen · Instructors: Pinjia He, Junjie Hu, and Wenjing Ma
From number systems to recursion, classes, and data structures, ending with how large language models work. Step through what Python does in memory, predict each output before it appears, and finish every lecture with a self-check.
Open course in English · Instructors: Mingtao Xia and Boxi Yu
The language of proof and the objects of computer science: definitions and theorems, sets and counting, quantifiers, relations, contradiction and induction, functions, probability, number theory, and graphs. Key ideas come with demos you can step through, and every exercise has a hint and a full solution.
From01Definitionsto12Graphs
Python and Java versions · Instructors: Boxi Yu and Yintong Huo
The structures behind every program, from arrays and linked lists to search trees, hash tables, heaps, and graphs, with complexity and proof methods as the common language. Pick Python or Java: the concepts are shared, the code is in your language.
From01Complexity and Proof Methodsto09Graphs and Graph Algorithms
Open course in English · Designer: Boxi Yu · Inspired by MIT 18.06
From three pictures of a linear system to the singular value decomposition and computing with matrices. Every number in the notes is computed by running code, and every unit ends with a machine-learning example and a section on working with AI: what to ask, and how to check the answer.
FromL01–05Equations and eliminationtoL29–30Computing with matrices
CUHK–Shenzhen · Lectures 9–11 · Instructor: Jicong Fan
From a single neuron to attention: backpropagation on computational graphs, convolution and pooling, recurrent networks with LSTM and GRU, and the Transformer. Adjust the inputs, predict each result, and run the NumPy labs.
CUHK–Shenzhen · Fall 2026 · Instructor: Jinsheng Ba · Official course site
How LLMs and agents work, then how to use them across the software lifecycle: prompts, MCP and skills, requirements, coding, and team collaboration. Step through the agent loop, prompts, and tool calls, then try each step with a coding agent of your own.
Interactive notes and videos built with Peiyu Zhineng (培玉智能). I designed AI for All and LA101, teach DM101 with Mingtao Xia and Data Structures with Yintong Huo, and co-designed the AIE1001, DDA3020, and CSC4801 notes; their slides and course content belong to each instructor.
Agents generate. You verify, decide, and explain.
A practical twelve-week course for building with coding agents while keeping specifications, tests, review, security, and judgment in the loop.
Beyond my courses, I have mentored ten students. All ten co-authored peer-reviewed papers with me at ICSE, ESEC/FSE, or ISSTA, and eight are now Ph.D. students. See where they went.
Co-founder
Alongside my research I co-founded OneAI and Pacific Studio, where we build AI products that keep people in the decision seat.
ONEAI
Turn a learning objective into a question set, rubric, and student-level feedback loop. Teachers make the final call; AI drafts the first pass.
In use at four schools in Guangzhou and Shenzhen, including olympiad training in mathematics, physics, chemistry, biology, and informatics at The Affiliated High School of South China Normal University and Guangzhou University Affiliated Middle School.
Co-founder · build, prove, understand
Three Pacific Studio products show how AI can support the software lifecycle: ship the work, check the proof, and read the ideas that shape it.
Build
Issue-driven orchestration for the full software lifecycle, with Claude Code, Codex, and Gemini CLI working across design, development, testing, and deployment.
Prove
AI-assisted formal verification that generates proof obligations from source code and checks the proofs by machine, with an offline workflow for C, Java, Dafny, and Rust.
Understand
An interactive paper reader that combines layout-aware parsing, sentence-level highlights, bilingual reading, and answers that point back to the source.
AI education · free and open
A good tutor is still a luxury. I want every child to understand why math works, whatever their family can pay and whichever language they speak. So I create lessons, starting with math, that are free to watch and free to reuse, using AI tools I release as open source.
Learn
Short K–12 math videos in English and Chinese. Each answers one “why” and draws it so you can see it. Organized by stage rather than school year, with the matching year in China, the US, the UK, and Singapore.
Try
The same stage-based approach for the other K–12 subjects, from Chinese and English to physics, history, and computing. Each subject starts from a few core ideas, and every lesson is something to try with your own hands.
Make
The tools behind the videos: a format for describing a teaching scene and a vector animation engine, so any teacher can make videos like these without a studio.
FrontierCS 2, a benchmark I co-authored, is out on arXiv. It tests whether AI agents can do computer science R&D: each of its 48 tasks removes a paper's key contribution from its released code, and an agent passes only by beating the original on hidden workloads. Watch the one-minute video or see the publication entry.
I started Boxi Learn, which takes the stage-based approach of Boxi Math to other K–12 subjects. It has 109 hands-on lessons across nine subjects so far, from Chinese and English to physics, history, and computing, in English and Chinese. See Education equity on the home page.
We released CXBench, a benchmark built with Genesys that scores LLM agents in regulated customer-experience workflows on both the business outcome and the required process. Across seven models, 80.1% of outcome-passing runs failed a process check. The paper is now on arXiv; see the research summary.
BlogMore time for coaching, a tutor that is always there
What changes when olympiad coaches and students work with AI: better preparation and problem selection for teachers, and help that is available whenever a student is stuck.
