Build understanding.
Then put it to the test.
AI education for technical readers. Connecting research ideas to examples you can inspect, question, and use in engineering practice.
A passing test suite.
An incorrect function.
This Python function is meant to return the largest value in a non-empty list of integers. Two tests pass. Before adding a third, predict an input that could expose a bug.
You will practice: identifying an untested input class and distinguishing test success from a correctness claim.
Prerequisite: basic Python. This is a deterministic teaching example, not a live AI model or a reproduction of a research benchmark.
# Return the largest number
def largest(numbers):
best = 0
for n in numbers:
best = max(best, n)
return best[2, 7, 4]7Pass[0, 3, 1]3Pass[-8, -2, -5]-2Fail: 0All tests pass. Is the function correct?
What changed
when the test changed?
- State the contract.
The answer must be an element of the non-empty input list and must be at least as large as every other element.
- Challenge the assumption.
Initializing the result to zero works on these positive examples. It fails when every input is negative.
- Improve the implementation and the tests.
Initialize from the first element, then test negative values, duplicates, zero, and single-element lists. Define empty-input behavior explicitly.
- Keep the conclusion proportional.
Broader tests provide stronger evidence. They do not establish that every possible behavior of a complex AI system is correct.
See a corrected implementation
def largest(numbers):
if not numbers:
raise ValueError("expected a non-empty list")
best = numbers[0]
for n in numbers[1:]:
best = max(best, n)
return bestThis contract is for lists of integers. Other input types need their own specification.
From example to research
Explore the researchThe same question becomes more consequential when tests are used to score coding agents. UTBoost studies rigorous evaluation on SWE-Bench; SWE-ABS examines inflated success rates on test-based benchmarks.