The Test Pyramid: Building an Automated Testing Strategy

Automated tests are how agile teams keep moving quickly without breaking what already works. But "write more tests" is not a strategy. A team can have thousands of tests and still wait an hour for feedback, chase random failures and miss the bugs that matter.

The test pyramid is the best-known model for thinking about this. It is also one of the most argued-about, with rival shapes such as the testing trophy and the honeycomb. This guide explains the pyramid, the layers it describes, the trade-offs between them, and how to build a strategy that fits your system rather than a diagram.

For how tests are written in a test-first rhythm, see our guide Test-Driven Development Explained. This guide focuses on the mix of tests across a whole suite.

Where the Pyramid Came From

Martin Fowler writes that most people know the test pyramid through Mike Cohn, who described it in his 2009 book Succeeding with Agile as the "Test Automation Pyramid." According to Fowler, Cohn first drew it in conversation with Lisa Crispin in 2003 to 2004.

Fowler summarizes its core argument: tests that run end to end through the user interface tend to be brittle, expensive to write and time-consuming to run, so you should do much more automated testing through unit tests. In his Practical Test Pyramid article, also published on Fowler's site, Ham Vocke boils the model down to two rules: write tests with different granularity, and the more high-level you get, the fewer tests you should have.

The Layers and Their Trade-Offs

Names vary, but most versions describe three broad layers.

• Unit tests check a small piece of the program, such as a function or class. A 2015 Google Testing Blog post by Mike Wacker describes them as fast, reliable and good at isolating failures; when one fails, you only need to search that small unit to find the bug. Their weakness is that units can each work alone yet fail together.
• Integration tests check that a small group of components work together. Wacker describes an integration test as taking a small group of units, often two, and testing their behavior as a whole. They catch problems unit tests miss, at the cost of more setup and slower runs.
• End-to-end tests exercise the whole system, often through the user interface, the way a real user would. They are the closest to real behavior, but Wacker lists their practical problems: you wait for the whole product to be built and deployed, flaky results are common, and a failure could come from anywhere.

Teams with many services often add contract tests between those layers. Vocke describes how each interface between services has a provider and a consumer, and contract tests check that both sides keep to the agreed interface without running the whole system.

Proportions: Useful Guide, Not a Rule

Google has published two rough ratios. Wacker's post says that, as a good first guess, Google often suggests a 70/20/10 split: 70 percent unit, 20 percent integration and 10 percent end-to-end tests, adding that the exact mix will differ by team. The book Software Engineering at Google, in a chapter by Adam Bender that is available free online, gives a rough guideline of about 80 percent narrow unit tests, 15 percent medium-scope integration tests and 5 percent end-to-end tests, counted by number of test cases.

Treat these as starting points. The same chapter says every team's mix will be a little different. Fowler also notes an important exception: the pyramid assumes broad tests are slower and more brittle than focused ones, and he writes that if your high-level tests are fast, reliable and cheap to modify, lower-level tests aren't needed.

Shapes to Avoid

Both Google sources describe two anti-patterns.

• The ice cream cone: many end-to-end tests, fewer integration tests and very few unit tests. The Google book says such suites tend to be slow, unreliable and difficult to work with, and that the pattern often appears in projects rushed from prototype to production.
• The hourglass: many unit tests and many end-to-end tests, but few integration tests in the middle. The book says this results in many end-to-end failures that medium-scope tests could have caught more quickly, and that it often occurs when tight coupling makes it hard to test pieces in isolation.

Trophies, Honeycombs and the Definition Problem

Not everyone agrees with the pyramid. Kent C. Dodds introduced the testing trophy in 2018, following his blog post titled "Write tests. Not too many. Mostly integration." His trophy adds a layer for static analysis and, in line with that slogan, emphasizes integration tests. Others have proposed a honeycomb shape with a similar emphasis.

Fowler's 2021 article on test shapes argues the disagreement is partly about words. "Unit test" has always been murky. Some people mean a "solitary" test that replaces every collaborator with a test double; others mean a "sociable" test that lets real collaborators run. Fowler suggests that what honeycomb advocates call integration tests often sound like sociable unit tests, which makes much of the debate moot. His practical advice: when someone talks about testing categories, ask what they mean by their words.

Google sidesteps the naming issue by classifying tests by size. In the Google book, small tests run in a single process, medium tests on a single machine, and large tests may span multiple machines. The reasoning is that speed and determinism matter more than the label.

Building Your Own Strategy

A few principles hold up regardless of shape.

1. Push tests down. Vocke recommends pushing tests as far down the pyramid as you can, and writing a lower-level test whenever a higher-level test finds a bug that no lower-level test caught. Fowler gives the same advice: replicate the bug with a unit test before fixing it.
2. Avoid duplication. Vocke warns that every test is additional baggage to write, read, maintain and run. Higher-level tests should focus on what lower levels cannot cover.
3. Fight flakiness. The Google book warns that flaky tests erode trust until engineers stop reacting to failures, eliminating the value the suite provided.
4. Keep feedback fast. Wacker describes the ideal feedback loop as fast, reliable and able to isolate failures. Run the quickest tests on every change and slower suites later in the pipeline.
5. Mind coverage metrics. The Google book warns that a coverage target such as 80 percent can turn into a ceiling rather than a floor.

A Hypothetical Example

Imagine a team whose web app has 400 browser-driven end-to-end tests and almost no unit tests. The full suite takes 90 minutes and fails randomly a few times a week. Rather than deleting everything, the team adds unit tests around pricing and validation rules, adds integration tests for the database and payment interface, and keeps a small set of end-to-end tests for critical journeys such as signing up and checking out. Over several months, the team could expect faster feedback, then use its retrospectives to judge whether the new mix is catching the bugs that matter.

Key takeaways

• The test pyramid, popularized by Mike Cohn, favors many fast, focused tests and few slow, broad ones.
• Google has published rough mixes of 70/20/10 and 80/15/5, but both sources say each team's mix will differ.
• Avoid the ice cream cone and the hourglass; both slow feedback and make failures harder to diagnose.
• Debates over pyramids and trophies are partly about what "unit" and "integration" mean, so define your terms.
• Push tests down, avoid duplication, fix flaky tests and keep feedback fast.

Sources

• Martin Fowler, Test Pyramid
• Ham Vocke, The Practical Test Pyramid, martinfowler.com (2018)
• Martin Fowler, On the Diverse And Fantastical Shapes of Testing (2021)
• Mike Wacker, Just Say No to More End-to-End Tests, Google Testing Blog (April 22, 2015)
• Adam Bender, Testing Overview, in Software Engineering at Google
• Kent C. Dodds, The Testing Trophy and Testing Classifications (2021)