Testing Experience Issue 12, December 2010

The open source tools issue

The issue. Testing Experience was a free quarterly English magazine for professional testers, published in Berlin by a testing consultancy and training company and edited by Jose Diaz. Issue 12 is dated December 2010 and closed the magazine's third year. It followed the September 2010 issue, which included a research article, and it was one of the run of themed issues from 2009 to 2011 that gave the magazine its character: security testing, agile testing, open source tools and, in March 2011, testing in specific industries.

What it covered. The theme was open source testing tools. This page does not name the tools the issue discussed or list its articles and authors; the theme itself is what is known. It is easy to see why the subject earned a whole issue. In 2010 the open source testing tool landscape was changing fast, comparative overviews were rare, and a free magazine that surveyed the field was worth reading for any team choosing tools on a small budget or trying to justify moving away from an expensive commercial suite.

Tool choice, 2010 to 2026

Tool choice in 2010. Many large organisations still standardised on commercial test management and automation suites, bought centrally and licensed per seat. Open source tools tended to arrive team by team, adopted by an engineer who had tried them on a side project. Evaluation was informal: install the tool, automate a few tests, and check that the licence would pass a legal review. The questions a tester asked were whether the tool worked with the application, whether anyone else was using it, and which licence family it carried.

How the practice changed, 2010 to 2026. Between December 2010 and 2026 the questions a tester asks of an open source tool changed. In 2010 the questions were whether the tool worked and which licence it carried; the Open Source Definition, unchanged in substance since 1998, settled what counted as open source, and licence family was the main risk. Since then the risk has moved to the supply chain: whether the project is maintained, who has merge rights, whether releases are signed and whether the dependencies the tool pulls in are themselves safe. The Open Source Security Foundation, formed in 2020, publishes the criteria (maintainer count, release signing, dependency update policy) that a 2026 evaluation checks and a 2010 one did not.

Where to read about it now

Where to read about it now. A reader looking for what issue 12 offered, a survey of open source testing tools covering what exists, which are mature and what the licences allow, should start with the open source testing tools guide. It deliberately describes tool categories, maturity signals and licence families rather than naming products, so that it stays true as products come and go. Its Table 1 sets out eight categories with what each does, the maturity signals to look for and the typical licence family. Its ten-point evaluation checklist runs from release cadence (a release in the last six months) and issue health (median age of open issues under 90 days) through contributor spread, breaking-change discipline, documentation, language fit, CI integration, extensibility and licence compatibility to exit cost. Half of those ten points concern project health rather than features, which is the 2010 to 2026 shift in one list. The section on how to run a bake-off is the method for choosing between two finalists.

Choosing a tool is only half of the job. The test automation strategy guide covers what to automate first and in what order, with a scoring table for candidate regression areas and a 12-month roadmap. The test automation pyramid guide explains why the unit-level category dominates adoption and how the layers above it should be sized. The Test Automation hub holds the rest of the material on tools and automation.

Then and now

Guides that continue this subject

• Test AutomationOpen source testing tools: categories, licenses and evaluation
• Test AutomationTest automation strategy: what to automate and in what order
• Test AutomationTest automation pyramid: ratios, costs and when to break it

Back to top