Testing Experience Issue 13, March 2011

The testing by industry issue

The issue. Testing Experience was a free quarterly English magazine for professional testers, published in Berlin by a testing consultancy and training company and edited by Jose Diaz. Issue 13, dated March 2011, opened the magazine's fourth year. It followed the December 2010 issue on open source testing tools and was the last of the run of themed issues from 2009 to 2011 that began with security testing and agile testing.

What it covered. The theme was testing by field of application, sometimes called sector or industry testing: how the subject being tested determines what has to be tested, how deeply and with what evidence. In practice that means questions such as which failures would cause harm rather than inconvenience, what an external assessor expects to see written down, and how long test evidence has to be kept after release. This page does not list which industries the issue covered or name its articles and authors; the theme is what is known. It suited the magazine's readership. Testing Experience circulated mainly in a European market where testing in regulated industries is a large part of the profession, and many of its readers worked under a sector standard rather than a general one.

Industry testing, 2011 to 2026

Industry-specific testing in 2011. For most European readers in 2011, testing in an industry meant testing under that industry's own standard: medical device software, automotive, rail and finance each had theirs, with its own vocabulary, documentation demands and audit expectations. A tester moving between sectors had to learn the process again. The publisher was active in the community at the same time: it ran Knowledge Transfer tutorials for testers and managers, including one on agile program management in early 2011, and it was a media partner of events such as Eclipse Testing Day.

How the practice changed, 2011 to 2026. What changed by 2026 is the amount of shared ground. ISO/IEC/IEEE 29119 gave the sector standards a common test process to reference, and the TMMi model, maintained by the TMMi Foundation and now at release 1.3, became the maturity framework that regulated organisations most often assess against. Its process areas (test policy and strategy, test planning, test monitoring and control, test design and execution, and test environment at level 2; peer reviews and non-functional testing among the level 3 areas) map onto what an auditor asks to see, whatever the sector. Risk-based testing became the common method for deciding where effort goes. The sector standards still exist and still differ in detail, but a test manager who knows the common process and the maturity model can now move between industries far more easily than in 2011.

Where to read about it now

Where to read about it now. A reader interested in the issue 13 theme usually wants to know two things: how a regulated industry decides what must be tested and to what depth, and how it proves that to an assessor. The risk-based testing guide answers the first. Its 5 by 5 matrix (Figure 1) scores likelihood times impact, and the table under allocating effort by risk class maps each class to depth, techniques and share of effort, with a six-feature worked example. The TMMi guide answers the second. Its table lists the sixteen process areas of TMMi release 1.3 by maturity level, and its section on how a formal assessment runs describes what an assessor asks for.

Where security is the regulated property, as it increasingly is in finance, health and connected vehicles, the security testing guide's five-type table sets out which kind of test answers which question. A dedicated guide on testing in regulated industries is planned for this site; until it exists, the risk-based testing and TMMi guides are the honest continuation of the theme. The issue index lists all 28 issues of the magazine, including the neighbouring issues of 2010 and 2011.

Then and now

Guides that continue this subject

• Test ManagementRisk based testing: scoring, effort allocation, worked example
• Test ManagementTMMi test process improvement: the five maturity levels and what an assessment involves
• Testing TechniquesSecurity testing: types, OSSTMM and where it fits in delivery

Back to top