Skip to content
TestingExperience

AI Tool Tests

Independent tests of AI tools

27 guides on testing detectors, watermarks, provenance pipelines and disclosure labels.

Open section

Start here, in order

Methods and protocols

How we test 2.0%CI 1.3 to 3.1 False positive rate in the worked benchmark: 20 of 1,000 human-written documents wrongly flagged. Read the protocol
Editorial methodGlossaryReport an errorNo tool maker pays for a test

Software testing

Reference guides for testers

26 guides on planning, automating and designing tests, written to outlive any tool.

Testing Techniques

Sections

Recently updated

Test ManagementTest AutomationAgile TestingTesting TechniquesGuides name no products

Magazine

The magazine, 2008 to 2014

A free quarterly for professional software testers, published in Berlin across 28 issues.

The archive

Archive

Issue pages

28 issues Each issue page states the date and the theme where it is known, and nothing beyond it. The original issues are not distributed.
The archiveAll issuesEditorial methodRecords, not reproductions

About Testing Experience

Who runs testingexperience.com, what changed at the 2026 relaunch, how guides are written and reviewed, the editorial rules and how to send a correction.

AboutUpdated

In short

Testing Experience runs independent, method-first tests of AI tools such as text detectors, AI writing tools and provenance checks, and publishes reference guides for professional software testers on test management, test automation, agile testing, testing techniques and testing AI systems. It also keeps an archive of the magazine of the same name, published from 2008 to 2014. One editor is responsible for every page. There is no advertising, no sponsorship and no cookies.

Propose a test or a guide

Tell the editor what you want tested or explained, and why.

How to contribute

What the site is

Testing Experience is an independent site for people who have to decide whether to trust an AI tool, and for the testers who check such tools. Its test reports state the sample set, the procedure, the tool version and the date, so a result can be rerun. Its reference guides cover test management, test automation, agile testing and testing techniques, and it keeps an archive of the magazine of the same name, published from 2008 to 2014.

The relaunch

The original Testing Experience was a free quarterly magazine from a Berlin testing consultancy, edited by Jose Diaz, which stopped publishing after its 28th issue in December 2014. Testing Experience was relaunched in 2026 by a new, independent owner. This site is not the original publisher, has no connection to that company or its staff, does not hold the magazine's archive, and does not distribute the original issues.

It keeps the name because the name describes what the site is about, and it keeps a page for each well-known issue so that readers who remember the magazine can see what it covered and where the subject lives now.

Who runs the site

Daniel Weber is the editor responsible for every page on the site. He has worked as a tester, test automation engineer and test manager on product and consultancy teams in Germany and the United Kingdom, and writes the guides from that experience and from the standards and books each guide cites.

Every guide and issue record carries his byline and the month it was last reviewed. His editor profile sets out his background and how he works, and he can be reached through the contact page.

How guides are written

Each guide starts from a question a working tester actually searches for, states the direct answer in a quick-answer card, and then earns that answer with a figure or table, a worked example, and sources named so that a reader can check them. Guides are reviewed against ISO/IEC/IEEE 29119, the ISTQB syllabi and glossary, the TMMi model and the books in their sources list. Numbers in worked models are labelled as illustrative where they are not from a named source, with an instruction to replace them with the reader's own data. Tool tests follow a separate, pre-registered protocol and scoring rubric, published on how we test.

A guide begins as a question written the way a tester types it into a search box, and a one-line answer the editor is prepared to defend. The draft is built outward from that answer in a fixed order: the figure or table that opens the page, the worked example with its numbers, the comparison table, the limits, and only then the prose that connects them.

Before publication each guide is checked against one named standard or book: the test management guides against ISO/IEC/IEEE 29119 and Rex Black's Managing the Testing Process, the technique guides against Lee Copeland's A Practitioner's Guide to Software Test Design and the ISTQB syllabi, the agile guides against Crispin and Gregory and against Adzic, and the AI and authenticity guides against the ISTQB AI testing syllabus, the C2PA technical specification and the NIST overview of synthetic content. A claim the check cannot support is cut, or rewritten as an opinion and labelled as one.

How figures are built

Every figure and table is drawn from data that is either cited or declared. Where a figure shows published numbers (the relative cost of a defect by stage, a false positive rate from a named study) the source is in the guide's list and the figure repeats the value without smoothing it.

Where a figure shows a worked example, the counts are invented but realistic, the caption says so, and the arithmetic is shown in the text so that a reader can substitute their own numbers. Figures are drawn as plain vector graphics served by this site, with the values in the markup, so that they read correctly in a screen reader and print without colour.

Editorial rules

  1. 01Guides name no products so they stay true as tools change. Evidence pages name a product only to report what a university, study or vendor has published about it.
  2. 02Test reports name the tool, the version and the date because a result is only useful if you know what produced it.
  3. 03No tool maker pays for a test sees a result before publication, or buys a placement or a better position in a table.
  4. 04No advertising and no sponsorship Some links to tools we have tested may carry a referral code; a link never changes a result.
  5. 05No cookies and no cross-site tracking; the only measurement is cookieless page statistics in aggregate.
  6. 06Every source is named so that a reader can check it.
  7. 07Every page is signed with the editor's byline and the month it was last reviewed.
  8. 08Nothing is written for length and a page that cannot meet its floor honestly is cut.
  9. 09Confirmed errors are fixed within a week and substantive corrections are noted on the page.

What the site will not publish

What we publish

  • Test reports with the sample set, the procedure, the date and the tool version
  • Reference guides on test management, test automation, agile testing and testing techniques
  • Guides that open with a figure or table and a direct answer
  • Worked examples with the arithmetic shown
  • Issue records with the date, the known theme and the guides that continue it
  • Dated notes of substantive corrections on the affected page

What we never publish

  • A verdict without its method, sample set and date
  • Star ratings, best-overall badges or rankings that a test did not measure
  • Claims that a detector, watermark or provenance system is reliable beyond what the test showed, or that any tool defeats one
  • Statements about an issue's contents beyond its known theme and authors
  • Guides that cannot be tied to a standard, a book or a published study
  • Anything written to reach a length
  • Anything sent by a vendor or an agency

The archive

Each issue page gives the date, the theme where it is known and the guides on this site that continue the subject. Nothing is asserted about an issue's contents beyond its known theme, and an author is named only where that is established.

An issue page states only what a reader can check elsewhere: the issue number and date, the theme where it is known, and an author only where that is established. Contributions from people who wrote for or hold the magazine are marked as contributed, so that the two kinds of statement never mix. The rule costs the issue pages some detail, and it is what keeps them honest.

If you were an author or reader of the magazine and can correct or extend an issue page, the contribute page explains how.

Corrections

Errors are fixed within a week of being confirmed, and the page's review month is updated. Substantive corrections are noted at the foot of the affected page.

The workflow is fixed: a report arrives through the contact page with the page, the sentence and a source; the editor confirms it against the source within a week, or replies to say why it does not hold; a confirmed error is corrected, the review month is updated, and a substantive correction is noted at the foot of the page with the date. A correction that changes a figure or a table triggers a re-read of the whole guide before the review month changes.

How a test is run

  1. Decide the depth

    Risk-based testing sets how deep an evaluation goes, with the cost of a wrong flag on a person in the impact column.

  2. Build the evaluation set

    Record where every item came from and how its label was established, in slices of 300 human and 300 machine documents.

  3. Fix the criteria first

    Write the threshold and the pass criterion down before the run.

  4. Pin what is tested

    Name the exact version under test and the environment, and record every input by hash.

  5. Run and keep everything

    Save raw outputs and timestamps, stored write-once.

  6. Report the full matrix

    Publish the full confusion matrix, with every rate given its denominator and a Wilson interval.

  7. Make it rerunnable

    Pin enough that a second team reruns the evaluation to the same rates within the stated interval.

Contact

Use the subject line Correction, Archive or Proposal so the message is sorted correctly. Every message is read by the editor.

Email the editor
Replies
Within a week; proposals within two weeks
Corrections
Fixed within a week of being confirmed
No form
Email only

Back to top