Scientific validation

What validation compares

Validation compares the verified values with a reference set that a person built by hand. It answers one question: on papers where the right answer is known, how often was it found?

A reference value can match only a value from its own article, so a coincidence between two papers cannot raise the score. Numbers are compared within a tolerance the set declares, because two people copying a table to different decimal places have not disagreed.

Synthetic and real sets stay apart

Each set is declared as synthetic or real, and the two are reported separately and labelled everywhere. A score on a synthetic set shows that the process works; it says nothing about accuracy on real papers.

Building a set

Each step is explicit, so that the set can be trusted.

• Create the set and say whether it is synthetic or real.
• Upload the reference file. It is stored unchanged.
• Confirm which column means what. Nothing is matched by guesswork.
• Import, then run the comparison. Results are stored for each run so that runs can be compared.

Reading the result

Results are kept with the run that produced them, so a later run gives you a comparison and does not replace the earlier one. The size of the set is shown with the score: a high score on twelve papers is a high score on twelve papers.