Data-yield scoring

A way to order the work

Scoring estimates whether a paper contains numbers worth extracting. It does not judge the quality of the science. It orders the included papers so that effort on full text goes where it is most useful, and a low score never excludes a paper by itself.

Only papers that screening did not exclude are scored.

Four questions

Four things are scored separately and added up. The scoring rules and thresholds come from the study configuration, and a copy is kept with each run.

• Target plant protein: is the tested material one of the proteins in scope?
• Target property: is a property in scope actually measured?
• Conditions or treatment: are the processing conditions reported?
• Extractable numbers: are there values in the text, a table or a figure?

Thresholds

The total places each paper in one of four groups: exclude, review, extract or high priority. The full-text stage tries to obtain papers marked extract or high priority; papers marked review wait for a person to decide.

Thresholds belong to the project. A higher threshold makes the study narrower and faster; a lower one makes it broader and slower. Either choice is recorded.

Repeated scores that disagree

A project can score each paper more than once and use the average. When repeated scores differ by more than the project allows, the paper is flagged and every individual score is kept.

A disagreement flag deserves more attention than a low score. It usually means the paper sits on a boundary your criteria have not settled, which is better fixed once in the criteria.