Results

assess() returns an AssessmentResult. The library uses a common response format so applications can read the same fields regardless of assessor. Check IDs belong to the selected assessor; scores are present only when that assessor supplies them.

A check answers one question, such as whether licence information is present. A metric groups checks about one aspect of FAIR. The tests field holds individual checks; metrics holds their group results.

Read the result

Field	Meaning
`tests`	Each check's result, points, explanation and the information used.
`metrics`	Results for groups of checks. A check's `metric` matches the group's `id`.
`coverage`	Counts of decided checks, undecided checks, errors and checks that do not apply.
`diagnostics`	Messages about problems with the supplied metadata.
`status`	Whether the assessment finished normally or encountered an error.
`raw`	The selected assessor's output from the offline run.
`profile`	The chosen assessment configuration and its version.
`provenance`	The software and data versions used, and a checksum identifying the input.
`schema_version`	Version of the library's response format; currently `1`.

status="completed" means the assessment finished. Check tests and diagnostics to see whether anything failed or could not be assessed. completed_with_errors means a check could not finish because of an error.

A checksum is a value calculated from contents to detect changes. The recorded versions and checksums help identify exactly what was assessed and with which rules.

Inspect results and messages

for check in result.tests:
    print(check.id, check.outcome, check.message)

for diagnostic in result.diagnostics:
    print(diagnostic.code, diagnostic.message)

One check from the quickstart

{
  "id": "FsF-F1-01MD-1",
  "outcome": "indeterminate",
  "score": null,
  "level": null,
  "metric": "FsF-F1-01MD",
  "reason_code": "missing_evidence",
  "message": "No conclusive evidence supplied.",
  "evidence": []
}

This check validates metadata_url. The quickstart does not supply that argument.

Outcomes

Outcome	Meaning
`pass`	The check's requirements were met.
`partial`	Some requirements were met.
`fail`	The check ran and its requirements were not met.
`indeterminate`	The library cannot decide from the information available.
`not_applicable`	The check does not apply.
`error`	An error prevented the check from finishing.

coverage.evaluated counts pass, partial and fail. An indeterminate check receives no score and is not counted as a zero-point failure. Read its reason_code and message for the reason, such as missing information or a check that cannot run offline.

A failure in raw can appear as indeterminate in tests when the information needed to decide was unavailable offline. Use tests and coverage to determine what the assessment established.

Scores

score and level may be null. Champion returns outcomes without numerical scores or maturity levels; its metric summaries are not benchmark scores.

Where supplied, a score contains:

observed_earned: points awarded by the checks that could run.
maximum: the maximum points for the check or metric.
complete: whether enough information was available to determine the full score.
percent: the percentage, calculated only when the score is complete and maximum is greater than zero. Otherwise it is null.

Use the returned metric scores. Assessors can combine their checks in different ways, so adding up individual check points may give the wrong total.

An incomplete score describes only the checks that could be decided. Display it with that limitation instead of presenting it as a complete FAIR percentage. Where supplied, level records an additional rating defined by the assessor.

An incomplete metric from the quickstart

{
  "id": "FsF-F1-01MD",
  "outcome": "indeterminate",
  "score": {
    "observed_earned": 0.0,
    "maximum": 1.0,
    "complete": false,
    "percent": null
  },
  "level": null,
  "principles": ["F1"]
}

Information used by a check

The evidence field points to the supplied information used by a check. Each entry includes its name, checksum and location. For example, /metadata refers to the metadata document you supplied.