Quality assurance and quality control

<accesscontrol>members of projects</accesscontrol>

This page is a knowledge crystal of subtype method. The page identifier is Op_en2089
Moderator:Nobody (see all) Click here to sign up.
Upload data {{#opasnet_base_link:Op_en2089}}

Quality control measures are needed to ensure that the risk assessments are reflecting the current best scientific knowledge and the state-of-the-art methods. Quality control will impact the process of making an assessment, as well as the contents of the product. Assessments that fulfil the quality criteria at every part are called full open risk assessments. If an assessment fulfils the criteria only partly, it is called a draft open risk assessment.

Four different quality controls of the content

The content of an assessment is evaluated for each object (assessment, variable) separately.

A systematic literature reviews are performed for

• the data sub-attribute of each variable
• the causality sub-attribute of each variable
• the definition attribute (the variable list) of an assessment [is this possible?]

A critical evaluation using open participatory discussion is performed for

• the scope attribute of the assessment
• the analyses sub-attribute of the assessment
• the conclusion sub-attribute of the assessment
• the scope attribute of each variable [how is this different from the definition attribute of an assessment?]

A technical quality check is performed for

• the results sub-attribute of the assessment
• the unit sub-attribute of each variable
• the formula sub-attribute of each variable

An uncertainty analysis is performed for

• the result attribute of each variable

Systematic literature review

A systematic literature review is performed according to the rules of Cochrane reviews.

Open participatory discussion

An open participatory discussion is performed according to a pragma-dialectical argumentation theory.

A discussion is considered saturated when [possible suggestions for a systematic criteria]

• explicit comments has been received from all identified stakeholders or their representatives
• more than 300 independent contributions have been received
• the new contributions do not bring substantive information to what exists already

Technical quality check

A technical quality check is performed by peer review by two experienced experts of the methodology used.

Uncertainty analysis

For a variable, the data, causality, and formula subattributes result in a probability distribution that in theory reflects the current understanding of the uncertainty about the variable. However, this is only true for variables that have gone through the whole quality control process. Ucertainty analyses need to answer the following question about the level of uncertainty:

• Which of the following is true (about the absolute amount of data used)?
  • The data used is only a placeholder for illustration
  • The data used is based on some understanding of the issue, and it is a reasonable guess
  • The data used is based on reasonable amount of information about the expected value, but inadequate data about uncertainties
  • The data allows for a reasonable estimate of the full probability distribution.
  • The data is extensive and it is unlikely that even large amounts of new data would change the current estimate substantially.

In addition, draft risk assessments should answer the following question:

• Which of the following is true about the data used (in relation to the amount of existing data)?
  • No existing data was used
  • Only some general knowledge was used
  • A few easily available information sources were used
  • A non-systematic literature review or an informal expert elicitation was used.
  • A systematic literature review or a formal expert elicitation was used.

See also

• Measuring Author Contributions to the Wikipedia
• A Method for Measuring Co-authorship Relationships in MediaWiki