Quality assurance and quality control

From Opasnet
Revision as of 14:22, 4 November 2009 by Jouni (talk | contribs) (some edits, half way)
Jump to navigation Jump to search


Quality control measures are needed to ensure that the risk assessments are reflecting the current best scientific knowledge and the state-of-the-art methods. Quality control will impact the process of making an assessment, as well as the contents of the product. In open assessments, this is especially challenging, as the contents of the product may be changed by any participant. Special procedures are needed to make this possible but ensure the assurance of the quality at the same time.

Scope

Definition

Data

Dependencies

Result

Four different quality controls of the content

The content of an assessment is evaluated for each object (assessment, variable) separately.

A systematic literature reviews can be performed for

  • the data sub-attribute of each variable
  • the causality sub-attribute of each variable
  • the definition attribute (the variable list) of an assessment [is this possible?]

A critical evaluation using open participatory discussion can be performed for

  • the scope attribute of the assessment
  • the analyses sub-attribute of the assessment
  • the conclusion sub-attribute of the assessment
  • the scope attribute of each variable [how is this different from the definition attribute of an assessment?]

A technical quality check can be performed for

  • the results sub-attribute of the assessment
  • the unit sub-attribute of each variable
  • the formula sub-attribute of each variable

An uncertainty analysis can be performed for

  • the result attribute of each variable

Systematic literature review

A systematic literature review is performed according to the rules of en:Cochrane reviews.

Open participatory discussion

An open participatory discussion is performed according to a pragma-dialectical argumentation theory[1].

A discussion is considered resolved, when the outcome of the discussion is synthesises based on all arguments in the discussion, and the resolution is incorporated into the relevant place in the main page. It should be noted that a resolution does not mean that the discussants agree on the topic; it means that they agree on what the agreements and disagreements are, and this is reflected in the discussion as a lack of new contributions.

Technical quality check

A technical quality check is performed by peer review by two experienced experts of the methodology used.

Uncertainty analysis

The result of a variable is a probability distribution based on all information in the definition, and in theory it reflects the best current understanding of the uncertainty about the variable. However, this is only true for variables that have gone through a process where a large group of scientific experts have contributed and showed their acceptance about the validity of the estimates.[2] Thus, the probability distribution, as a subjective probability, is always subject to acceptance by individuals that are considered experts, and ultimately, by myself.

Development, evaluation, and application of environmental models

There is an EPA report about this topic [3]

"This Guidance recommends best practices to help determine when a model, despite its uncertainties, can be appropriately used to inform a decision. Specifically, it recommends that model developers and users: (a) subject their model to credible, objective peer review; (b) assess the quality of the data they use; (c) corroborate their model by evaluating the degree to which it corresponds to the system being modeled; and (d) perform sensitivity and uncertainty analyses. Sensitivity analysis evaluates the effect of changes in input values or assumptions on a model's results. Uncertainty analysis investigates the effects of lack of knowledge and other potential sources of error in the model (e.g., the “uncertainty” associated with model parameter values) and when conducted in combination with sensitivity analysis allows a model user to be more informed about the confidence that can be placed in model results. A model’s quality to support a decision becomes known when information is available to assess these factors."

Page control

Quality control in Heande is based on different levels of control for different pages, depending on the need. These are briefly described below.

  1. User control level
    • This level applies to all new pages by default. A page will be on this level until there are specific needs to apply a more strict protection (see below).
    • If users want to edit pages, they must register and give their real name either in the user name or in their own user page.
    • Edits must be done according to the good practice of editing. However, this is not protected by technical means.
    • Users may also use the contribution tool if they do not want to edit the page content directly.
    • Moderators will check for new edits and remove material that is against policies or guidelines.
    • For major edits, a discussion must be held at the respective Talk page before the edits are made.
  2. Page protection level
    • This level applies to pages that are so a) large, b) complex, c) interconnected, or d) contradictory that it is (in the moderator's view) likely that the page will suffer damage from poor edits. The moderator should give motivation why she thinks that this particular page is under greater risk of damage than other pages.
    • Edits by other users than the moderator of the page are technically prevented.
    • Users contribute using the contribution tool. In this tool, contributions are handled as individual pieces of text that will be given a unique identifier and that cannot be edited afterwards (except that moderators will remove contributions that are against policies or guidelines).
    • The moderator of a protected page will take all new substantive material from the contribution and locate it into a relevant place in the talk page into a formal or informal discussion. The conclusions of the discussions will be inserted into the main page.
  3. Peer review level
    • This level is used with topics that have a high scientific or policy importance. The content is worth a laborious formal peer review. In this case, it is important that the page is not changed after the peer review without making it very clear to the reader what things have changed and why. The reader must be able to trust that the contents of the page fulfils strict scientific quality standards.
    • The page goes through a formal peer review by experts of the field. Depending of the content the formal processes may be different. Some formal procedures for ensuring quality of a scientific article are listed below, and they may be used:
      • CONSORT statement - All randomised controlled trials [2]
      • QUOROM statement - All systematic reviews [3]
      • EVEREST statement - All economic evaluations [4]
      • STARD statement - All diagnostic research papers [5]


Tiukka tietosuoja: pitää tehdä yhteistyötä suomalaispankkien kanssa: heanden henkilötunnistus voidaan hoitaa pankkien kautta. D↷


See also


References

  1. ↑ van Eemeren and Grootendorst, 2004
  2. ↑ In a previous version we promoted the idea of explicating the absolute amount of data used for deriving a probability distribution, but we abandoned it.
  3. ↑ The Council for Regulatory Environmental Modeling: Draft Guidance on the Development, Evaluation, and Application of Regulatory Environmental Models. Models Guidance Draft - November 2003. [1]
Retrieved from "https://en.opasnet.org/index.php?title=Quality_assurance_and_quality_control&oldid=11631"