Plausibility test

This page is a knowledge crystal of subtype method. The page identifier is Op_en2383
Moderator:Nobody (see all) Click here to sign up.
Upload data {{#opasnet_base_link:Op_en2383}}

<section begin=glossary />

Plausibility tests are procedures that clarify the goodness of variables in respect to some important properties, such as measurability, coherence, and clarity. The four plausibility tests are clairvoyant test, causality test, unit test, and Feynman test.

<section end=glossary />

1. Clairvoyant test (about the ambiguity of a variable): If a putative clairvoyant (a person that knows everything) is able to answer the question defined in the scope attribute in an unambiguous way, the variable is said to pass this test. The answer to the question is equal to the contents of the result attribute.
2. Causality test (about the nature of the relation between two variables): If you alter the value of a particular variable (all else being equal), those values that are altered are said to be causally linked to the particular value. In other words, they are directly downstream in the causal chain, or children of the particular variable.
3. Unit test (the coherence of the variable definitions throughout the network): The function defining a particular variable must result (when the upstream variables are used as inputs of the function) in the same unit as implied in the scope attribute and defined in the unit attribute.
4. Feynman test (about the clarity of description): If you cannot explain it to your grandmother, you don't understand it well enough yourself. (According to the quantum physicist and Nobel laureate Richard Feynman.)

The specification of variables proceeds in iterative steps, going into more detail as the overall understanding of the assessed phenomena increases. First, it is most crucial to specify the scopes (and names) of the variables and their causal relations. As part of the specification process, in particular the name and scope attributes, the clairvoyant test can be applied. The test helps to ensure the clarity and unambiguity of the variable scope.

Addressing causalities means in practice that all changes in any variable description should be reflected in all the variables that the particular variables is causally linked to. At this point, the causality test can be used, although not always necessarily quantitatively. In the early phases of the process, it is probably most convenient to describe causal networks as diagrams, representing the indicators, endpoints, key variables and other variables as nodes (or boxes) and causal relations as arrows pointing from upstream variables to downstream variables. In the graphical representations of causal networks the arrows are only statements of existence of a causal relation between particular variables, more detailed definitions of the relations should be described within the definition attribute of each variable according to how well the causal relation is known or understood.

Once a relatively complete and coherent graphical representation of the causal network has been created, the specification process for the identified indicators may continue to more detail. The indicators, the leading variables, are of crucial importance in the assessment process. If, during the specification process, it turns out that the indicator would conflict with one or several of the properties of good indicators, such as calibration, it may be necessary to consider revising the scoping of the indicator or choosing another leading variable in the source - impact chain to replace it. This may naturally bring about a partial revision of the whole causal network affecting a bunch of key variables, endpoints and indicators. For example, it may happen that no applicable exposure-response function is available for calculating the health impact from intake of ozone. In this case, the exposure-response indicator may be replaced with an intake fraction indicator affecting both the downstream and upstream variables in the causal network in the form of e.g. bringing about a need to change the units the variables are described in.

The description, unit and definition attributes are specified as is explained in the previous section. The unit test can be applied to check the calculability, and thus descriptive coherence, of the causal network. When all the variables in the network appear to pass the required tests, the indicator and variable results can be computed across the network and the first round of iteration is done. Improvement of the description takes place through deliberation and re-specification of the variables, especially definition and result attributes, until an adequate level of quality of description throughout the network has been reached. The discussion attribute provides the place for deliberating and documenting deliberation throughout the process.

Clairvoyant test

Scope

What are the questions that need to be explicitly answered before the object is considered to pass the clairvoyant test?

Result

• In population variables, it must be clear whether it is about the population mean or about a random individual. This is a critical difference, as the former ignores any variability, while in the latter the variability is explicitly addressed as probability. On the other hand, mean values are more robust and easier to estimate reliably. In the end, the critical issue is whether we are interested in an assessment on a population level or an individual level.
• The time dimension should be defined if relevant. A variable can be for a fixed date. However, I encourage to use variables like 'Current body weight of a random 0-2-year-old Finnish child'. This way, the variable itself remains useful in new assessments; of course, the data used to find a result gradually becomes out-of-date, but it is only data.