This page is a encyclopedia article. The page identifier is Op_en2377
Moderator:Nobody (see all) Click here to sign up.

Upload data {{#opasnet_base_link:Op_en2377}}

Pyrkilo is a name by which the collaborative assessment method-tool compound, currently known as open assessment method and Opasnet workspace, used to be called in early phases of its development. See open assessment and Opasnet.

This page contains a compilation of descriptive texts on Pyrkilo. It considers both the methodology of (risk) assessment and the tool to support the use of the method. These texts primarily have a historical purpose, up-to-date descriptions and discussions on the topic are found under the terms open assessment and Opasnet.

The current definition of pyrkilo is the following:

Pyrkilo theory is a theory that attempts to answer the following research question: How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?

Pyrkilo method is a method that applies the contents of the pyrkilo theory in practical risk assessments.

Open risk assessment is a risk assessment (covering both the process of making it, and the final product) that has been carried out utilising the pyrkilo method.

Some of the terms used are (tentatively) defined below.

Scientific information = information that can be obtained by using the scientific method ^[1]
Value judgement = preference or value a person or a group assigns to a particular condition or state of the world
Societal decision-making = process of reaching conclusion about actions that can be taken by the institutes of the society, or the members of the society in general, in aim to increase societal values
Societal values = value judgements that are so widely accepted in the society that they can be thought of representing a large part (not necessarily majority) of the members of the society
Improving decision-making = practices that, on average, lead to decision-making that, on average, leads to outcomes with higher societal values than an alternative practice
Open participation = a process where a non-organised, non-fixed, non-restricted group of individuals can get information from and contribute to the outcome of the process (in this context, a risk assessment report).

Pyrkilo is currently most actively used in the field of risk assessment. Pyrkilo as a method differs from the other risk assessment methods in three essential aspects:

The risk assessment product IS a causal diagram where all elements are described as variables using a specified structure. This sets structural limits to the contributions. In traditional risk assessments, the assessment product typically is a report using a structure of a book, not that of a causal diagram. The causal diagrams are used for illustration only, or within models that calculate some results to the assessment report.
Formal argumentation (according to the pragma-dialectical argumentation theory) is used as the primary means to describe and resolve any disputes about scientific or valuation issues within the assessment. In traditional risk assessments, there is guidance to describe major disputes, but there are no structural rules for this. In addition, many disputes are (implicitly) resolved using conventions without challenging the foundations of the convention.
The principles of mass collaboration are used in the risk assessment work. In practice, anyone is allowed to contribute (i.e. bring in information and value judgements that are within the scope of the assessment), unless there are explicitly specified reasons to restrict participation in the particular case. The traditional risk assessments are expert-driven exercises with possibly some public hearings about the scoping or results, but little involvement of the stakeholders to the actual making of the assessment.

References

↑ Popper, Karl R. 1935. Logik der Forschung. Julius Springer Verlag.

Intarese method is a method for performing environmental health risk assessments in an efficient way to produce high-quality quantitative assessments for improving societal decision-making. The aim is to improve the making of risk assessments by offering guidance, tools, and examples. Another aim is to improve the availability and usability of the assessments by offering assessments and other contents for anyone to read, learn, and apply. The help pages that describe parts of the method in detail belong to the Category:Intarese method, and links to them can also be found from there. D↷

The ultimate purpose is to improve societal decision-making.

← The Intarese method helps in this by providing relevant information in the form of quantitative assessments about risk situations.

← In general, the information provided is about predictions on the impacts of possible decisions on some outcomes that have a societal value. However, the scope of the information can be narrowed in a particular case for example in the following ways:

The only decision considered is business-as-usual. This results in estimates of the burden of the current situation.
Outcomes considered are not explicitly valuated, but implicitly it is assumed that they relate to something that has intrinsic value. For example, pollutant concentrations can be estimated, although it is the related health impact that has the intrinsic value, and the concentration is only a proxy of this.

The Intarese method is closely related to the pyrkilo method that is being developed in KTL, Finland. Template:Pyrkilo definition

Basic ideas of the method

The structure of a risk assessment is completely rethought compared with the established NRC (1983) concept described in the "Red Book":

A risk assessment consists of variables. Each of them describe a particular piece of reality. Importantly, variables are independent of the risk assessment they belong to, and each variable can be used is several risk assessments.
A variable is also independent of all variables except those that are mentioned in its definition attribute. (These are called the upstream variables.) Thus, the result of a single variable can be calculated if its definition is described in a precise enough way and the results of the upstream variables are available.
Thus, a risk assessment does not consist of a single risk assessment model to calculate the outcome results. Instead, it consists of several integrated models, one for each variable needed in the assessment. This approach has several important implications.
1. The risk assessment work can be effectively decentralized after the essential variables have been identified.
2. The actual computing tasks can also be decentralized. They can be performed using any relevant software, as long as there is a standardized interface and information structure between the variable description system and the result computing system. This interface is called Variable transfer protocol (VTP). This also makes it possible that the model and computing effort of a variable can be very large, and still it does not cause severe load on the toolbox that maintains the risk assessment work (see below).
3. The variable results, if collected in a centralized way, is a valuable database for making further risk assessments and utilising the information in decision-making.
4. The structure gives freedom to further development, as the different parts of the whole system can be operationalized using the software and programs best suited for that purpose, as long as the interfaces between them follow the VTP.
To be able to utilise the risk assessment method, a toolbox must be developed. The toolbox is a collection of software and functionalities that provide the guidance, information structure, and data for the participants of a risk assessment. The toolbox functionalities are described in more detail below.

The main parts of the toolbox

Intarese toolbox is a software for applying the Intarese method. Details of is are shown on the graph below. Click on a detail to get more specific information.

Template:Toolbox and its functionalities

The phases of a risk assessment

The risk assessment work has six phases. The work during a particular phase is always built on the results of the previous phases. The phases are not clear-cut periods in time, because the work is iterative in nature, and the previous phases must frequently be revisited when the development of the product and feedback brings in new understanding. Thus, the next phase is usually started when the previous phase is still a draft at least in some aspects.

The phases of risk assessment are called:

Scoping of the assessment. In this phase the purpose of the assessment, the question(s) asked in the assessment, intended use of the output, the temporal and spatial boundaries of the scope of the assessment, and the participatory width of the process are defined and described.
Applying previous information about the issues being assessed. In this phase the existing information that is available e.g. in the encyclopedia, as existing variables from previous assessments, or as Classes (general properties shared by a group of variables) are sought out and applied.
Drawing a causal diagram. In this phase the decisions, outcomes, and variables of importance related to the assessment are described in the form of a sketch of a causal network diagram.
Designing variables. In this phase the variables included in the assessment (the causal network diagram) are described more precisely, including defining causal relations between them. This phase may also include definition of quality criteria and plans for collecting the necessary data or models to estimate the results of the variables. It should be noted that previously this phase was considered to apply to the assessment as a whole. Now we have realised that in practice the design work happens separately for each variable (although usually approximately at the same time). The same applies to the next phase, execution.
Executing variables and analyses. This phase is actually about collecting the data needed, executing the models described in the Definition attributes of different variables, and storing the results in the result database. Assessment-specific analyses such as optimisation, decision analysis, value-of-information analyses and so on are carried out.
Reporting In this phase, the results of indicators and assessment-specific analyses are communicated to the users of the outputs. The results are discussed and conclusions are made about them, given the scope of the risk assessment. The communication includes necessary background information.

The phases 1-3 are collectively called the issue framing phase.

Structural elements of the method

Help:Collaborative workspace Workspace is a working environment for participants of risk assessment work, and readers of the assessments.
Help:Risk assessment structure is the information structure of the Intarese method.
Help:Variable
Help:Decision variable
Help:Value judgement variable
Help:Class
Help:Data gateway
Help:Encyclopedia

Crucial processes when using the method

Practical guide on stakeholder involvement is practical guidance for how to involve stakeholders and get their contribution.
Help:Mass collaboration tells about the role of stakeholder involvement in the Intarese method, including theoretical foundations.
Help:Variable transfer protocol
Resolving disputes

Introduction

This page provides an overview and brief descriptions of all the specific methods belonging to the Intarese general method. The specific methods are grouped under different categories according to the scope of influence that the methods are intended for. This page functions also as an index providing links to more detailed descriptions and guidance of each particular method and this page can be used as the main page or entry point to all Intarese general method relevant pages. All these relevant pages are also categorized to the Intarese general method category. If a specific method is included in the list, but there is no link, it means that the method has been identified as important, but the method is still lacking a description.

The list of specific methods is not yet fully comprehensive and the groupings of the specific methods under different categories may not yet be quite perfect. However the page already provides a good overview to the Intarese general method and illustrates the different roles of different specific methods in carrying out a full integrated risk assessment.

This page also describes briefly the seven fundamental new elements that have been identified during the method development work in Intarese project. These seven elements are also included in the Intarese general method. These elements are issues that, when used together, result in completely new kinds of risk assessments and new ways of making better risk assessments. None of these seven elements are completely new as such, but currently they are not commonly used in a tight combination in carrying out risk assessments. These elements are described in brief below. More thorough descriptions are included in the descriptions of the specific methods they are related to.

For clarity, some definitions:

Phase = a certain stage in the risk assessment process
Step = a certain stage in the causal chain
Variable = an object described as a part of the causal chain description
Indicator = a variable which is of special interest

The seven plus one fundamental new elements in the Intarese general method

1. Purpose: Better societal decisions
Risk assessments should always be done for a purpose. When the purpose is identified and kept clear in mind and preferrably explified and made public, it helps to guide the process in producing a desired kind of assessment product. The ultimate general purpose is to improve societal decision making by providing good descriptions of chosen parts of reality for the use of the decision-makers. The process of describing the purpose is now considered as an essential and integral part of the method. Proper identification of the purpose of risk assessment crucially affects the assessment process and the content and essence of the final product. For further details, see General properties of good risk assessments.

2. Causality
The Intarese method is based on the full-chain approach. This implies that the assessment products produced in the assessments should be causal network descriptions that cover the relevant phenomena from emissions to exposures to health effects and their impacts in accordance with the chosen endpoints and purpose. However, it should be emphasized that the method does not only describe issues that are associated with the full chain. It describes those issues that cause effects along the full chain, and it describes how the causes and effects are related. This, of course, makes risk assessment a challenging, or even difficult, process. Strict emphasis on causality, however, should be the way to e.g. estimate the impacts of policies on emissions and consequently to health effects. For further details, see Guidance and methods for indicator selection and specification.

3. Collective structured learning
Science is inherently an area that is based on learning from previous experience, and this is not a new concept. However, collective structured learning means something more than that. It is an idea based on intentionally producing generally applicable pieces of information that can also be used as the building blocks of new assessments. (In discussions within Intarese, these have been referred to as e.g. templates, objects, or general variables, but a common term has not yet been established.) In addition, collective structured learning builds on the principle of collaborative work, which implies that the pieces of information produced in a particular assessment are set commonly available for use in other assessments. What is new in this concept is that there are specific procedures for extracting pieces of information from previous work and assessments, and making them centrally available for others. This enhances the efficiency and effectiveness of the assessments made by applying the Intarese general method. For further details, see Collective structured learning.

4. Value judgements
Risk assessment is about estimating impacts that have positive or negative value judgements attached to themselves or to the determinants that causally affect them. These values must be acknowledged in the process of making the assessments, not only in the risk management phase, otherwise there is a risk of compromising the relevance of the assessment. Combining phenomena of physical reality with the value judgements related to them requires methods to distinguish these two things from each other and bringing the value judgements to explicit scrutiny within an assessment.

5. Variable structure
In order to make coherent descriptions of reality in assessments, there must be a certain clear structure within the assessment. As we also want to produce descriptions that coherent also between assessments, there must be a universal structure for all the assessments. Variables with a certain set of attributes, and linkages between these variables are the universal structure of the assessments. For further details, and the current suggestion of the attributes, see Guidance and methods for indicator selection and specification |. The universal assessment structure is essential for coherent inclusion of causality in assessments, enabling of collective structured learning, collaborative work as well as combining value judgements with descriptions of physical reality.

6. Collaborative work
Collaborative work means more than just dividing tasks within a group into pieces that belong to someone. It is a new way of working together on a shared set of tasks for a common goal. Its best properties can be seen in situations where there is a diverse (and maybe unknown) group of potential participants whose knowledge and value judgements on the issues relevant to the particular assessment are combined in the description. With the aid of fairly simple rules, this group can work with little or no central guidance and still gather rapidly relevant information. The Intarese general method should contain these rules in its aims, so as to be able to facilitate stakeholder involvement and the collection and synthesis of information. To facilitate such synthesis, risk assessments are carried out in an assessment workspace which functions as the platform for collaborative work. The purpose of the assessment workspace is to offer an interface for accessing and applying all the specific methods included in the Intarese general method. Even more importantly, its purpose is to function as a platform for carrying out and managing the assessments in one virtual location. This feature of assessment workspace also enhances the possibility to extract and re-use information from the previous assessments that have been performed using the workspace. For further details, see Collaborative work | Managing stakeholder involvement | Pyrkilö-wiki vision | Assessment workspace (Intarese method) | Intarese toolbox.

7. Dealing with disputes
When a diverse group of contributors participate in making a risk assessment, it is obvious that disputes may arise. One of the most instructive features of risk assessment is to understand both these disputes and the reasons why a particular outcome occurs. Intarese general method must include methods that help to deal with disputes, find resolutions and document the choices made so that they can be defended afterwards. Argumentation theory offers a basis for these methods. For further details, see a manuscript dealing with this issue.

+1. Information repository in public domain
A large part of the costs of making a risk assessment arises from collecting basic information. There are large quantities of data available, but extracting the right information is expensive due to several reasons: it takes time to go through publication databases and find relevant articles; the data is usually not in a directly usable format, but it needs organising and synthesising; the source of information is copyrighted, and it cannot be used as such without an explicit permission from the copyright owner, usually the journal. To decrease the costs of a risk assessment, relevant information should be systematically collected into a repository that is in public domain, i.e. the contents are freely usable by anyone. Risk assessors and researchers should be encouraged to provide the information that they have collected for their own assessments. Such a repository would benefit other assessors and the society at large. The extra work needed from information providers should be acknowledged as work for general good.

Specific methods related to Intarese general method

The specific methods are grouped below based on their relation to either:

making of an assessment (process); or
the contents of the assessment (product)

The methods related to the content of the assessment product are further divided under three categories:

methods related to the full chain
methods related to particular steps of the full chain; and
methods related to individual variables and indicators.

Process: making an assessment

Method	Brief description
General properties of good risk assessments: Purpose of assessments	Based on the purpose of risk assessment and the relations of risk assessment with its societal context, the general properties of risk assessment can be described in terms of effectiveness and efficiency. The general properties also function as the general performance criteria of risk assessment.
Causality: Full chain approach	Risk assessments as causal network descriptions of the real-world phenomena from source to impact that are relevant in relation to the chosen endpoints and purpose of the assessment.
Collective structured learning	Improving efficiency and effectiveness of assessments with commmonly applicable and re-usable, shared pieces of information and inter-assessment collaboration. Builds on the idea of a universal assessment structure.
Combination of value judgements and physical world descriptions	Risk assessments should be able to explicitly scrutinize and combine value judgements of people with the descriptions of physical phenomena.
Collaborative work	Carrying out risk assessments as sets of shared tasks in diverse groups for a common goal.
Managing stakeholder involvement	How and why to involve the stakeholders and decision-makers in the assessment work.
Assessment workspace	The assessment workspace is a virtual working environment that provides a platform for managing collaborative assessment work using Intarese general method.
Dealing with disputes	How to deal with disputes that arise from the plurality of views, values and perceptions among the diverse group of contributors to the assessment.
Argumentation theory external links: Argumentation theory \| Pragma-dialectics	Argumentation provides the solution to dealing with disputes - no greater power than the power of a better argument.
Presentation of the assessment description	How to present the contents of the assessment.

Product: contents of an assessment

Methods related to the full chain

Method/guidance name	Brief description
Scoping and issue framing: Scoping tool for causal diagrams	Defining the research questions, contents, and boundaries of the assessments.
Guidance and methods for indicator selection and specification	Selecting and specifying of the variables of specific interest across the causal network.
Creating a policy assessment protocol	Creating the plan for the assessment phase
Identification and evaluation of policy options	How to identify, define, choose and apply different policy options within the assessment
Uncertainty and variability analysis \| Value of information analysis \| RIVM/MNP unicertainty guidance	Understanding and addressing the different aspects of uncertainty and variability in the assessment. Value of information can be used for identifying the importance of different variables influence to the chosen indicators and decision options.
Probability theory and probabilistic methods	Probability methods are used in many parts of the assessment. Basically all variable results are distribution estimates rather than point estimates.
Monte Carlo modelling 2D Monte Carlo for separation of variability and uncertainty 3D MC for separating variability, uncertainty, and ignorance	Monte carlo is a standard method for probabilistic simulations. It also provides means for simultaneosly and separately addressing different aspects of probability.
Bayesian analysis \| Bayesian belief networks External links: Bayes' theorem \| Bayesian probability \| Bayesian inference	Bayesian methods provide means for estimating probability distributions using heterogenous sources of data and subjective probabilities.
Decision analysis (external link) Decision heuristics: utilitarian, egalitarian etc. \| Risk aversion, risk premium Ethical value systems	Decision analysis is a method for combining probabilities with preferences
The four indicator sets	The four indicator panels are compilations of indicator sets taht can chosen for use according to the goals of the assessment. This approach is developed by RIVM.
1 Policy deficit indicators	E.g. exposures, concentrations, aims & goals, ...
2 Health impact indicators	E.g. number affected, severity, duration, ...
3 Economic consequence indicators	E.g. disease, absenteeism
4 Risk perception indicators	E.g. voluntariness, dread, ...
Alternative risk assessment approaches	There are alternative approaches to describing the phenomena to be assessed. Many of these have similarities and overlaps with what are included in the Intarese general method.
DPSEEA model^[1]	(Driving forces - Pressures - State - Exposure - Effects - Actions)
DPSIR ^[2]	(Driving forces - Pressures - States - Impacts - Responses)
Health impact assessment (HIA) \| Review of health impact issues	Health impact assessment is a combination of procedures, methods and instruments used for assessing the potential health impacts.
Impact pathway approach (IPA)	Impact pathway approach is mainly based on estimating impacts and monetizing them.
MNP Environment and health planner	A risk assessment approach developed by MNP
MEME model	MEME: Multiple exposures, multiple effects.
Pyrkilo method \|	Pyrkilo method is a risk assessment method which emphasizes the importance of openness and involvement of multiple parties throughout the assessment process.

Methods related to the steps of the full chain

Method/guidance name	Brief description
Pressures	Identification of pressures
Sources and Releases: Emission modelling	Identification of sources, Emission factor approach, description how the stressor is released into the environment (e.g. emissions to air, leakage, dumping, corrosion)
Media: Environmental fate and transport modelling \| Atmospheric modelling \| Groundwater modelling \| Multimedia modelling	Different media have different characteristics that need to be taken into account and that require different kinds of methods.
Exposure: Exposure modelling \| Intake fraction modelling \| Source-exposure modelling \| Exposure-source attribution	Models to estimate exposures given the emissions, and emission contributions given the exposures
Special exposure issues: Exposure mixtures \| Multiple exposures, multiple routes
Dose: Physiologically based pharmacokinetic modelling (PBPK)
Dose metrics: Margin of exposure (MoE) \| Tolerable daily intake (TDI) \| Acceptable daily intake (ADI) \| Hazard quotient (HQ) \| No-observed adverse effect level (NOAEL)
Dose-response
Functions, shapes, and underlying assumptions (linearity, threshold, hormesis)	Theoretical issues related to the concept of dose-response.
Estimating dose-responses: Meta-analysis \| Multibias modelling \| Combining epidemiology and toxicology \| Epitox workgroup	Meta-analysis is a method for combining data. Multibias modeling is a method to estimate the dose-reponse when there is external information about biases related to a published estimate. Methods on how to combine data from toxicological studies with data from epidemiological studies.
Extrapolation beyond data: Animal-to-human extrapolation \| High-to-low-dose extrapolation \| Endpoint extrapolation	How to use data on animal tests to estimating effects to humans? How to use data acquired from high-dose observations to low-dose situations? How to use data that is about a not exactly right endpoint?
Summary variables about dose-reponse: Reference dose (RfD) \| Cancer slope factor (CSF)
Health effects and other impacts	Health effects are the effects of the assessed phenomena to human health. Impacts include also other than health, such as monetary or ecological impacts.
Metrics for quantifying health effects: Life-table method \| Attributable deaths and cases of illness
Issues related to health effects: Vulnerable groups and sensitivity	Some groups are more vulnerable because of higher exposure or because of higher sensitivity to a particular exposure.
Valuation	Combining different kinds of health outcomes and other impacts for comparison and/or summing up total impacts
Combining health effects: Quality-adjusted life years (QALY) (external link) \| Disability-adjusted life years (DALY) \| Years of life lost (YOLL)	QALYs and DALYs use slightly different weighting for health impacts.
Monetization: Indirect methods: Hedonic Price Method \| Averting Behavior Method \| Travel Cost Method \| Contingent Behavior Method; Direct methods: Contingent Valuation Method \| Attribute Based Choice Modelling \| Participatory Approaches \| Surveys for preferences of public decision makers; All these methods may ask for: Willingness-to-pay (WTP) \| willingness-to-accept (WTA); Discounting \| Cost effectiveness analysis and cost benefit analysis	There are several monetisation methods to combine all kinds of impacts into a single currency, money. Valuations should be applied coherently in one assessment and preferrably also between assessments
Actions: Developing decision options	Actions can be targeted to any part(s) of the causal chain and thus have influence(s) to a number of variables and indicators

Methods related to variables and indicators

Method/guidance name	Brief description
Indicators and variables: Structure and attributes of variables	Indicators are variables of specific interest. Describes the standard attribute list that can be used for all variables.
Model and function selection: Standardized set of validated methods	Guidelines for evaluating and choosing models and functions for different purposes. Evaluated and recommendable methods to be used for desribing variables belonging to different steps in the causal chain.
Specific model use	Guidelines for using models in Intarese assessments
Data identification, evaluation, and selection	Guidelines for evaluating and choosing data for different purposes
Data use and processing: Meta-analysis	Guidelines for using data in Intarese assessments. Meta-analysis is a method for combining results from several studies
Expert judgements	Method for eliciting subjective probabilities from experts using formal procedures.
Including stakeholder inputs: Having discussions about the content	Stakeholder involvement as an method of collecting important data. Formalized discussions as a method of incorporating the knowledge of stakeholders to the assessment content

Related issues

Environmental health action plan of the EU
Expofacts
U.S.EPA Exposure handbook
FDA tolerance values
ATSDR toxicological profiles
HHS/NTP substance profiles
NIOSH current intelligence bulletins and criteria documents
MEME appraoch
WHO ECOEHIS indicators
As low as reasonably achievable (Alara)
Tools needed in Intarese toolbox

Properties of a good risk assessment method. An old version (archived).

Template:Release

An 'axiomatic theory' of environmental health risk assessment

Axiom 1

Risk is such a property of a situation that tells that a) the situation may lead to an unfavourable outcome; b) it is uncertain which outcome actually will occur; and c) there is a need to affect which outcome will occur.

Axiom 2

Risk assessment is a process of developing a rational and reasonable description of a particular risk so that more undestanding is achieved about how to reduce the need to affect the outcome.

Axiom 3

A risk assessment consists of statements that are either variables (statements about particular properties of the world, including both physical properties (what is?) and value judgements (what should be?)) or links (statements about the relationships between variables). Each statement has a focus and a scope.

Axiom 4

Risk assessment science is a discipline that answers to the following question: "When should I, as a rational critic who judges reasonably, regard a risk assessment as acceptable?"

Axiom 5

A statement in risk assessment is regarded validated iif it holds against critical argumentation about the content and relevance (according to van Eemeren and Grootendorst). The argumentation must be based on accepted premises and discussion rules. Falsification (according to Popper) is a special case of argumentation, where the validation is based on the consistence between the statement and existing data. The validation process and its result are parts of the statement.

Axiom 6

The axiom of efficiency: A new variable must not be created, if there already exists a variable with the same focus and scope.

Axiom 7

The scientific statements, value judgements, and validation rules in a risk assessment must all be internally coherent and consistent. If the assessment contains conflicting statements, there must be a rule for determining which of the statements holds in a particular situation.

Rules derived from axioms:

Based on axioms 3 and 6: Focus and scope of a statement must not change during the risk assessment. Statements may be structured hierarchically.
Based on axiom 7: A risk assessment based on one or more invalid statements is invalid.
Based on axiom 5: To be able to compare risk assessments, they must be based on cohorent premises and discussion rules. Thus, there is a need to develop systematic sets of standardized premises and discussion rules that will be used.
Based on axiom 6: The focus and scope of a statement must not change. This is because a statement may be used in several assessments, and the statement may lose its valid content or relevance, if its focus and content are changed.
Based on axiom 5: Validation process of a statement may change the content (but not the scope and focus) of the statement. This is allowed. However, the risk assessor must be explicit about which version of the statement he refers to, otherwise it will be difficult to follow the reasoning.
Based on axiom 7: Examples of inconsistent statements include the following:
- Probability distributions: Only one value is possible in each particular situation, and random sampling is the rule how to determine which one. Correlation matrices are also this kind of rules.
- Disagreement variables express disagreement about values. The rule may be e.g. that the risk assessor chooses the one he thinks relevant, or that all values are evaluated one at a time.
- Dispute variables express dispute about scientific statement. What is said about disagreement variables, applies also here.

Estates of risk assessment

The philosophical estate:

Q: When should I, as a rational critic who judges reasonably, regard a risk assessment as acceptable?

A: When the risk assessment solves the difference of opinion in accordance with "problem valid" assessment rules (solving the problem at issue) that are also acceptable to the parties.

Onko RA samanlainen kahden osapuolen väittely kuin argumentaatio? Kuka yritetään vakuuttaa? Ei se ole, vaan se on enemmän kuin Toulminin ja Perelmanin näkemys puhujasta, joka yrittää vakuuttaa skeptisen kuulijakunnan (joka ei osallistu aktiivisesti). Kuitenkin voidaan ajatella, että on olemassa jokin kysymys, ja siihen liittyvä haaste jonka todenperäisyys täytyy selvittää. Kun nämä molemmat on kuvattu, se määrää pitkälle sen, kuinka yksityiskohtaisen täytyy riskinarvioinnin olla, jotta selvyys kysymykseen saadaan.

Se, että joissakin tilanteissa kysyjä ja vastaaja on sama (kuten huippuyksikössä) ei muuta tilannetta. Toisaalta, riskinarviointi on jatkuva prosessi, jossa lisäkysymyksiä voi tulla matkan varralle kun ensimmäisen kierroksen tulos julkaistaan. Kuitenkin täytyy olla sääntö siitä, milloin riskin arviointi päättyy ja alkaa uusi arviointi. Yleissääntö kai on se, että kun alkuperäiseen kysymykseen on vastattu, arviointi päättyy, ja lisäkysymykset ovat lisäarvioinnin paikka.

Tieteen voi ajatella jakautuvan seuraavasti:

Kokeellinen tiede tekee havaintoja koeasetelmista, joissa tuntemattomat tekijät on pyritty satunnaistamaan pois.
Havainnoiva tiede tekee havaintoja perustuen tiettyyn menetelmään ja tutkimusasetelmaan jolla pyritään hallitsemaan tuntemattomia tekijöitä.
Mallitus perustuu ennustuksiin, joiden toteutumista arvioidaan (validoidaan) havainnoin.
Riskinarviointi perustuu ennustuksiin, joiden toteutumista ei voida kaikilta osin arvioida havainnoin. Tällöin ainoaksi validoinnin keinoksi jää koko tarkastelun ja ennusteiden alistaminen vapaaseen kriittiseen arviointiin. Arvioinnista selviytyminen ei todista ennustetta oikeaksi mutta se lisää todennäköisyyttä että väärät on osoitettu vääriksi prosessin aikana. → Tässä on nimenomaan kyse argumentaatioprosessista.

The theoretical estate:

Q: Which instruments are available to me for treating problems concerning the acceptability of a risk assessment systematically?

A: I can make use of an ideal model of risk assessment aimed at describing a risk and a series of rules for the assessment procedure that are relevant.

Millainen on RAn ideaalimalli? Rajataan aihe, määritellään tarkoitus (tarkasteltava päätös), valitaan lopputulos, kuvataan syy-seuraussuhteet ja tehdään VOI. Tämä asetetaan yleisesti arvioitavaksi.

Voidaanko luoda yleinen sääntökokoelma, jota täytyy noudattaa? Käytännössä tarvittaisiin useampia kokoelmia, joista täytyy valita (tai joukko asioita, joista pitää laatia säännöt soveltamisesta).Olisi hyvä olla jokin perustuslaki, joka ainakin täytyy täyttää. Tuo yllä mainittu voisi toimia semmoisena.

The analytical estate

Q: How can I obtain a clearer picture of everything that is relevant for my evaluation of a risk assessment?

A: By reconstructions the risk assessment as an attempt to resolve a difference of opinion by conducting the required dialectical transformations.

Tässä voisi ajatella syy-seuraussuhteiden nostamista ylätasolle, ja yksityiskohtien painamista alatasoille. Olisiko tämä toimiva systeemi muuntaa riskinarviointi noudattamaan oikeaa formaattia? Voisi myös edellyttää, että riskinarvointi aluperinkin laaditaan käyttäen dialektisia sääntöjä. Toisaalta, on hyvä olla mahdollisuus siihen, että arviointi tehdään alunperin helpolla tyylillä käyttäen luonnollista kieltä, ja vasta myöhemmässä tarkastelussa se muutetaan noudattamaan formaalisia sääntöjä.

The empirical estate:

Q: Which knowledge about risk assessment reality that may be of special use to me can I acquire?

A: I can investigate which factors and processes are important in risk assessments to convince someone who is in doubt of the acceptability of an assessment.

Tämä on periaatteessa helppo: tarkastellaan oikeita riskinarviointeja ja arvioidaan niiden sisältöä kriittisesti ideaalimallin näkökulmasta. Katsotaan myös sitä, miten eri riskinarvioinnit on otettu vastaan.

The practical estate:

Q: How can I contribute to the improvement of risk assessment practice?

A: I can promote reflection on the procedures that are used in different risk assessment practices and the skills that are required for an adequate production, analysis, and evaluation of risk assessment practices.

The texts below are from van Eemeren, Grootendorst: A systematic theory of argumentation (Cambridge University Press, Cambridge, 2004). The texts about risk assessment are created by analogy from the texts in the book. Some texts are shortened.

A systematic theory of argumentation

The philosophical estate:

Q: When should I, as a rational critic who judges reasonably, regard argumentation as acceptable?

A: When the argumentation solves the difference of opinion in accordance with "problem valid" discussion rules (solving the problem at issue) that are also acceptable to the parties.

The theoretical estate:

Q: Which instruments are available to me for treating problems concerning the acceptability of argumentation systematically?

A: I can make use of an ideal model of critical discussion aimed at solving a difference of opinion and a series of rules for the performance of speech acts that are relevant in such a discussion.

The analytical estate

Q: How can I obtain a clearer picture of everything that is relevant for my evaluation of an argumentative discourse or text?

A: By reconstructions the discourse or text as an attempt to resolve a difference of opinion by conducting the required dialectical transformations.

The empirical estate:

Q: Which knowledge about argumentative reality that may be of special use to me can I acquire?

A: I can investigate which factors and processes are important in argumentative discourse to convince someone who is in doubt of the acceptability of a standpoint.

The practical estate:

Q: How can I contribute to the improvement of argumentative practice?

A: I can promote reflection on the procedures that are used in different argumentative practices and the skills that are required for an adequate production, analysis, and evaluation of argumentative discourse.

Rules for argumentation

Rule 1

The difference of opinion may concern any standpoint. Any discussant may call into question any standpoint.

Rule 2

If someone has called a standpoint into question, he is entitled to challenge this discussant to defent his standpoint in the confrontation stage.

Rule 3

A discussant is obliged to defent his standpoint, if he and the antagonist agree on the premises and discussion rules.

Rule 4

The discussant who originally presented the standpoint is a protagonist, and the discussant who called this into question is an antagonist, unless they decide otherwise. The roles are maintaind until the end of discussion.

Rule 5

The protagonist and antagonist must agree on discussion rules and criteria on when the standpoint has successfully been defended or attacked.

Rule 6

The protagonist may defend the standpoint using argumentation. The antagonist may attack the standpoint by calling into question either the propositional content or the justificatory force of the argumentation. Other acts are not allowed.

Rule 7

Intersubjective identification procedure is used to determine if the argument has been successfully defended or attacked.

Rule 8

Intersubjective inference procedure is used to determine if the force of justification or refutation of the argumentation has been successfully defended or attacked.

Rule 9

An initial standpoint has been conclusively defended (attacked) if both (either) the propositional content and (or) the force of justification of the argumentation has been successfully defended (attacked).

Rule 10

The antagonist may at any time during the discussion call into question any argumentation that has not yet been successfully defended.

Rule 11

The protagonist may at any time during the discussion defend any argumentation that he has performed and not yet successfully defended against every attack by the antagonist.

Rule 12

The protagonist retains throughout the entire discussion the right to retract any argumentation that he has performed, and thereby to remove the obligation to defend it.

Rule 13

a) The protagonist and the antagonist may perform the same speech act with the same role in the discussion only once.

b) The protagonist and the antagonist must in turn make a move of (complex) speech acts with a particular role in the discussion.

c) The protagonist and the antagonist may not perform more than one move (complex) speech acts at one time.

Rule 14

Standpoints that have been cuccessfully attacked must be retracted, and those successfully defended must no longer be called into question. All other standpoints are open to defence and critique.

Rule 15

Clarification may be asked at any stage of the discussion, and clarification must be given when asked for.

Ten commandments for reasonable discussants

Discussants may not prevent each other from advancing standpoints or from calling standpoints into question.
Discussants who advance a standpoint may not refuse to defend this standpoint when requested to do so.
Attacks on standpoints may not bear on a standpoint that has not actually been put forward by the other party.
Standpoints may not be defended by non-argumentation or argumentation that is not relevant to the standpoint.
Discussants may not falsely attribute unexpressed premises to the other party, nor disown responsibility for their own unexpressed premises.
Discussants may not falsely present something as an accepted starting point or falsely deny that something is an accepted starting point.
Reasoning that an argumentation is presented as formally conclusive may not be invalid in a logical sense.
Standpoints may not be regarded as conclusively defended by argumentation that is not presented as based on formally conclusive reasoning if the defense does not take place by means of appropriate argument schemes that are applied correctly.
Inconclusive defences of standpoints may not lead to maintaining these standpoints, and conclusive defenses of standpoints may not lead to maintaining expressions of doubt concerning these standpoints.
Discussants may not use any formulations that are insufficiently clear or confusingly ambiguous, and they may not deliberately misinterpret the other party's formulations.

Toolbox is a software for applying the Pyrkilo method. Details of is are shown on the graph below. Click on a detail to get more specific information.

Collaborative workspace: user interface for the risk assessors: describing risk assessments and variables, using the principles of mass collaboration (a large group of interested people working together for a common goal) and argumentation (rules for resolving disputes using structured argumentation). The collaborative workspace contains four kinds of information:
- Openly available and usable, structured information. This is mainly in the form of variables.
- Openly available and usable, but freely structured information. The major part of this content is in the form of an encyclopedia and glossary, but there are also fact sheets, guidance documents, and data and models there.
- There can also be structured information as variables that have a limited access. This may be the case in some so called private risk assessments, where the needs of confidentiality are stronger than the benefits of mass collaboration.
- There can be metadata about the data existing in external sources, i.e. data that is not openly available and usable and is not structured as variables. This metadata is also called the Data gateway.
Separate calculation tools: a functionality to transform variable definitions into a model that can compute results using an external software, such as Analytica or R.
External models: If a variable requires complex computing (such as atmospheric modelling), the actual definition can be externalized to a collaborating institute. The institute runs the model, and only the result is made available in the result database.
Result database: contains previously calculated results of variables, so that they are available for risk assessments and for calculating downstream variables.
Analysis tool: This tool performs the assessment-specific analyses, such as scenario optimising, sensitivity analyses, or value-of-information analyses. (Assessment-specific means that the analysis is not a part of any single variable but is an analysis about a group of variables within an assessment.) These analyses are based on the result database.

Use this template instead of the image:Risk assessment method - process.PNG so you will get direct links to the help pages of the different parts of toolbox.

This document is produced for Intarese Subproject 3 in the startup phase. It contains guidance from Subprojects 1 (and 4?) to facilitate the policy assessment drafting. The deadline for this document is mid-May. About a page per topic.

Assessment framework

Define draft focus of the assessment. Is it driven by e.g. a decision at hand, or a particular outcome whose causes are assessed?
Define draft scope of the assessment. What is to be assessed and what is to be excluded from the assessment? Describe the value judgements that you use to defend your selection.
Define draft indicators that will be looked at, and select one as the main output indicator. The indicator list should always contain indicators from each of the following groups (or reasoning why a group was ignored).
- Policy target indicators (how far is the target)
- Impact indicators (health, cost)
- Public acceptability indicators (equity, perception)
- Appraisal indicators (cost-benefit analysis, multicriteria analysis)
Define draft links between the indicators and the focus within the scope. All variables must link to the focus directly or indirectly.
Remove everything that is not linked to the causal chain of the assessment, because it is irrelevant.
Describe the value judgements that relate to the causal chain of the assessment.
Present the draft assessment structure to the stakeholders.
Based on the deliberation, revise the assessment as needed.
Finalise the focus and the scope. Do not change them any more within this assessment.
Add and improve statements (variables and links) that are relevant, i.e. linked to the focus and within the scope.
Let stakeholders criticise your work using the discussion rules of critical argumentation. Stakeholders may also be imaginary.
If any statement is not convincing for a rational and reasonable critique, add more argumentation to defend it. If unsuccessful, invalidate the statement. (Invalidating means that they are labeled invalid and they are not used for deduction; however they must stay parts of the assessment anyway to show that they cannot be used.)
Stakeholders may add statements to the assessment. These must be added unless they can be invalidated.
When no further critique is presented, label the risk assessment as finished (until additional argumentation is presented).

A risk assessment is always conditional to the data existing at the time. (In MediaWiki environment this means that when a referred page changes, the risk assessment is based on a non-current version. To avoid confusion, permanent links should be used so that links always point to the version that actually was used to deduce the conclustions of the assessment).

Source-exposure

Exposure-response

Risk characterisation

Describe the focus of the assessment and reasons for selecting this focus.
Describe the scope of the assessment and reasons for selecting this scope.
Describe the main assumptions used in the assessment.
Describe the indicators of the assessment and reasons for selecting these indicators.
Describe the dimensions and properties of the indicators. Does it represent average, a random individual, or certain fractile? Describe uncertainty bounds around the estimate. Especially describe the different kinds of indicators
- Policy target indicators (how far is the target)
- Impact indicators ( health, cost)
- Public acceptability indicators (equity, perception)
- Appraisal indicators (cost-benefit analysis, multicriteria analysis)
If possible, compare the importance of each uncertain variable to the output indicator. (e.g. value-of-information analysis or importance analysis).
Describe the main conclusion derived from the output indicator in respect to the focus.
Describe other conclusions.

Further reading related to risk characterisation

Environmental health planner for policy
Bayesian hierarchical modeling review
Feedback about Intarese method
D17 Risk characterisation protocol
Integration of policy WPs: D18 Risk characterisation methodology report

[1] Popper, Karl R. 1935. Logik der Forschung. Julius Springer Verlag.

[2] DPSEEA http://www.euro.who.int/EHindicators/Indicators/20030527_2

[3] DPSIR http://www.eea.europa.eu/documents/brochure/brochure_reason.html

[1]

[1]

[2]

Pyrkilo

Contents