Talk:Science-policy interface

Application to the Academy of Finland 25.9.2013

The application was sent to the Academy of Finland on 25.9.2013 15:25. The wiki version was slightly shortened: figure 2 was removed, and references were made shorter to fit on one line each. Archived version.

• Instructions

Metadata about document Avoimen päätösvalmistelun työkalutestaus (rahoitushakemus SA:lle 25.9.2013) (in Finnish) help

Nimi: Avoimen päätösvalmistelun työkalutestaus (rahoitushakemus SA:lle 25.9.2013) Tehtäväryhmä: 6.00.01.08 Tehtäväryhmän nimi: Rahoitushakemus Rekisteröinti: Vastuualue: YMAL Vastuuhenkilö: Jouni Tuomisto Luotu: 25.9.2013 Säilytysaika: 15 v Säilytysmuoto: Sähköinen Julkisuus: Julkinen Suojeluluokka: 2 (tutkimuksellisista, juridisista tai taloudellisista syistä suojeltavat asiakirjat) Lisätiedot: Pysyvä linkki: [10] Hyväksyntä: Ei ole

Public summaries

In English

There is a clear need for improved evidence-based decision making practices. This applies especially to environmental and health issues, as they are a part of most decisions but rarely dominate and are often forgot. This project implements a novel open dynamic decision support (ODDS) system that coherently combines a number of existing decision analytic, impact assessment, probabilistic, and participatory methods with web-based workspaces and tools. The ODDS method is implemented, tested and developed at municipality and national (AVI and ELY centres) level in several case studies together with local authorities and citizens. Some cases are very small and focussed on properties of a single tool or practice, while others are large and support the whole decision process of e.g. an environmental impact assessment and related environmental permission process. There are several research questions, e.g. about manageability. All work is done openly; see http://en.opasnet.org/w/ODDS

Suomeksi

Yhteiskunnassa on selkeä tarve tieteeseen perustuvalle päätöstuelle. Tämä pätee erityisesti ympäristö- ja terveysasioihin, jotka ovat osa useimpia päätöksiä, harvoin hallitsevat niitä ja usein tyystin unohtuvat. Tämä hanke soveltaa avointa mukautuvaa päätöstukea (ODDS, open dynamic decision support), joka yhdistää johdonmukaiseksi kokonaisuudeksi päätösanalyysin, vaikutusarvioinnin, todennäköisyyslaskennan ja osallistamisen nykymenetelmiä sekä verkkotyötiloja ja -työkaluja. ODDS-menetelmää sovelletaan, testataan ja kehitetään kunnallisella ja valtakunnallisella (AVIt ja Ely-keskukset) tasolla useissa tapaustutkimuksissa yhdessä viranomaisten ja kansalaisten kanssa. Jotkin tapaukset ovat aivan pieniä yksittäisen käytännön tai työkalun kehittämistä. Toiset ovat laajoja ja kattavat esim. kokonaisen ympäristövaikutusten arviointi- ja luvitusprosessin. Tutkimuskysymyksiä on useita liittyen esim. käytäntöjen hallittavuuten. Kaikki työ tehdään avoimesti, katso http://en.opasnet.org/w/ODDS

Abstract

The Finnish Government recently (5th September) said aloud a clear and general need: "We live in an information society only when knowledge and scientific information is systematically used to support decisions". This applies especially to environmental and health issues, as they are a part of most decisions but rarely dominate and are often forgot. Indeed, many decisions are very complex, and new practices to manage this complex information is needed. Many methods, practices, and web tools as well as citizen groups have recently emerged making this possible.

This project is based on a recent result that the bottleneck to use information is in decision making practices and capabilities. Therefore, there is a specific need to offer improved practices and tools especially to decision makers, and to study the applicability of them in real situations. This project implements a novel open dynamic decision support (ODDS) ecosystem (self-organised group of people working together) that coherently combines a number of existing decision analytic, impact assessment, probabilistic, and participatory methods with web-based workspaces and tools. The ODDS ecosystem is implemented, tested and developed at municipality and national (AVI and ELY centres) level in several case studies together with local authorities and citizens. Some cases are very small and focussed on properties of a single tool or practice, while others are large and support the whole decision process of e.g. an environmental impact assessment and the related environmental permission process.

The ODDS ecosystem does not replace decision making or current decision support practices such as expert committees, TV debates, or public hearings. Rather, it acts as a way to collect, synthesise, and improve the information available to participants. ODDS ecosystem supports open participation, and the contributions are managed with clear and specific rules about e.g. relevance rather than limiting participants or freedom of speech.

The research in this project will be about the applicability and performance of the method: What are the major information or resource deficiencies that prevent the use of ODDS ecosystem in municipalities? How can they be overcome? How can such an ecosystem be developed for open participation in societal decision support? The work is organised and done in an open web-workspace Opasnet designed for this task, and it is managed by trained facilitators. All work is done openly; see http://en.opasnet.org/w/ODDS

Research plan

Guidance for the research plan: Finnish English

Project info

• Principal investigator: Jouni Tuomisto
• Title of project: Open dynamic decision support (ODDS)
• Site of research: National Institute for Health and Welfare, Department of Environmental Health, Kuopio
• Duration: 48 months, 1.9.2014 - 31.8.2018

Background

Significance of the research

Evidence-based decision making is a mega-trend in Finland and in other Western countries. Prime Minister Jyrki Katainen recently said that we live in an information society only when research knowledge is systematically used as a basis of decision making. In the Government plan there is an objective to utilise information about environment and health in all decision making . Enterprise architecture, a management system focussing on information and practices, is in a running-in phase in Finnish administration. In addition, organisational changes are under way to improve the capability of Finnish research institutes to answer societal needs. There is clearly a strong tendency to improve the use of knowledge in the societal decision making, and good research-based solutions are needed.

There are challenges especially in the capabilities of decision makers and decision making processes to actually utilise existing information. This is seen as unhappiness of decision makers about data usability, and also unhappiness of researchers about data use. In this project we will demonstrate, implement, and further develop an open dynamic decision support (ODDS) that consists of several methods, practices, tools, and web-workspaces. It especially helps to structure scientific information in a helpful format for decision support, and enhances critical syntheses of open discussions on policy issues.

Indeed, there is a need for systematic decision support especially with issues like environment and health. They are widely accepted as important, but in many decisions they only play a small role among other interests and are easily ignored, if the relevant information is not readily available for the decision maker. Climate emissions, biodiversity, or fine particles from combustion are examples of widely dispersed and crucial issues that rarely dominate decision making.

A decision support system does not attempt to replace actual decision making. However, it can organise information, offer a discussion forum, and spread understanding to the society about what should or should not be done and why. Such a system can be seen as similar to recommendations of evidence-based medicine (käypä hoito) containing the best scientific evidence about how patients should be treated in particular situations. This project attempts to create an ODDS ecosystem (a group of self-organised people working together for a defined goal) for producing evidence-based decision support. In the ecosystem, open participation is allowed, and the process is managed by clear and specific rules.

Even if a particular evidence-based advice is - due to lack of information - so simplistic that it does not help the decision maker, it may still be useful if done openly and shared. First, it may be illuminating to a stakeholder who is interested but not aware of the details. Second, describing decisions may be helpful for other decision makers in similar situations. Third, a scrutiny of multiple decisions at the same time may improve understanding of a bigger picture, leading to better decisions and outcomes for all. Therefore, evidence-based efforts should not be evaluated based on their impact on a single case only.

Because real-life problems are complex and fuzzy, we benefit if more people contribute their knowledge and bring in multiple views and ideas. However, this requires that the information can be received, synthesised, and analysed properly. Methods and tools for such work exist, and one systematic collection of them is called open policy practice or ODDS practice (see Previous research), and the need and capability to utilise them are about to meet. This requires dedicated implementation and research on the possibilities, problems, and new solutions of the implementations. This is what this project is about.

There are specific research needs when ODDS is applied with municipalities and national authorities such as AVIs or ELY centres. First, there is a need for large case studies, where open impact assessments are tested as a part of decision process (e.g. environmental impact assessment EIA and environmental permit processes). Second, the applicability of existing environment and health impact models should be tested and further developed. Third, practices and models should be tested and developed in several very small case studies that have immediate applicability in municipalities and require no additional resources. This approach helps to raise interest in municipalities and to identify immediate information needs.

Previous research

There is active research going on about improving the societal use of scientific results. For example, there are suggestions that the policy relevance of scientific assessments must be improved (Perrings et al., 2011) and that they should better reflect the reality of policy making and include local and non-scientific knowledge (Briggs and Knight 2012, Hulme et al., 2011) . However, the effectiveness also depends on the capability of a decision maker to utilise information as a part of decision making process (Lankinen et al., 2012, Junnila, 2012) . Little attention has been paid to information use, and most related research has focussed on information production (Jones 2009)

A recent study has found that a major problem in the science-policy interface actually lies in the inability of the current political processes to utilise existing scientific knowledge in societal decision making (Pohjola, 2013) . This inability applies also to knowledge about citizens' and other stakeholders' values. Evidence-based decision making requires multifaceted, justifiable, practical information production and effective information use.

This observation has lead to the development of a pragmatic guidance for closer collaboration between researchers and societal decision making. The guidance is called ODDS practice. It was developed by National Institute for Health and Welfare (THL) in 2013. The aim is to improve environmental health assessments in Finnish municipalities, but it is generic and widely applicable. One notice in the work was that knowledge practices should be developed simultaneously in both research and decision making, otherwise either the information supply does not answer the need or vice versa.

The ODDS practice consists of guidance, practices, and tools facilitating production and use of relevant information for a decision. The practice encourages a decision maker to express the objectives of the decision and options considered, and this information is used to guide all work. A large part of the work is to perform an impact assessment that covers all areas of interest (as defined by the decision maker) and synthesises contributions from anyone interested. The work is constantly evaluated and managed according to specific guidance about properties of good decision support. The work is managed by facilitators, who are knowledgeable about the decision situation, research, and the rules of the practice.

One important part of the ODDS practice is a web-workspace to facilitate decision support, share knowledge and learn from others. In the web-workspace both scientific knowledge and policy alternatives and objectives are systematically represented. All relevant information is stored in a structured way. This structure also guides the work of information collection and synthesis in an assessment.

ODDS ecosystem does not replace the actual decision making or current processes such as debates or committees. Instead, it facilitates improved knowledge practices to describe decision-related information that is relevant in the eyes of a decision maker, stakeholder, or researcher. A main advantage is that specific rules improving information (such as scientific criticism) can be applied within the ecosystem even if they cannot be applied in other policy forums.

ODDS practice follows rules that lead to open and reusable information products. This is done by utilising an open web-workspace as described above, by focussing on topics that are influenced by the decision or influencing the outcomes of interest, and by applying explicit rules about which statements or estimates to reject based on relevance and facts. The practice has been documented in reports (Pohjola, 2013) (Pohjola ym., 2012) , method descriptions (Pohjola et al., 2013) , method testing (Sandström et al, in press) (Pohjola et al., 2012b) , websites , and technical documentation .

A key idea in ODDS practice is to focus on information work, and support the management of that work at the same time. The work is organised in a way that it is easy to obtain the information that is necessary, and also to share the information each participant has. This approach is close to the management system enterprise architecture that looks at four things simultaneously: information, information practices, information systems, and ICT. Enterprise architecture is becoming mainstream in Finnish administration, and therefore there is a clear need for compatible practices that can be extended to new areas such as municipality decision making. ODDS practice benefits also from several grassroot activities in Finland about sharing and using information in decision making (see WP3, WP4).

ODDS practice is a synthesis of large body of research on different areas, from where we in THL have screened, hand-picked and adjusted excellent ideas into a coherent practice. Only the most important ones are shown on Table 1. To our knowledge, this is the first time when these methods will be implemented in a coherent way in decision support in large, real-life decision situations.

Table 1. Properties needed in ODDS practice and rules or methods applied to achieve the properties.

Property strived for	Method to be used	Description and reference
Participation and contributions
Anyone can participate in decision support.	An open wiki web-workspace: Opasnet.	Interface similar to Wikipedia. Shared information objects.
Discussions converge to a resolution.	Pragma-dialectic argumentation rules.	Rules define how a statement is accepted or rejected.
Value judgements are expressed and critically evaluated.	Quasi-realistic moral philosophy	Moral statements are expressions of individuals. They are evaluated like factual propositions.
Preferences of several stakeholder groups are assessed.	Stakeholder preference elicitation	Stakeholders rank different outcomes. Probability distributions describe the results.
Citizen feedback can be given as maps.	Mapita and other map interfaces	Web tools collect and show data simply by clicking maps.
Criticism and uncertainties
Scientific reasoning is used.	The scientific method of criticism	Falsification of hypotheses based on observations.
Uncertainties are described quantitatively.	Systematic use of probabilities	Subjective (Bayesian) probabilities and approaches.
Estimates are used systematically even if there are no measurements.	Elicitation of expert judgement	Experts produce probability distributions that are weighted by experts' performance.
Modelling
Decision descriptions give justifiable guidance.	Decision theory and decision analysis	Probabilities and utilities express decision options, impacts, and valuations.
Discussions and quantitative modelling synthesised seamlessly.	Structured discussions, ovariables, and OpasnetUtils	A systematic information structure with standardised information objects. Further work in this part in WP1., Rintala et al 2013
Evaluation and management of work
The contributions of self-organised stakeholders are managed.	Wisdom of crowds and mass collaboration	The work is chopped into small independent pieces in a decentralised way and then synthesised.
The work process is evaluated and managed.	Properties of good assessment	Evaluation criteria for the current and foreseeable progress, according to the objectives.
Open participation process is managed.	Interactional expertise	Facilitators follow and manage contributions using management skills and rules.
Work process management follows national guidelines.	Enterprise architecture	Four perspectives: practices, information, information systems, and ICT.
Practice development according to the social and health sector.	Innovillage	Guidance about how to develop, implement, and evaluate practices.

Links to other research by the team

The project implements methods that have been developed by the research team in previous research projects about decision analysis, impact assessment, and decision support. Such projects include EU projects Beneris, Intarese, Heimtsa, and Hiwate (integrated environmental health assessment, 2005-2011), Tekes project Minera (environmental and health risks of mining, 2010-2013), and ministry-funded Tekaisu (environmental health assessments in municipalities, 2012-2014). The projects have developed a) methods, models and web tools for impact assessment and b) practices that support integration and use of scientific knowledge and value judgements.

These methods and practices have subsequently been used in many projects. E.g. EU-funded Urgenche (2011-2014) looks at health impacts of climate policies at municipality level and uses Opasnet web-workspace for modelling and project management. Academy-funded CONPAT looks at the sources, behavior and fate of microbial and chemical contaminants and their health and economical impacts. TEKES-funded POLARIS (2009-2012) looked at sustainable water quality management in artificial groundwater production. For previous work about Innovillage or open democracy, see WP3 and WP4.

For illustration, we describe one model developed in previous projects, namely the Wated Guide model, and its use in decision support. The quality and health impacts of drinking water is the responsibility of municipality authorities. The quality control nowadays focusses on the end product, which is always too late to prevent microbial outbrakes. There is a clear need for a tool that enables prediction of impacts and their prevention in different special situations and future investment scenarios. WHO has launched a procedure Water Safety Plan to promote such work, and some countries such as the Netherlands have implemented quantitative microbial risk assesssment in this planning.

Water Guide model (http://fi.opasnet.org/fi/Vesiopas) is a web-based tool for quantitative microbial risk assessment on waterworks level and can be used quickly with little training by the professionals in the municipalities and waterworks. Water Guide has been successfully used in research projects, but it has not yet made a breakthrough as practical tool for professionals. This project will evaluate the hindances (especially WP2) and solve them (WP1).

Objectives

Research objectives

The ultimate objective is to improve the outcomes of societal decisions involving environmental and health issues by developing and promoting the ODDS practice, which consists of a) practical high-quality environmental information, b) systematic work practices, c) advanced impact assessment methods, and d) modern ICT technology to support societal decision making in general and in Finnish municipalities in particular.

There are some critical research topics that need to be tackled before it is possible to implement the practices described above in a large scale. These objectives are classified into four groups and presented as research questions.

• Methodological:
  • How can aspects from structured discussions be included in continuously updated assessment models during a policy process without breaking the structure and functionality of the model?
  • How can elicitations of stakeholder valuations be used in a coherent way with multiple stakeholder groups? How can the results be used systematically within probabilistic assessment models?
• Practice-oriented:
  • Which of the current societal decision-making practices are in conflict with the ODDS practice? How can these conflicts be resolved?
  • How to apply the methods of Innovillage within environmental health assessments?
• Communications-oriented:
  • What are the major information or resource deficiencies that prevent the use of ODDS practice in municipalities? How can they be overcome?
  • What are the practical needs of Finnish municipalities or national authorities related to environmental health assessments?
• ODDS-ecosystem-oriented:
  • How can an ecosystem be developed for societal decision support in such a way that everyone (decision makers, experts, stakeholders, and developers) can effectively participate and want to do it?
  • How should interactional expertise be applied in order to moderate the contributions and work within such an ecosystem?

Hypotheses

• The ODDS practice helps participants to focus on the decision options, possible outcomes, and value judgements of outcomes, i.e. the information needs of a particular decision. This will reduce situations where the focus of work is determined based on the availability of data rather than actual policy need and situations where the focus is on authority, power, procedures, responsibilities, or negotiation tactics.
• Practical guidance for including structured discussions by participants in the continuously updating models - based on Bayesian or other online-learning techniques - can be developed. This reduces the need for experts as mediators and thus improves the manageability of assessment processes. The inclusion of discussions can be done without causing large modelling risks such as crashes of the model, memory overflows or non-convergence of estimates.
• Uncertainties about facts can be systematically described based on probabilistic approaches and the contributions from experts and other participants and with the help of facilitators. This will help integration of uncertain information from various sources with appropriate weights, and increase the acceptability of the assessment outcomes.
• The quality of decision support can be measured by constantly evaluating the quality of content, applicability, and efficiency of the information production. This evaluation will improve - in a measurable way - the execution of work processes.
• Organisations participating in the case studies of the project will generally find the practices and models as an effective and feasible way to support evidence-based decision making. Criticism presented does ruin the foundations of the ODDS practice but rather can be effectively used to improve it.
• Identification of critical communication needs and problems will improve the project communication and facilitate the recruitment of participants to case studies and ODDS ecosystem.
• The ODDS practice enables self-organised decision support processes that are independent of this research project. Ecosystems will emerge where municipality decision makers, experts, and citizens launch self-organised activities to support particular decisions of their own interest. This is an ultimate test for the applicability of the method.

Materials and methods

Research methods

Figure 1. An example of a decision diagram, which represents a complex multi-decision, multi-stakeholder decision situation. The main parts and their causal relations are shown according to the open assessment method. For details, see text.

The overall method to describe a complex decision situation is impact assessment and typically decision analysis (Raiffa 1997). It consists of a description of the decision (D) and its options considered, a causal network (C) to outcomes of interest (O), and value judgements (V) of the outcomes by the decision maker. However, typically decision analysis looks at a single decision by a single decision maker at a time. The approach is extended in this project to cover multiple decisions and decision makers. In addition, value judgements can be expressed by any stakeholder group (S_i) even if they are not in the position to decide. Thus, participants can also learn what other options would be chosen based on other valuations present in the society. All participants share the same causal network, i.e. the description on how things are and how things affect each other. Valuations are expressed as ranks of preference and operated using probabilities (Cooke, 2007) .

The causal network is described as a quantitative model. Different methods and model types (such as deterministic or statistical models) can be used in applicable situations, but the paradigm is based on the idea of a Bayesian network, where the issues are described using subjective probabilities, and the relations are described as conditional probabilities. Deterministic and other models are embedded within the paradigm as necessary. The work starts from a coarse description of all possible options and outcomes, and implausible combinations are rejected as the understanding of the causal network increases. We have recently developed an approach that enables model descriptions with very coarse and very sophisticated manner (Rintala et al 2013). Thus, an assessment can use the same modelling approach irrespective of the complexity. The ODDS practice looks at and manages the whole chain of decision making from support to outcomes. However, the focus of new methods developed and tested in this project is mostly on the decision support.

The project uses web-workspace Opasnet (http://en.opasnet.org). It is open for reading and using, and it implements all decision support methods described in Table 1. It also contains several large environmental assessments. Opasnet offers strong support for data management, modelling, and even original research. It consists of a wiki, a modelling software R, a database for small and large data sets, and a web tool developer. Sharing and borrowing assessments, data, and models is made easy.

All of the methods mentioned have been tested and implemented, but this project offers such a unique, coherent combination that has not been implemented before. However, there are also challenges. Only a small fraction of decision-making problems are quantified and assessed since (i) uncertainty is often huge and challenging to quantify; (ii) sufficiently accurate and unbiased computational models may not be available for 'objective' evaluation; (iii) values play an important role and any quantification scheme for them seem biased. This project will reduce all these problems by using subjective probabilities, expert elicitation, coarse models for documentation even when quantification fails, and explicit inclusion of stakeholder preferences within the models (see Table 1 for more details). Also, the ODDS practice will make all this transparent and subject to criticism.

There are several incentives for decision-makers to use the open dynamic decision support ODDS. Transparency in general is found important in Finland, and collecting feedback from larger stakeholder group gives an opportunity to anticipate public reactions before decisions are made. Also, there is an active international movement of open data and open democracy, so we anticipate new practical tools and methods to become applicable during the project. A main bottleneck is to gather critical masses of participants in the Finnish scale, but looking at many similar decisions in several municipalities at the same time decreases this problem. This will also increase the efficiency: laborious tasks are immediately used by larger groups, thus making it more motivating to accomplish them.

Research material. Research material is obtained from the case studies from the participants (including the project researchers). It will be e.g. scientific literature, descriptions of objectives, minutes from public hearings, web discussions, and new models. Any information in basically any format is made available to people reading a page about a case study. All relevant information will be synthesised into a quantitative description. The material will be used to improve the decision support including quantitative models of that case. Subsequently, the material will be used to improve other similar decision processes and decision practices in general.

Materials management plan. The data will be stored publicly in Opasnet. In cases where the data cannot be published, it is stored in a password-protected area. In rare cases if sensitive personal or other data is used, the applicable rules of THL are obeyed in handling and storage. Daily backup copies are taken from the website. Practices developed will also be stored and published in Innovillage. Opasnet has a built-in version control and archiving functionality.

Because all material is available as open data in standard structures, most of the typical hindrances of data use are solved by default. In addition, WP2 aims to spread the word about existing data to promote further use. The material is published using CC-BY-SA license, which in practice does not limit use. However, the merit and ownership of the material stays with the contributor. All articles, if possible, will be published in open access journals. In any case, the final drafts will be published in Opasnet.

Ethical issues. The project does not involve research on patients, and handling of sensitive data requiring ethical permissions is not anticipated. In any case, the ethical rules of THL will be obeyed.

Risk management. There is a common fear that open decision processes like in this project will lead to a huge unmanageable flow of low-quality contributions, and the process will fail in chaos. Our experience in practice has been the opposite: the most difficult part of such a process is to get stakeholders (including researchers and decision makers) interested and involved. In this project, this risk is minimised by (i) applying ready-made models that can produce useful results quickly, (ii) by further developing these models more user-friendly based on feedback, and (iii) using the skills in WP2 to identify and solve critical hindrances of participation.

Another major risk is that the complexity of decision support aimed at is, after all, too high. However, this project carefully builds on a foundation with tested and validated methods that have been used in a large scale elsewhere. We have also paid a lot of attention to make sure that the methods used are not contradictory with but rather supporting each other. The question is actually whether such decision support system is effective enough to be usable without dedicated research funding. The project will give answers to this.

A third risk is the amount of interactional expertise needed to facilitate the work. We expect to learn a lot about training needs in this project, but we also anticipate that being an interactional expert is a demanding task that requires quite a lot of training. Therefore, in a typical decision process e.g. in a municipality, this expertise must come from outside. Thus, there is a need for teaching interactional expertise to a larger group who could be recruited to future decision processes in municipalities. However, this larger training is out of scope of this project, and other funding will be applied for it from elsewhere.

Implementation and budget

Workpackage 1: Methodology development

Leader: Jouni Tuomisto (adjunct professor). Personnel: Päivi Meriläinen (PhD): quantitative impact modelling, valuations

WP1 work is based on an existing system that can support most parts of decision support work, including modelling. However, there are three main tasks of further research. First, we study practices to include structured stakeholder contributions in quantitative probabilistic models in a systematic way. This work is based on previous work on pragma-dialectics on discussion side and OpasnetUtils on modelling side (see Table 1.) We need to solve theoretical and technical questions about e.g. inclusion of novel Bayesian techniques. We also need to develop practices and guidance for the users to implement it. Modelling experts in WP4 will participate in this work.

Second, we need to implement methods to include valuations in impact models. This research is based on Cooke 2007, but new research is needed for applying the method to multi-stakeholder situations. Also, tools and guidance are needed to use the improved method in the web-workspace. Third, there is a need to update and build new modules for impact assessment. For example, mining exposure models, life-table models, and energy balance models have been developed in three different projects. These will be updated in such a way that mining model feeds to both others and and energy balance feeds to life table model using standardised structures and interfaces. Similar standardisation is also done to the Water Guide model.

Workpackage 2: Communications and influencing

Leader: Mervi Pitkänen. Personnel: Kaarina Wilskman

WP2 will communicate about the project to the target group of municipality and regional decision makers. The aim is to increase awareness and interest among environmental health authorities but particularly find and recruit participants for small and large case studies (see WP5). This is a challenging task, because especially the large case studies require participants interested in a specific topic from among decision makers, experts, and stakeholder groups. In small case studies with tool testing, the recruitment is targeted to identified groups that would benefit most from using such a tool. The skills of the THL Department of Communications and Influencing are therefore needed and used.

Feedback about the practices and tools will be systematically collected in the case studies, and this information will be used to guide further development in WP1. We will study user experience, usability, and need for the user in all development of tools and practices. The communications work also promotes the development of an ODDS ecosystem with a stable community of people interested in developing decision support in Finland (see WP3, WP4).

Workpackage 3: Integration of existing practices

Leader: Pasi Pohjola (PhD; Innovillage). Personnel: Tapani Kauppinen (PhD; health and social impact assessment)

WP3 utilizes the existing Innovillage environment for developing the decision making practices and local solutions developed specifically in the selected case studies. Innovillage is a national web-based collaborative development environment for developing, implementing and evaluating methods in social care and health services in Finland. Currently Innovillage contains about 650 models and their local implementations of practices from various areas. The environment is used in national development programs, such as the National Development Programme for Social Welfare and Health Care, run by the Ministry of Social Affairs and Health (http://www.stm.fi/en/strategies_and_programmes/kaste). In WP3 Innovillage works as the environment where the decision-making practices of the case studies are developed and evaluated.

One key area of work is to develop the existing administrative impact procedures (e.g. IVA, SOVA for human and social impact assessments) as an integral part of ODDS practice. Through the use of Innovillage, the outcomes of the research project are disseminated and spread for wider audience. As an open innovation environment, it enables other municipalities and decision makers to utilise the model developed in the project case studies. In this way, Innovillage is developed into a seamless part of ODDS practice.

Workpackage 4: ODDS ecosystem

Leader: Sami Majaniemi (PhD). Personnel: Leo Lahti (PhD), Mikko Pohjola (PhD)

WP4 work aims to develop an ODDS ecosystem for societal decision support, particularly with regard to decision making with ecological and health significance. The ecosystem is based on existing open-society activities such as Open Knowledge Finland, the Finnish Association for Online Democracy, Sorvi, Avoinministeriö, Kansan muisti and Deliberatiivisen demokratian instituutti. In addition to setting up and organizing a network of actors with interest in participatory decision support, WP4 will study the specific requirements for interactional expertise as well as develop and implement corresponding practices for supporting broadly collaborative decision support within the ecosystem.

WP4 can thus be considered as having practical dimension and a theoretical dimension. The practical dimension focuses on linking the possibilities provided by existing open-society activities with the case studies of WP5 with the purpose of enabling broad collaboration in model-based assessments for decision support. This includes both the arrangement of work by different organisations and individuals around specific assessment/decision cases and solving the technical challenges in fitting together different tools and platforms applied by different embers of the ecosystem. The theoretical side then scrutinises the needs for interactional expertise arising in the collaborations in the WP5 case studies. It thereby attempts to identify and characterise the most important and crucial aspects of interactional expertise required in collaborative decision support. The organisation of collaboration is developed when the understanding of requirements for interactional expertise increases. The scrutiny of interactional expertise builds e.g. on the periodic table of expertise by Collins and Evans (2007). The success of collaborative decision support cases is evaluated based on the methods for evaluation and management in the ODDS practice.

Workpackage 5: Management of case studies

Leader: Jouni Tuomisto (adjunct professor). Personnel: Päivi Meriläinen (PhD; management of case studies), Hannu Komulainen (research professor); risks of mining and metals), Ilkka Miettinen (adjunct professor; risks and safety of drinking water)

WP5 manages the case studies and takes care of communication within the project. This includes regular online meetings and an open project website about upcoming tasks and progress of work.

WP5 also builds and executes impact assessments for case studies. Many of these are existing models (such as the Water Guide) that are, however, originally designed for a narrower use and require development into a more generic and thus more usable form. Also new models are developed for selected priority cases. In addition, research on user experience is performed to guide development. There is also a need for training and support for decision makers and stakeholders about the new tools, and this will be organised by WP5. Another training activity is about assessment methods and interactional expertise within the project assessors; however, larger training for outside need is not in the scope of this project.

Timetable

Table 2. Timeline of the project and tasks.

WP, task	Year 1	Year 2	Year 3	Year 4
WP1
Task 1: Develop practice for including discussions in models	XXXXX
Task 2: Develop practice for eliciting stakeholder values	XXXXX	XXXXX
Task 3: Develop generic impact assessment models	XXXXX	XXXXX	XXXXX
WP2
Task 1: Communicate the practices of the project	XXXXX	x x x	x x x	XXXXX
Task 2: Recruit participants to case studies	XXXXX	x x x
WP3
Task 1: Implement project in Innovillage	XXXXX	x x x	x x x	x x x
Task 2: Compare and merge methods with administrative impact assessments		XXXXX
WP4
Task 1: Create ODDS ecosystem for open decision making	XXXXX	XXXXX
Task 2: Study requirements of interactional expertise		XXXXX	XXXXX
WP5
Task 1: Develop large case studies (mining)	XXXXX
Task 2: Execute large case studies		x x x	XXXXX	XXXXX
Task 3: Develop and maintain small case studies	XXXXX	XXXXX	x x x	x x x
Task 4: Offer training and support for decision makers, assessors, and other stakeholders	XXXXX	x x x	x x x	x x x

Budget

----#: . Budjetti tulee osaksi hakemusta, joten tämä on tässä tiedoksi mutta taulukko poistetaan lopullisesta tutkimussuunnitelmasta tilaa viemästä. --Jouni 17:20, 22 September 2013 (EEST) (type: truth; paradigms: science: comment)

Most of the costs occur as personnel costs in workpackages other than WP4. The justification is given in the WP descriptions, and the timetable above shows roughly the reasoning for the distribution of the costs over time. The emphasis of work on WP1, WP3, and WP4 is in the first half, while in WP2 and WP5 there is also a fair amount of work in the end. Training, outside collaboration, and case study management will also create service costs (mostly in WP4 and WP5), as not all of that work is done within THL. Additional funding for ecosystem development and collaboration will be applied from elsewhere. Travel costs are for visits to case study municipalities, 5 - 10 visits per year. The project budget is calculated for the period 1.9.2014 - 31.8.2018 (48 months). The indirect personnel costs in THL are 55 %, and overhead is 61 %. Jouni Tuomisto (PI) will spend 20 % of his working time on this project; this resource comes from the THL budget.

Table 3. Project budget

	2014	2015	2016	2017	2018	Total
Salaries
Postdoctoral researcher 3200 e/pmo (WP1+WP5)	4	10.5	10.5	10.5	7	13600
Assisting personnel 2700 e/pmo (WP2)	4	5.5	5.5	5.5	5.5	70200
Postdoctoral researcher 3200 e/pmo (WP3)	3	6	6	5.5	2	72000
Salaries, total	33200	67650	67650	66050	43650	278200
Indirect employee costs, total	18260	37208	37208	36328	24008	153012
Total overheads share	31391	63963	63963	62451	41271	263039
Other costs
Services	6000	15000	15000	15000	9000	60000
Travel expences	2000	2000	2000	2000	2000	10000
Other costs, total	8000	17000	17000	17000	11000	70000
Total costs	90851	185821	185821	181829	119929	764251
Funding plan
Own organisation	27255	55750	55750	54550	35980	229285
Funding contribution from other sources %	30.00	30.00	30.00	30.00	30.00	30.00
Academy funding contribution €	63596	130071	130071	127279	83949	534966
Academy funding contribution %	70.00	70.00	70.00	70.00	70.00	70.00

Research environment

Merits of research team members

Team in THL / Department of Environmental Health: Chief researcher Jouni Tuomisto (MD, Dr Med Sci, adjunct professor) has 20 years of expertise in environmental health, toxicology, risk assessment, decision analysis, and decision support. He is a key person in the development of open assessment, Opasnet, and ODDS practice. Researcher Päivi Meriläinen (PhD) has a key role in quantitative microbial risk assessment (QMRA) development at THL. She has 10 years of experience on risk assessment and has been involved with several EU-funded projects on environmental health research (INTARESE, HiWATE, SecurEau) with special focus on drinking water risk assessment. Hannu Komulainen (research professor) is a toxicologist with wide and long experience in toxicology and health risk assessment of different chemical contaminants (heavy metals, chemical contaminants in indoor air, drinking water, contaminated soils, mine environments etc.). His main contribution in the project will be implementation and dissemination of risk assessment methods for decision making. Ilkka Miettinen (chief researcher, title of docent) is an expert in exposure to harmful microbes originating from different water environments. He has participated in numerous national and international research projects during the last 20 years. He has many expert tasks concerning water purification, water quality monitoring and water safety and is the leader of national task group participating in waterborne outbreaks.

Team in THL / Service System Department and Department of Health, Functional Capacity and Welfare: Pasi Pohjola (PhD, Social Sciences, Development Manager) coordinates the implementation of KASTE, the National Development Programme for Social Welfare and Health Care. Previously he has been responsible for developing Innovillage, national open innovation environment for social care and health services. Previously he studied knowledge building and collaborative creativity in the University of Helsinki. Tapani Kauppinen (PhD) is a chief developer in the Unit of Health and Social Inequalities.

Team in THL / Department of Communications: Mervi Pitkänen is the chief editor of the THL websites and a long-term communications expert. She is responsible for communications of the Division of Health Protection. Kaarina Wilskman is a chief developer and the responsible person for communications of the Division of Health and Social Services.

Team of visiting researchers: Sami Majaniemi (PhD, MSc. (Tech), visiting researcher at THL) is project manager at Forum Virium Helsinki with 20 years of experience in international research collaboration in the fields of theoretical physics and materials science. More recently, his work has focused on the development of tools and practices of collaborative decision making and policy analysis through such programs as Action Programme on eServices and eDemocracy and Open Government Partnership Initiative coordinated by the Ministry of Finance. Mikko Pohjola (PhD, MSc. (Tech), visiting researcher at THL) is a research consultant in Nordem Ltd. He made his doctoral thesis on effective decision support by environmental health assessment and is the other main developer of open assessment, Opasnet and ODDS practice. He has worked in several research projects both internationally and nationally (e.g. INTARESE, HEIMTSA, BENERIS, Tekaisu) with particular emphasis on knowledge practices to advance health and wellbeing in societies. Leo Lahti (D.Sc. (Tech.); B.Sc. (Pol. Sci.), postdoctoral research fellow) is affiliated with University of Helsinki, Finland and Wageningen University, Netherlands. He has specialized in machine learning and applied probabilistic analysis and data integration, with applications in computational biology and open government data. He develops open source algorithmic tools for these topics. He is actively involved in the domestic and international open government data community by e.g. coordinating a Finnish Open Science work group. He is a main developer of the sorvi toolkit for Finnish open government data.

Site of research. The work is done mostly in THL in four different departments (see personnel). THL is a large governmental reasearch and expert institute with strong support to basic and applied research that has clear societal and policy relevance such as this project. THL offers good facilities and infrastructure to the work, including maintenance of Opasnet web-workspace which is a key resource in the project, and typical equipment such as computers. There is neither laboratory work nor field measurements in this project, thus such equipment is not needed.

Key national and international collaboration. The project partners have close relations to many key experts in the area, some of which are mentioned here. However, the project does not contain specified tasks to them. Rather, they are consulted in an informal way as needed, and they are being informed about the development of the project. Prof Roger Cooke in Technical University of Delft (NL) and Resources for the Future (USA) is an expert in decision analysis and Bayesian networks, and he has developed the methods of expert elicitation and stakeholder preference elicitation, both of which are used in the project. Prof Jyri Seppälä from the Finnish Environment Institute and lecturer Gregory Norris from Harvard University (USA) are experts in life cycle assessment, a key decision support method using quantitative modelling of environmental impacts. Prof John Evans from Cyprus International Institute for Environmental and Public Health is an expert in decision analysis and environmental health risks.

Other partners. Strategic Centres for Science, Technology and Innovation are not involved in the project. Other partners include Nordem Ltd, Open Knowledge Finland, Forum Virium, and Verkkodemokratiaseura (a society promoting online democracy). The project has people who are involved in these organisations and work as a link between them, the project, and stakeholder groups. This is especially important in the case studies. However, these other partners do not have a budget of their own, but the resources are applied mostly from elsewhere. In the budget, there is a total of 60000 € for services. This will be mainly used for training and case study management, and a part of that may direct to these partners, depending on the outcomes of acquisition competitions.

Use of international and national research infrastructure. The project is not affiliated with research infrastructure organisations.

Mobility. There are no planned visits to other research institutes longer than 0.5 months.

Training and careers

All researchers in the project are at least postdoctoral researchers, so doctoral training is not anticipated. However, there is a clear training need for all researchers, because they will apply and facilitate assessment methods in their own respective areas but few of them are actually trained assessors or interactional experts. The skills needed will be trained within the project (see WP5).

Gender equality is an important thing and a real challenge in this project. This is because almost all of the activists in different self-organised open democracy organisations are male in Finland. Therefore, special attention will be paid to make sure that enough female participants are found to case studies from the municipalities.

Expected results

The ODDS project has the practical objective of giving decision support at municipal and national level. If successful, several changes should be seen in a measurable way:

• The use of online tools will clearly increase among case study participants and also elsewhere.
• Impact assessments become more common in decision processes also when it is not required by law.
• More open decision processes will be organised. This can be measured e.g. from the number of such processes launched in Opasnet web-workspace.
• The ODDS ecosystem becomes functional, and people start working together to promote evidence-based decision support and also to participate in decision processes.
• Municipalities and national organisations start seeing open decision support work as an effective and efficient way of working. This will increase the resources allocated to decision support and impact assessment.
• The growing interest in open decision support will increase demand for training of decision support methods.

Bibliography

Links to Opasnet tools and models mentioned in the application: Minera mining risk model · Fine particle emissions and impacts · Water Guide model · Health impact assessment · Energy balance model

Bibliography

• 1. Press release from the Government of Finland, 5th September, 2013. [1]
• 2. The plan of the Government of Jyrki Katainen, 22 June 2011. [2]
• 3. Tuomisto J. A saga of industrial pollution. Science 2013;341:238-239.
• 4. Mervis J. Agencies Rally to Tackle Big Data. Science 2012;336:22.
• 5. Perrings C et al. Science 2011;331:1139-1140.
• 6. Briggs SV, Knight AT. Science 2011;333:696-697.
• 7. Hulme M et al. Science-policy interface: beyond assessments. Science 2011;333:697-698.
• 8. Lankinen T et al. Prime Minister’s Office Publications 3/2012. Edita Prima, Helsinki.
• 9. Junnila M. THL, Tampere, 2012. http://urn.fi/URN:ISBN:978-952-245-527-7
• 10. Jones H. Policy-making as discourse: a review of recent knowledge-to-policy literature. EADI, Bonn, Germany, 2009.
• 11. Pohjola M. Doctoral dissertation. THL, Research 105, Helsinki, 2013. [3]
• 12. Pohjola M, Pohjola P, Tuomisto J. Ympäristö ja terveys 2012;10:6-11. [4]
• 13. Pohjola M et al. Int J Environ Res Public Health 2013;10:2621-2642.
• 14. Sandström V et al. Evaluating effectiveness of open assessments on alternative biofuel sources. Sustainability: Science, Practice & Policy 2013: in press.
• 15. Pohjola M et al. Puijon metsien käyttösuunnitelman päätöksenteko. Opasnet/Puijo
• 16. Opasnetin kirjoittajat: Opasnet. Verkkotyötilan kuvaus. [5]
• 17. Rintala T, Happonen E, Tuomisto J. OpasnetUtils. Version 1.0.0. CRAN, 2013. [6]
• 18. Van Eemeren FH, Grootendorst R. Cambridge, UK, 2004. ISBN 0-521-83075-3
• 19. Blackburn S. Ruling Passions. Clarendon, Oxford, 1998.
• 20. Cooke RM. Stakeholder preference elicitation. In: Environmental Security in Harbors and Coastal Areas. Springer, 2007;149-160.
• 21. Kyttä M and coworkers: Mapita Ltd web tools [7]
• 22. Popper, Karl. Routledge, London, 2004. ISBN 0-415-28594-1.
• 23. Bernardo JM, Smith AFM. Bayesian Theory. John Wiley & Sons Ltd., Chichester, England, 2000.
• 24. Cooke RM. Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New York, 1991. ISBN 9780195064650
• 25. Howard R. Decision analysis: introductory readings on choices under uncertainty. McGraw Hill, 1997. ISBN 0-07-052579-X.
• 26. Tuomisto JT, Pohjola M. Open Risk assessment. National Public Health Institute, 2007.
• 27. Surowiecki J. Wisdom of crowds. Little, Brown & Company 2004. ISBN 0-316-86173-1
• 28. Tapscott D, Williams AD. Wikinomics: how mass collaboration changes everything. Portfolio Trade, 2006. ISBN 1-59184-367-7.
• 29. Pohjola M. Properties of good assessment. Opasnet, 2013. [8]
• 30. Collins H, Evans R. Rethinking expertise. The University of Chicago Press, Chicago, 2007.
• 31. Ministry of Finance: Yhteentoimivuus [9]
• 32. Innovillage, THL, 2013. [10]

See also

• Quasi-realism
• The detailed example of multidimensional decision space (sudoku) was radically shortened. The original story is archived.
• Application files and decision
  • application_275658_complete
  • decision-275658-v1
  • Tuomisto_Jouni_275658_Paneelilausunto_Valmis_27519