Decision analysis and risk management: Difference between revisions
m (some rescheduling) |
|||
Line 22: | Line 22: | ||
=== Content === | === Content === | ||
{| | {| | ||
|+ '''Schedule and contents of the course.''' <br>Each row is a three-hour block of lectures, exercises, discussions, or organised group work, in total 69 hours of teaching. In addition, self-organised group work and individual learning is expected to take ca. 100 hours. <br>Changes are likely especially in the content, but the dates presented here are more or less fixed. Abbreviations: RA = risk assessment, DA = decision analysis, RM = risk management, RC = risk communication. | |+ '''Schedule and contents of the course.''' <br>Each row is a three-hour block of lectures, exercises, discussions, or organised group work, in total 69 hours of teaching. In addition, self-organised group work and individual learning is expected to take ca. 100 hours. <br>Changes are likely especially in the content, but the dates presented here are more or less fixed. Abbreviations: RA = risk assessment, DA = decision analysis, RM = risk management, RC = risk communication. | ||
|- | |- | ||
! Date | ! Date | ||
! Time | ! Time | ||
! Lecturer | ! Lecturer | ||
! Lecture room | |||
! Theory lectures and exercises | ! Theory lectures and exercises | ||
! Material | ! Material | ||
! Case study discussions and exercises | ! Case study discussions and exercises | ||
|- | |- | ||
| 28.2. | | 28.2. | ||
| 10-13 | | 10-13 | ||
| '''Jouni''', Mikko, Marko | | '''Jouni''', Mikko, Marko | ||
| | |||
| '''Practical introduction to course.''' (Jouni) | | '''Practical introduction to course.''' (Jouni) | ||
*Objectives, aims, expectations. | *Objectives, aims, expectations. | ||
Line 40: | Line 42: | ||
*Example from risk assessment & management study? | *Example from risk assessment & management study? | ||
| | | | ||
| '''Introduction to the case study and exercises''' (Mikko) | | '''Introduction to the case study and exercises''' (Mikko) | ||
*swine flu vaccination campaign in Finland 2009-2011: effects and side effects? | *swine flu vaccination campaign in Finland 2009-2011: effects and side effects? | ||
Line 47: | Line 49: | ||
|- | |- | ||
| 3.3. | | 3.3. | ||
| 9-12 | | 9-12 | ||
| '''Jouni''', Mikko, Marko | | '''Jouni''', Mikko, Marko | ||
| | |||
| '''[[:File:Introduction to risk management.odp|Introduction to risk management]]'''. What is managed, who is responsible, what is included? Traditional paradigm. | | '''[[:File:Introduction to risk management.odp|Introduction to risk management]]'''. What is managed, who is responsible, what is included? Traditional paradigm. | ||
*Look from decision-maker's point of view: needs, communication, and assessment all included. | *Look from decision-maker's point of view: needs, communication, and assessment all included. | ||
Line 62: | Line 65: | ||
| | | | ||
|- | |- | ||
| 3.3. | | 3.3. | ||
| 13-16 | | 13-16 | ||
| '''Mikko''', Jouni, Marko | | '''Mikko''', Jouni, Marko | ||
| | | | ||
| | | | ||
| | |||
| '''Beginning of the swine flu story:''' How the swine flu problem emerged (spring 2009) and spread in the world. '''[[:File:Swine flu story.odp|Swine flu story]]''' (summer 2009): Preparedness in Finland and internationally. Vaccination campaigns, counter-campaigns. | | '''Beginning of the swine flu story:''' How the swine flu problem emerged (spring 2009) and spread in the world. '''[[:File:Swine flu story.odp|Swine flu story]]''' (summer 2009): Preparedness in Finland and internationally. Vaccination campaigns, counter-campaigns. | ||
*Should we launch a vaccination campaign? | *Should we launch a vaccination campaign? | ||
*Should I take vaccination? Should I not take vaccination? | *Should I take vaccination? Should I not take vaccination? | ||
|- | |- | ||
| 4.3. | | 4.3. | ||
| 9-12 | | 9-12 | ||
| '''Mikko''', Jouni, Marko | | '''Mikko''', Jouni, Marko | ||
| | |||
| '''Introduction to using Opasnet''' | | '''Introduction to using Opasnet''' | ||
*reading and browsing | *reading and browsing | ||
Line 85: | Line 91: | ||
| | | | ||
|- | |- | ||
| 4.3. | | 4.3. | ||
| 13-16 | | 13-16 | ||
| '''Jouni''', Mikko, Marko | | '''Jouni''', Mikko, Marko | ||
| | |||
| '''[[:File:Decision analysis.odp|Decision analysis]]: introduction''' | | '''[[:File:Decision analysis.odp|Decision analysis]]: introduction''' | ||
*Purpose of assessment: Why it is done | *Purpose of assessment: Why it is done | ||
*Concepts: decisions, objectives, optimization, uncertainty, decision trees | *Concepts: decisions, objectives, optimization, uncertainty, decision trees | ||
*Measures of outcome: [[ | *Measures of outcome: [[Utility]], [[DALY]], [[QALY]], [[WTP]] | ||
'''[[:File:Probability theory.odp|Probability theory]]''' | '''[[:File:Probability theory.odp|Probability theory]]''' | ||
*[[Subjective probability]] | |||
*[[Subjective probability]] | |||
*Practical exercise | *Practical exercise | ||
*Bayesian rule and Bayesian networks | *Bayesian rule and Bayesian networks | ||
*Importance of exhaustiveness (coverage) | *Importance of exhaustiveness (coverage) | ||
| *[http://heande.opasnet.org/heande/extensions/mfiles/mf_getfile.php?anon=true&docid=3365&fileid=3365&filename=Lindley_PhilosophyOfStatistics_JRSS_2000.pdf Dennis Lindley: Philosophy of statistics] | |||
| *[http://heande.opasnet.org/heande/extensions/mfiles/mf_getfile.php?anon=true&docid=3365&fileid=3365&filename=Lindley_PhilosophyOfStatistics_JRSS_2000.pdf Dennis Lindley: Philosophy of statistics] | |||
| '''Exercise part one begins:''' make a decision analysis study plan to find out good policies in this case. Which models to use, what to assess? | | '''Exercise part one begins:''' make a decision analysis study plan to find out good policies in this case. Which models to use, what to assess? | ||
|- | |- | ||
| 8.3. | | 8.3. | ||
| 9-12 | | 9-12 | ||
| Jouni | | Jouni | ||
| | |||
| '''Decision analysis (continued)'''. | | '''Decision analysis (continued)'''. | ||
| | | | ||
| '''Swine flu case as a decision problem:''' what are the relevant decisions, outcomes and questions in the case? | | '''Swine flu case as a decision problem:''' what are the relevant decisions, outcomes and questions in the case? | ||
|- | |- | ||
| 9.3. | | 9.3. | ||
| 9-12 | | 9-12 | ||
| Jouni | | Jouni | ||
| | |||
| '''Decision analysis (continued)'''. | | '''Decision analysis (continued)'''. | ||
| | | | ||
| | | | ||
|- | |- | ||
| 10.3. | | 10.3. | ||
| 9-12 | | 9-12 | ||
| Marko | | Marko | ||
| | |||
| '''Modeling and Monte Carlo simulation''' | | '''Modeling and Monte Carlo simulation''' | ||
*What is modeling or model? | *What is modeling or model? | ||
Line 132: | Line 144: | ||
| '''Swine flu story''' (autumn - winter 2009): First deaths of swine flu (threat was real). Problems with implementation: long queues in health centers. Clear-cut case falls apart. False alarm? Disease was milder than thought. | | '''Swine flu story''' (autumn - winter 2009): First deaths of swine flu (threat was real). Problems with implementation: long queues in health centers. Clear-cut case falls apart. False alarm? Disease was milder than thought. | ||
|- | |- | ||
| 11.3. | | 11.3. | ||
| 8-11 | | 8-11 | ||
| Marko | | Marko | ||
| | |||
| '''Calculation exercise'''. | | '''Calculation exercise'''. | ||
*Case: Mortality due to PM in Kuopio: | *Case: Mortality due to PM in Kuopio: | ||
**Exposure-response function | **Exposure-response function | ||
**Exposure concentration | **Exposure concentration | ||
**Background mortality | **Background mortality | ||
| | | | ||
| | | | ||
|- | |- | ||
| 16.3. | | 16.3. | ||
| 9-12 | | 9-12 | ||
| Jouni | | Jouni | ||
| | |||
| '''[[Bayesian updating]]''' | | '''[[Bayesian updating]]''' | ||
*[[:File:Probability theory.odp|Probability theory.odp]] | *[[:File:Probability theory.odp|Probability theory.odp]] | ||
| | | | ||
| | | | ||
|- | |- | ||
| 17.3. | | 17.3. | ||
| 9-12 | | 9-12 | ||
| Jouni | | Jouni | ||
| | |||
| '''Calculation exercise of Bayes''' | | '''Calculation exercise of Bayes''' | ||
| | | | ||
| | | | ||
|- | |- | ||
| 18.3. | | 18.3. | ||
| 13-16 | | 13-16 | ||
| Jouni | | Jouni | ||
| | |||
| '''Overall view of different models''': [[Producing result from rationale]]. | | '''Overall view of different models''': [[Producing result from rationale]]. | ||
* [[:File:Overall view of models.odp|Overall view of models.odp]] | *[[:File:Overall view of models.odp|Overall view of models.odp]] | ||
*Deterministic, heuristic, and probabilistic estimates. | *Deterministic, heuristic, and probabilistic estimates. | ||
*Functional, differential, logical, and probabilistic relations. | *Functional, differential, logical, and probabilistic relations. | ||
Line 171: | Line 188: | ||
| '''First drafts of DA study plans.''' Brief presentations by students of and discussions about DA study plan drafts. | | '''First drafts of DA study plans.''' Brief presentations by students of and discussions about DA study plan drafts. | ||
|- | |- | ||
| 21.3. | | 21.3. | ||
| 9-12 | | 9-12 | ||
| Jouni, Marko, Mikko | | Jouni, Marko, Mikko | ||
| | |||
| '''Decision making under uncertainty''' | | '''Decision making under uncertainty''' | ||
* [[:File:Evaluating an assessment.odp|Evaluating an assessment.odp]] | *[[:File:Evaluating an assessment.odp|Evaluating an assessment.odp]] | ||
*Uncertainty and it's impact (1h Jouni) | *Uncertainty and it's impact (1h Jouni) | ||
*Sensitivity analysis (1h Marko) | *Sensitivity analysis (1h Marko) | ||
*Value of information (1h Marko) | *Value of information (1h Marko) | ||
*Acceptability | *Acceptability | ||
*Analysis of model inputs, outputs, and structure | *Analysis of model inputs, outputs, and structure | ||
Line 185: | Line 203: | ||
| '''Swine flu story:''' Use of adjuvants. How similar is the same? Can/Should be aim at zero risk? Benefit-risk comparisons. | | '''Swine flu story:''' Use of adjuvants. How similar is the same? Can/Should be aim at zero risk? Benefit-risk comparisons. | ||
|- | |- | ||
| 22.3. | | 22.3. | ||
| 9-12 | | 9-12 | ||
| Jouni, Mikko, Marko | | Jouni, Mikko, Marko | ||
| | |||
| '''Decision-making under uncertainty (continued)'''. [[Value of information]], [[Value of information analysis]]. | | '''Decision-making under uncertainty (continued)'''. [[Value of information]], [[Value of information analysis]]. | ||
*Assessment performance? Quality of evidence? Impacts of uncertainty in decision-making. Hindsight. (Mikko) | *Assessment performance? Quality of evidence? Impacts of uncertainty in decision-making. Hindsight. (Mikko) | ||
Line 194: | Line 213: | ||
| | | | ||
|- | |- | ||
| 25.3. | | 25.3. | ||
| 9-12 | | 9-12 | ||
| Marko | | Marko | ||
| | |||
| '''VOI calculation exercise. ''' | | '''VOI calculation exercise. ''' | ||
*Case 1: Weather example: | *Case 1: Weather example: | ||
Line 209: | Line 229: | ||
| | | | ||
|- | |- | ||
| 29.3. | | 29.3. | ||
| 9-12 | | 9-12 | ||
| Jouni, Mikko | | Jouni, Mikko | ||
| | | | ||
| | | | ||
| | |||
| '''Full drafts of DA study plans.''' Presentations by students of and discussions about study plans. Apply the model to assess the case based on your own research plan. Defend your plan in plenary. | | '''Full drafts of DA study plans.''' Presentations by students of and discussions about study plans. Apply the model to assess the case based on your own research plan. Defend your plan in plenary. | ||
*Are plans executable? What outputs are expected? Do they serve the intended needs? | *Are plans executable? What outputs are expected? Do they serve the intended needs? | ||
Line 221: | Line 242: | ||
|- | |- | ||
| 30.3. | | 30.3. | ||
| 9-12 | | 9-12 | ||
| Mikko, Jouni | | Mikko, Jouni | ||
| | |||
| '''Risk management''' (theory). What is managed, who is responsible, what is included? Traditional paradigm. | | '''Risk management''' (theory). What is managed, who is responsible, what is included? Traditional paradigm. | ||
*Look from decision-maker's point of view: needs, communication, and assessment all included. | *Look from decision-maker's point of view: needs, communication, and assessment all included. | ||
Line 240: | Line 262: | ||
|- | |- | ||
| 31.3. | | 31.3. | ||
| 9-12 | | 9-12 | ||
| Jouni, Mikko | | Jouni, Mikko | ||
| | |||
| '''[[Trialogue]], collective learning''' | | '''[[Trialogue]], collective learning''' | ||
* [[:File:Shared understanding.odp|Shared understanding.odp]] | *[[:File:Shared understanding.odp|Shared understanding.odp]] | ||
*[[:en:Justified true belief|Justified true belief]] and its problems | *[[:en:Justified true belief|Justified true belief]] and its problems | ||
*[[Inference rules]]: how do we know what we know? | *[[Inference rules]]: how do we know what we know? | ||
Line 252: | Line 275: | ||
*Risk assessment as a collaborative project of information production. | *Risk assessment as a collaborative project of information production. | ||
| | | | ||
| '''Swine flu story:''' Secret connections to drug industry? Narcolepsy. | | '''Swine flu story:''' Secret connections to drug industry? Narcolepsy. | ||
|- | |- | ||
| 1.4. | | 1.4. | ||
| 9-12 | | 9-12 | ||
| Jouni, Mikko | | Jouni, Mikko | ||
| | |||
| '''[[Inference rules]]. [[Scientific method]]''' as open discussion. [[Falsification]]. | | '''[[Inference rules]]. [[Scientific method]]''' as open discussion. [[Falsification]]. | ||
*Do we need pre-peer-review? | *Do we need pre-peer-review? | ||
| | | | ||
| | | | ||
|- | |- | ||
| 7.4. | | 7.4. | ||
| 13-16 | | 13-16 | ||
| Mikko | | Mikko | ||
| | |||
| | | | ||
'''[[Discussion]]''': Lectures of concepts | '''[[Discussion]]''': Lectures of concepts | ||
Line 283: | Line 308: | ||
| '''Swine flu story (autumn 2010)''': Vaccination campaign halted. THL remains silent about possible risks. Narcolepsy analysis. Impacts of openness. How methods learned should be applied now? Why openness is needed? | | '''Swine flu story (autumn 2010)''': Vaccination campaign halted. THL remains silent about possible risks. Narcolepsy analysis. Impacts of openness. How methods learned should be applied now? Why openness is needed? | ||
|- | |- | ||
| 8.4. | | 8.4. | ||
| 9-12 | | 9-12 | ||
| Mikko | | Mikko | ||
| | | | ||
| | |||
| | | | ||
| '''Swine flu story (January 2011):''' Publishing of narcolepsy results and apologies. Searching for the guilty. | | '''Swine flu story (January 2011):''' Publishing of narcolepsy results and apologies. Searching for the guilty. | ||
|- | |- | ||
| 11.4. | | 11.4. | ||
| 9-12 | | 9-12 | ||
| Jouni, Mikko | | Jouni, Mikko | ||
| | | | ||
| | |||
| | | | ||
| '''Final seminar:''' Presentations (exercise parts one and two), discussions, and summary of perspectives to the case. | | '''Final seminar:''' Presentations (exercise parts one and two), discussions, and summary of perspectives to the case. | ||
|- | |- | ||
| 12.4. | | 12.4. | ||
| 8-11 | | 8-11 | ||
| Jouni, Mikko | | Jouni, Mikko | ||
| | |||
| '''Wrap-up and course feedback.''' | | '''Wrap-up and course feedback.''' | ||
| | | | ||
Line 307: | Line 335: | ||
'''For the case study description, see {{disclink|Case study exercise (draft)}}.''' | '''For the case study description, see {{disclink|Case study exercise (draft)}}.''' | ||
<br> | |||
;Learning outcomes | ;Learning outcomes |
Revision as of 15:33, 22 February 2011
This page is a lecture.
The page identifier is Op_en3047 |
---|
Moderator:Jouni (see all) |
|
Upload data
|
Decision analysis and risk management is a university course [1] for Master's Degree Programme in General Toxicology and Environmental Health Risk Assessment ToxEn.
Scope
What is such a good course content about decision analysis and risk management that
- it gives a good overview of modern assessment methods,
- is not overlapping with other courses in the MSc program,
- utilises modern web workspaces for learning,
Result
- Name
- Decision analysis and risk management
- ECTS-credits
- 6
- Abbreviation
- DARM
Content
Date | Time | Lecturer | Lecture room | Theory lectures and exercises | Material | Case study discussions and exercises |
---|---|---|---|---|---|---|
28.2. | 10-13 | Jouni, Mikko, Marko | Practical introduction to course. (Jouni)
|
Introduction to the case study and exercises (Mikko)
| ||
3.3. | 9-12 | Jouni, Mikko, Marko | Introduction to risk management. What is managed, who is responsible, what is included? Traditional paradigm.
|
|||
3.3. | 13-16 | Mikko, Jouni, Marko | Beginning of the swine flu story: How the swine flu problem emerged (spring 2009) and spread in the world. Swine flu story (summer 2009): Preparedness in Finland and internationally. Vaccination campaigns, counter-campaigns.
| |||
4.3. | 9-12 | Mikko, Jouni, Marko | Introduction to using Opasnet
|
|||
4.3. | 13-16 | Jouni, Mikko, Marko | Decision analysis: introduction
|
*Dennis Lindley: Philosophy of statistics | Exercise part one begins: make a decision analysis study plan to find out good policies in this case. Which models to use, what to assess? | |
8.3. | 9-12 | Jouni | Decision analysis (continued). | Swine flu case as a decision problem: what are the relevant decisions, outcomes and questions in the case? | ||
9.3. | 9-12 | Jouni | Decision analysis (continued). | |||
10.3. | 9-12 | Marko | Modeling and Monte Carlo simulation
|
Swine flu story (autumn - winter 2009): First deaths of swine flu (threat was real). Problems with implementation: long queues in health centers. Clear-cut case falls apart. False alarm? Disease was milder than thought. | ||
11.3. | 8-11 | Marko | Calculation exercise.
|
|||
16.3. | 9-12 | Jouni | Bayesian updating | |||
17.3. | 9-12 | Jouni | Calculation exercise of Bayes | |||
18.3. | 13-16 | Jouni | Overall view of different models: Producing result from rationale.
|
First drafts of DA study plans. Brief presentations by students of and discussions about DA study plan drafts. | ||
21.3. | 9-12 | Jouni, Marko, Mikko | Decision making under uncertainty
|
Swine flu story: Use of adjuvants. How similar is the same? Can/Should be aim at zero risk? Benefit-risk comparisons. | ||
22.3. | 9-12 | Jouni, Mikko, Marko | Decision-making under uncertainty (continued). Value of information, Value of information analysis.
|
|||
25.3. | 9-12 | Marko | VOI calculation exercise.
|
|||
29.3. | 9-12 | Jouni, Mikko | Full drafts of DA study plans. Presentations by students of and discussions about study plans. Apply the model to assess the case based on your own research plan. Defend your plan in plenary.
| |||
30.3. | 9-12 | Mikko, Jouni | Risk management (theory). What is managed, who is responsible, what is included? Traditional paradigm.
Opasnet and other web-based decision support systems. |
Exercise part two begins: Risk management actions and options. | ||
31.3. | 9-12 | Jouni, Mikko | Trialogue, collective learning
|
Swine flu story: Secret connections to drug industry? Narcolepsy. | ||
1.4. | 9-12 | Jouni, Mikko | Inference rules. Scientific method as open discussion. Falsification.
|
|||
7.4. | 13-16 | Mikko |
Discussion: Lectures of concepts
Structuring of discussions in practice and theory
|
Swine flu story (autumn 2010): Vaccination campaign halted. THL remains silent about possible risks. Narcolepsy analysis. Impacts of openness. How methods learned should be applied now? Why openness is needed? | ||
8.4. | 9-12 | Mikko | Swine flu story (January 2011): Publishing of narcolepsy results and apologies. Searching for the guilty. | |||
11.4. | 9-12 | Jouni, Mikko | Final seminar: Presentations (exercise parts one and two), discussions, and summary of perspectives to the case. | |||
12.4. | 8-11 | Jouni, Mikko | Wrap-up and course feedback. | Final seminar (cont'd). |
For the case study description, see D↷.
- Learning outcomes
- The student will learn about the fundaments of decision analysis; and the connections between societal decision-making and risk assessment. He is able to apply the scientific method and falsification in the context of risk assessment. He also knows how to build an assessment based on the requirements of risk management.
- Learning material
- Online course readings in Opasnet: http://en.opasnet.org
- Degree Programme(s)
- ToxEn, Environment health risk assessment
Implementation
- Time
- Spring 2011
Teaching / Learning Methods
- Lectures 36 h
- Tutorial 0 h
- One-to-One Teaching 0 h
- Group Work 30 h
- Independent Study 96 h
- Web-Based Learning 30 %
- Additional Information on Teaching / Learning Methods
- Self-organised group work requires on-line computers.
- Teaching Languages
- English
- Additional Information on Languages
- Offered for Students in Other Universities (JOO-opintoja)
- YES
ASSESSMENT
- Assessment is Based on
- Participation in lectures and discussions.
- Case study decision analysis plan in groups.
- Case study risk management plan individually.
- Additional Information
- The case studies will be evaluated based on contributions in Opasnet.
- Assessment
- 0 (Fail) – 5 (Excellent)
- Additional Information
- Prerequisites
Contacts
- Organising Departments
- Department of environmental science (in collaboration with the National Institute for Health and Welfare)
- Course Director
- Jukka Juutilainen
- Teachers
- Jouni Tuomisto, Mikko Pohjola, Marko Tainio
- Contact Persons
- mikko.pohjola(at)thl.fi
- Registration for the Course
- Wossikka
Further information
- Keywords
- Risk assessment, decision analysis, Bayesian theory, open assessment, risk management, societal decision making.
- Further Information
Rationale
Background information from other available Risk assessment study programs are provided here: Decision analysis and risk management - background information
Other relevant courses than may have supporting, similar, or overlapping content.
- Environmental Health for International Students, 5 op
- Toxicological and Environmental Risk Assessment and Risk Communication, 3 op
- Advanced Toxicological Risk Assessment, 9 op Note! Group meetings 24.02.11 to 10.00-12.00, 17.03.11 to 13.00-15.00
- Decision analysis and risk management, 6 op This course!
- Ecotoxicology and Risk Assessment, 4 op
- Environmental health risk assessment, 4 op
- The course Ympäristöterveys is openly available under Aducate.
See also discussions about the course content. D↷
See also
- Course contents in UEF website
- Ympäristöterveys-kurssi Avoimessa yliopistossa (Environmental health course in the Open University; in Finnish)
- Course material (in Finnish)
- Monimediainen oppimisympäristö ympäristöterveydestä
- Keeney (2006): Decision Science programs in the USA
Related further education courses (in Finnish, organised in Kuopio)
Täydennyskoulutuskursseja DARM-kurssin aiheista järjestetään osana ERACedu-riskinarviointikoulutusta. Koulutus järjestetään yksipäiväisinä koulutuksina Kuopiossa. Koulutuksessa voi osallistua yhteen tai useampaan päivään; muiden osien käyminen ei ole välttämätöntä, mutta samassa kuussa järjestettävät koulutukset ovat aihepiiriltään lähellä toisiaan.
- Päätösanalyysia ja riskinhallintaa tapausesimerkin kautta ("Mini-DARM") (5.5.2011)
- Sikainfluenssatapauksen pohjalta toteutettu johdanto päätösanalyysiin ja riskinhallinnan perusteisiin.
- Päätöksenteko epävarmuuden oloissa (19.5.2011)
- Päätösanalyysi ja -mallitus
- Keskustelu ja kansalaiskuuleminen päätöksenteossa: menetelmiä ja työkaluja (8.9.2011)
- Kuinka järjestetään tehokkaasti kansalaiskuulemisia ja avointa osallistumista päätöksentekoon tai vaikutusarviointeihin?
- Yhteisöllinen oppiminen (29.9.2011)
- Osallistava arviointi ja mallitus.
References
Decision analysis
Risk management
Swine flu
Related files
<mfanonymousfilelist></mfanonymousfilelist>