Talk:Decision analysis and risk management: Difference between revisions

From Opasnet
Jump to navigation Jump to search
(→‎Case study exercise idea: far from a clear description, but the basic idea should be explicit enough)
 
(13 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Draft synopsis ==
==Comments about DARM course (18.3.2011):==


Introduction to course: content and methods
* Are these methods used in the actual decision situation/in real world?
* Some of issues too complicated (especially Bayes lectures);
* Links of this course to other issues taught in environmental risk assessment;
* More concrete examples would be useful;


CASE 1-2: Intro: how problem emerged, global problem
== Some things to take into account in planning/arranging/conducting the course ==


Risk management: what is managed, who is responsible, what is included? traditional paradigm
# Harkkatyön ohjeistukset: [[:Category:DARM exercise]]. Ainakin RM harkan (osa 2) ohjeistuksessa lienee täsmentämistä. Muutenkin saa parannella niin paljon kuin osaa.
* Look from decision-maker's point of view: needs, communication, and assessment all included
# Kunhan tiedostojen jakelun systeemi saadaan taas tavalla tai toisella pelaamaan, N:\YMAL\Presentations\2011\0228_DARM\materials sisältää jakeluun sopivaa tavaraa. Ainakin NRC:n kirjojen executive summaryt minulla oli alunperin tarkoituksena panna heande-sivulle (m-filesiin).
* Openness: RA, RM, RC are not totally separate
#* Understanding risk –kirjasta en saanut executive summarya pdf:na ladattua nap.edu:sta. Voisi vaikka skannata, jos ei muuten onnistu.
* Performance: Context about what we actually aim to achieve. Ho do we know if we succeeded?
# Luento-/harkka-aikataulu on vielä aika epämääräinen ja sekava, etenkin loppupäästään (RM-osio). Jos kokonaiskuva alkaa kirkastumaan (esim. tämänpäiväisten kokemusten pohjalta), saa täsmentää kurssisivun kalenteriin vaikkapa sen kummemmin minulta kyselemättä.
* Developing risk management options.
# Sikaflunssatarinaakin voisi vähän miettiä. Esim.:
* Development of risk assessment questions.
#* Taustamateriaalilinkkejä [[Decision analysis and risk management#General information sources on AH1N1 influenza (Swine_flu)]] voisi tarkistella. Kirsti on niitä keräillyt ja varmasti ovat tolkullisia, mutta olisihan se hyvä jonkun muunkin tietää mitä kaikkea sieltä löytyy. Olisiko Teemu sopiva henkilö tähän?
* Science-policy interface. Why it does not exist.
#* Aikataulun ja linkkilistan aihe-/vaihejaottelu eivät ole aivan yhdenmukaiset. Sikaflunssastoorituokioiden rajauksia ja niiden sijoittelua aikatauluun voisi varmaankin parantaa/tarkentaa nykyisestä. Jouni & Marko, jos ehditte?
#* Edellisen yhteydessä voisi myös miettiä, jos sikaflunssalinkkejä nostelisi sivun lopusta aikataulun yhteyteen sopiviin kohtiin.
# Kunhan alkaa selviämään kuinka monta ryhmää ja yksilöä tulee harkat tekemään, voidaan miettiä tarvitaanko kahta päivää loppuseminaariin. Nyt seminaarivaraus on aikataulussa sekä 11.4. että 12.4.
# En vielä kovin hyvin tiedä miten RM osio tulee kurssin loppupuolella järjestymään, mutta olisiko ideaa varata 1-2 päivää lopusta koko kurssin kokonaiskuvan kertaamiseen ja esille nousevien/nousseiden erityiskysymysten lisätarkasteluun? Esim. DA summary & specific topics 7.4. ja RM summary & specific topics 8.4. Tai jotain sinne päin.
#* {{comment|# |Ehkä yksikin sessio riittäisi tähän. 8.4. tai 11.4.?|--[[User:Mikko Pohjola|Mikko Pohjola]] 13:16, 4 March 2011 (EET)}}


Intro to Opasnet
--[[User:Mikko Pohjola|Mikko Pohjola]] 13:08, 4 March 2011 (EET)


CASE 3: Preparedness in Finland and internationally
== Draft synopsis ==
 
CASE 4: Vaccination campaigns, counter-campaigns
* Should we launch vaccination campaign?
* Should I take vaccination? Should I not take vaccination?
Decision analysis: introduction: decision trees
* Purpose of assessment: Why it is done
* Concepts: decisions, objectives, optimisation, uncertainty
 
Subjective probabilities, exercise
* Bayesian rule and Bayesian networks.
* Denis Lindley: Philosophy of probabilities
 
CASE 5: First deaths of swine flu (threat was real)
CASE 6: Problems with implementation: long queues in health centers
 
CASE 7: Clear-cut case falls apart. False alarm? Disease was milder than thought.
 
Decision-making under uncertainty
* Assessment performance? Quality of evidence? Impacts of uncertainty in decision-making. Hindsight.
* Use of adjuvants. How similar is the same? Can/Should be aim at zero risk? Benefit-risk comparisons.
* Acceptability
* Value of information
 
 
CASE 8: Secret connections to drug industry?
 
Case 9: Narcolepsy
 
Trialogue, collective learning
* Justified true belief and its problems
* Inference rules: how do we know what we know?
* Shared information vs. private information
* Actions by a group based on shared information
* Shared belief systems
* Risk assessment as a collaborative project of information production.
 
Scientific method, falsification.
* Do we need pre-peer-review?


Discussion section here?
For a draft synopsis of the course, see a [http://en.opasnet.org/en-opwiki/index.php?title=Talk:Decision_analysis_and_risk_management&oldid=18242 previous version].


Opasnet section here?
== Case study exercise ==
CASE 10: Vaccination campaign halted.


CASE 10b: THL remains silent
Case study exercise description and instructions can be found at: [[:Category:DARM exercise]].


CASE 11: Narcolepsy analysis
== Lectures related to uncertainty ==


Why openness is needed
I was briefly discussing with Jouni from the uncertainty, Monte-Carlo and Value of Information path of the lectures.
* Impacts of openness on topics discussed.


Opasnet and other web tools for risk assessment.
I think this “path” has three main pieces:
* Practical and technical things to get started.
* Aim: to learn skills that are needed to perform and participate in practical training.


Discussion section here?
* What is uncertainty, how to identify uncertainty etc. This could include all kind of issues like expert elicitation, data analysis, literature search, expert estimates, modelers estimate etc.
* How to implement uncertainty to the calculations? Thus, how to actually combine uncertainties from different sources and/or how to create uncertainty functions. This could be divided to two different parts, Bayes and Monte-Carlo, since these two are the two most common ways of implementing uncertainty to the assessment.
CASE 12: Publishing of narcolepsy results and apologies
* Third phase consist sensitivity analysis and value of information. Thus, these are methods that are applied after the model is done and after the uncertainties have been implemented.


CASE 13: Searching for the quilty.
I think the basic idea of this is already included in the course structure, but we could try to make it clearer in the lecturing structure?


Discussion: Lectures of concepts
* Pragma-dialectic argumentation theory.
* Parts of argumentation.


Structuring of discussions in practice and theory
== Responsibilities among organizers ==
* Ready-made texts: discussions and organisation
* Homework: evaluate relevance - validity of arguments. Discuss in groups the next day.
* Build a structured discussion out of this in groups.
* Possibly utilise group writing tools?


Case study: practical work
The course basically consists of three temporal/contentual sections:
* Decision analysis
* Revisiting of policy question
* Impact of closedness/openness in this case (opportunity or threat?)
* Risk communication: how should it have been done?


=== Case study exercise idea ===
# Introduction 28.2.-4.3.
# DA 8.3.- 29.3.
# RM 30.3.-12.4.


Imagine that the Ministry of Social and Health affairs of Finland has given you, as an expert in protecting and promoting public health, an assignment to assess the case of AH1N1 (aka swine flu) vaccinations and the sudden increase in narcolepsy among young people in Finland. The minister expects your assessment to shed light on e.g following aspects of the issue:
The two main teaching/studying methods are employed in parallel more or less throughout the course:


* Did something go wrong? If so, what, when, and why?
* Theory lectures and exercises (classroom)
* How could have things been done better? What, when, and why?
* Case discussions and group/independent work (classroom and outside)
* With the knowledge we have now in this situation, what could/should be done?
* What can be learned about this case regarding possible similar urgent public health risk management situations in the future?
* Are there any more general risk management or other lessons to learn from this case?


'''Exercise one: group work (~3-5 people/group)
In order to allow us to concentrate on certain things below is suggestion for distribution of (primary) responsibility on certain issues along the course


Based on the lecture and exercise contents, the materials and discussions regarding the swine flu/narcolepsy case on this course, as well as all your own expertise and opinions, work out a plan how such a study could/should be made. The plan should address at least the following:
'''Intro


# Background description of the case, the most important question related to it, the relevant actors related to the case, and their roles in relation to the case
Jouni and Mikko to agree on who leads which parts, and prepare corresponding teaching materials. Basically exercise introduction and instruction prepared by Mikko, general course arrangement basically a responsibility of Jouni. Marko to participate and help as available/needed.
# Analysis of decision options, and their order of preference, at different times along the progress of the case
#* e.g. what are the relevant decisions, and options, and who are the decision makers?
#* e.g. did the emergence of new knowledge affect what should have been decided at different moments of time?
# Consideration of the quality of various bits of evidence (meaning virtually all relevant information) used as the basis for decisions?
#* e.g. in relation to properties of good assessment
#* e.g. in terms of value of information
#* The relationships between the main factors in the case affecting the decisions made/to be made


''(exercise 1 would correspond roughly to the introduction and methods sections of a scientific article)
'''DA


'''Exercise two: individual work
Marko and Jouni to agree on who leads which parts, and prepare corresponding teaching materials. Also developing the model for demonstrations a responsibility of Jouni and Marko (Can the study plans also be executed with the demonstrator model (updated according to needs of the plans)?). Jouni to create and keep up the linkage between case study/exercise and theory. Mikko will be back to participate in the exercise part 1 presentations and discussions. Also Mikko is available if assessment performance is wanted to be taken up in DM under uncertainty.


''This can also be combined as a part of the group work if so desired
'''RM


{{comment|# |Would it be possible to actually execute the analyses and produce the results according to the plans by using the demonstrator DA model that will (?) be put up in Heande and used in the course?|--[[User:Mikko Pohjola|Mikko Pohjola]] 13:54, 5 February 2011 (UTC)}}
Mikko to plan the story-/timelines for both theory and case for the last section. More detailed distribution of lead responsibilities to be agreed upon by Jouni and Mikko. Follow-up and assistance for post-seminar improvement work on exercises needs to arranged. Also exercise output checks and student evaluations need to be agreed upon.


Following your work done in exercise one, and discussions held regarding different outputs of different groups in exercise one, now consider the possible outputs of your analysis, the conclusions that could be made based on them, and the possibilities for their practical realization. In essence, explain what could have been done, try to reason what should have been done, and discuss the possible impacts of having done otherwise than what actually happened.
'''Other


'''General guidance
In addition there are certain general issues that also need to be agreed upon and taken care of, e.g:
* collection and preparation of study and background materials for both theory and case to be done by whom?
* '''classroom reservations to be done by whom? (us or UEF?)
** '''needs for specific equipment or space needs must be clearly indicated in the course schedule
* video recording / streaming of lectures? (which lectures are suitable?)
* is there a need for an increased continuous Opasnet up-keeping or help desk -service during the course?
* requirements/needs for keeping up participation lists? how is this to be done?


The main point is not to write long and detailed texts of any specific topic within this course. Instead the idea is to try to make use of what has been taught on the course by combining them in relation to a practical question. Most important issues in evaluating the exercises are:
== theory vs. story vs. case study exercise ==


* general clarity of thought
An idea for creating and keeping up a tight linkage between the swine-flu story and the theory lectures as well as case study exercises: Let us make the detailed theory lecture/exercise plans, so that in every possible theory topic there is a connection (at least a weak one) to the swine-flu story. It can be done e.g. by:
* comprehension and description of the big picture
* meaningful interrelations between the aspects of the case
* application of the knowledge and methods provided in lectures, exercises and discussions along the course
* ability to argue for or against different statements or actions


=== Suunnittelussa huomioonotettavaa ===
* taking lecture/exercise examples from the case to demonstrate the theory
* after presenting the theory, making students work out what the particular part of theory would mean in the case context
* making students explore and discuss a certain aspect(s) of the case as an introduction to the particular part of theory, presented afterwards.


* aikataulurajoitteet
The theory and the exercise are now implemented in the schedule to progress in unison. If a good linkage between the case story and theory lectures/exercises is achieved, it also guarantees contentual support for progressing the case study exercises. It is not necessary to be strictly bound to the chronological order of events in the story (the story parts will probably naturally fall into a nearly chronological order if in line with the theory and cases study exercise plans).
* tila-, väline- yms. vaatmukset
* tilavaraukset
* videointi, koneet, verkko?
* DA-mallin kehikko
* infomateriaali omatoimiseen työhön
** case
** käsitteistö ja teoria
* perusasialuennot ? työnjako, suunnitelmat, tausta- ja esitysmateriaalit
* harjoitustehtävät ? peruskäsitteistö yms. / DA-case / kyselyt kurssin kuluessa

Latest revision as of 10:12, 6 April 2011

Comments about DARM course (18.3.2011):

  • Are these methods used in the actual decision situation/in real world?
  • Some of issues too complicated (especially Bayes lectures);
  • Links of this course to other issues taught in environmental risk assessment;
  • More concrete examples would be useful;

Some things to take into account in planning/arranging/conducting the course

  1. Harkkatyön ohjeistukset: Category:DARM exercise. Ainakin RM harkan (osa 2) ohjeistuksessa lienee täsmentämistä. Muutenkin saa parannella niin paljon kuin osaa.
  2. Kunhan tiedostojen jakelun systeemi saadaan taas tavalla tai toisella pelaamaan, N:\YMAL\Presentations\2011\0228_DARM\materials sisältää jakeluun sopivaa tavaraa. Ainakin NRC:n kirjojen executive summaryt minulla oli alunperin tarkoituksena panna heande-sivulle (m-filesiin).
    • Understanding risk –kirjasta en saanut executive summarya pdf:na ladattua nap.edu:sta. Voisi vaikka skannata, jos ei muuten onnistu.
  3. Luento-/harkka-aikataulu on vielä aika epämääräinen ja sekava, etenkin loppupäästään (RM-osio). Jos kokonaiskuva alkaa kirkastumaan (esim. tämänpäiväisten kokemusten pohjalta), saa täsmentää kurssisivun kalenteriin vaikkapa sen kummemmin minulta kyselemättä.
  4. Sikaflunssatarinaakin voisi vähän miettiä. Esim.:
    • Taustamateriaalilinkkejä Decision analysis and risk management#General information sources on AH1N1 influenza (Swine_flu) voisi tarkistella. Kirsti on niitä keräillyt ja varmasti ovat tolkullisia, mutta olisihan se hyvä jonkun muunkin tietää mitä kaikkea sieltä löytyy. Olisiko Teemu sopiva henkilö tähän?
    • Aikataulun ja linkkilistan aihe-/vaihejaottelu eivät ole aivan yhdenmukaiset. Sikaflunssastoorituokioiden rajauksia ja niiden sijoittelua aikatauluun voisi varmaankin parantaa/tarkentaa nykyisestä. Jouni & Marko, jos ehditte?
    • Edellisen yhteydessä voisi myös miettiä, jos sikaflunssalinkkejä nostelisi sivun lopusta aikataulun yhteyteen sopiviin kohtiin.
  5. Kunhan alkaa selviämään kuinka monta ryhmää ja yksilöä tulee harkat tekemään, voidaan miettiä tarvitaanko kahta päivää loppuseminaariin. Nyt seminaarivaraus on aikataulussa sekä 11.4. että 12.4.
  6. En vielä kovin hyvin tiedä miten RM osio tulee kurssin loppupuolella järjestymään, mutta olisiko ideaa varata 1-2 päivää lopusta koko kurssin kokonaiskuvan kertaamiseen ja esille nousevien/nousseiden erityiskysymysten lisätarkasteluun? Esim. DA summary & specific topics 7.4. ja RM summary & specific topics 8.4. Tai jotain sinne päin.
    • ----#: . Ehkä yksikin sessio riittäisi tähän. 8.4. tai 11.4.? --Mikko Pohjola 13:16, 4 March 2011 (EET) (type: truth; paradigms: science: comment)

--Mikko Pohjola 13:08, 4 March 2011 (EET)

Draft synopsis

For a draft synopsis of the course, see a previous version.

Case study exercise

Case study exercise description and instructions can be found at: Category:DARM exercise.

Lectures related to uncertainty

I was briefly discussing with Jouni from the uncertainty, Monte-Carlo and Value of Information path of the lectures.

I think this “path” has three main pieces:

  • What is uncertainty, how to identify uncertainty etc. This could include all kind of issues like expert elicitation, data analysis, literature search, expert estimates, modelers estimate etc.
  • How to implement uncertainty to the calculations? Thus, how to actually combine uncertainties from different sources and/or how to create uncertainty functions. This could be divided to two different parts, Bayes and Monte-Carlo, since these two are the two most common ways of implementing uncertainty to the assessment.
  • Third phase consist sensitivity analysis and value of information. Thus, these are methods that are applied after the model is done and after the uncertainties have been implemented.

I think the basic idea of this is already included in the course structure, but we could try to make it clearer in the lecturing structure?


Responsibilities among organizers

The course basically consists of three temporal/contentual sections:

  1. Introduction 28.2.-4.3.
  2. DA 8.3.- 29.3.
  3. RM 30.3.-12.4.

The two main teaching/studying methods are employed in parallel more or less throughout the course:

  • Theory lectures and exercises (classroom)
  • Case discussions and group/independent work (classroom and outside)

In order to allow us to concentrate on certain things below is suggestion for distribution of (primary) responsibility on certain issues along the course

Intro

Jouni and Mikko to agree on who leads which parts, and prepare corresponding teaching materials. Basically exercise introduction and instruction prepared by Mikko, general course arrangement basically a responsibility of Jouni. Marko to participate and help as available/needed.

DA

Marko and Jouni to agree on who leads which parts, and prepare corresponding teaching materials. Also developing the model for demonstrations a responsibility of Jouni and Marko (Can the study plans also be executed with the demonstrator model (updated according to needs of the plans)?). Jouni to create and keep up the linkage between case study/exercise and theory. Mikko will be back to participate in the exercise part 1 presentations and discussions. Also Mikko is available if assessment performance is wanted to be taken up in DM under uncertainty.

RM

Mikko to plan the story-/timelines for both theory and case for the last section. More detailed distribution of lead responsibilities to be agreed upon by Jouni and Mikko. Follow-up and assistance for post-seminar improvement work on exercises needs to arranged. Also exercise output checks and student evaluations need to be agreed upon.

Other

In addition there are certain general issues that also need to be agreed upon and taken care of, e.g:

  • collection and preparation of study and background materials for both theory and case to be done by whom?
  • classroom reservations to be done by whom? (us or UEF?)
    • needs for specific equipment or space needs must be clearly indicated in the course schedule
  • video recording / streaming of lectures? (which lectures are suitable?)
  • is there a need for an increased continuous Opasnet up-keeping or help desk -service during the course?
  • requirements/needs for keeping up participation lists? how is this to be done?

theory vs. story vs. case study exercise

An idea for creating and keeping up a tight linkage between the swine-flu story and the theory lectures as well as case study exercises: Let us make the detailed theory lecture/exercise plans, so that in every possible theory topic there is a connection (at least a weak one) to the swine-flu story. It can be done e.g. by:

  • taking lecture/exercise examples from the case to demonstrate the theory
  • after presenting the theory, making students work out what the particular part of theory would mean in the case context
  • making students explore and discuss a certain aspect(s) of the case as an introduction to the particular part of theory, presented afterwards.

The theory and the exercise are now implemented in the schedule to progress in unison. If a good linkage between the case story and theory lectures/exercises is achieved, it also guarantees contentual support for progressing the case study exercises. It is not necessary to be strictly bound to the chronological order of events in the story (the story parts will probably naturally fall into a nearly chronological order if in line with the theory and cases study exercise plans).

Retrieved from "https://en.opasnet.org/index.php?title=Talk:Decision_analysis_and_risk_management&oldid=19987"