Talk:Decision analysis and risk management: Difference between revisions

From Opasnet
Jump to navigation Jump to search
(→‎Case study exercise idea: Perhaps a bit more clear description now)
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Draft synopsis ==
==Comments about DARM course (18.3.2011):==


Introduction to course: content and methods
* Are these methods used in the actual decision situation/in real world?
* Some of issues too complicated (especially Bayes lectures);
* Links of this course to other issues taught in environmental risk assessment;
* More concrete examples would be useful;


CASE 1-2: Intro: how problem emerged, global problem
== Some things to take into account in planning/arranging/conducting the course ==


Risk management: what is managed, who is responsible, what is included? traditional paradigm
# Harkkatyön ohjeistukset: [[:Category:DARM exercise]]. Ainakin RM harkan (osa 2) ohjeistuksessa lienee täsmentämistä. Muutenkin saa parannella niin paljon kuin osaa.
* Look from decision-maker's point of view: needs, communication, and assessment all included
# Kunhan tiedostojen jakelun systeemi saadaan taas tavalla tai toisella pelaamaan, N:\YMAL\Presentations\2011\0228_DARM\materials sisältää jakeluun sopivaa tavaraa. Ainakin NRC:n kirjojen executive summaryt minulla oli alunperin tarkoituksena panna heande-sivulle (m-filesiin).
* Openness: RA, RM, RC are not totally separate
#* Understanding risk –kirjasta en saanut executive summarya pdf:na ladattua nap.edu:sta. Voisi vaikka skannata, jos ei muuten onnistu.
* Performance: Context about what we actually aim to achieve. Ho do we know if we succeeded?
# Luento-/harkka-aikataulu on vielä aika epämääräinen ja sekava, etenkin loppupäästään (RM-osio). Jos kokonaiskuva alkaa kirkastumaan (esim. tämänpäiväisten kokemusten pohjalta), saa täsmentää kurssisivun kalenteriin vaikkapa sen kummemmin minulta kyselemättä.
* Developing risk management options.
# Sikaflunssatarinaakin voisi vähän miettiä. Esim.:
* Development of risk assessment questions.
#* Taustamateriaalilinkkejä [[Decision analysis and risk management#General information sources on AH1N1 influenza (Swine_flu)]] voisi tarkistella. Kirsti on niitä keräillyt ja varmasti ovat tolkullisia, mutta olisihan se hyvä jonkun muunkin tietää mitä kaikkea sieltä löytyy. Olisiko Teemu sopiva henkilö tähän?
* Science-policy interface. Why it does not exist.
#* Aikataulun ja linkkilistan aihe-/vaihejaottelu eivät ole aivan yhdenmukaiset. Sikaflunssastoorituokioiden rajauksia ja niiden sijoittelua aikatauluun voisi varmaankin parantaa/tarkentaa nykyisestä. Jouni & Marko, jos ehditte?
#* Edellisen yhteydessä voisi myös miettiä, jos sikaflunssalinkkejä nostelisi sivun lopusta aikataulun yhteyteen sopiviin kohtiin.
# Kunhan alkaa selviämään kuinka monta ryhmää ja yksilöä tulee harkat tekemään, voidaan miettiä tarvitaanko kahta päivää loppuseminaariin. Nyt seminaarivaraus on aikataulussa sekä 11.4. että 12.4.
# En vielä kovin hyvin tiedä miten RM osio tulee kurssin loppupuolella järjestymään, mutta olisiko ideaa varata 1-2 päivää lopusta koko kurssin kokonaiskuvan kertaamiseen ja esille nousevien/nousseiden erityiskysymysten lisätarkasteluun? Esim. DA summary & specific topics 7.4. ja RM summary & specific topics 8.4. Tai jotain sinne päin.
#* {{comment|# |Ehkä yksikin sessio riittäisi tähän. 8.4. tai 11.4.?|--[[User:Mikko Pohjola|Mikko Pohjola]] 13:16, 4 March 2011 (EET)}}


Intro to Opasnet
--[[User:Mikko Pohjola|Mikko Pohjola]] 13:08, 4 March 2011 (EET)


CASE 3: Preparedness in Finland and internationally
== Draft synopsis ==
 
CASE 4: Vaccination campaigns, counter-campaigns
* Should we launch vaccination campaign?
* Should I take vaccination? Should I not take vaccination?
Decision analysis: introduction: decision trees
* Purpose of assessment: Why it is done
* Concepts: decisions, objectives, optimisation, uncertainty
 
Subjective probabilities, exercise
* Bayesian rule and Bayesian networks.
* Denis Lindley: Philosophy of probabilities
 
CASE 5: First deaths of swine flu (threat was real)
CASE 6: Problems with implementation: long queues in health centers
 
CASE 7: Clear-cut case falls apart. False alarm? Disease was milder than thought.
 
Decision-making under uncertainty
* Assessment performance? Quality of evidence? Impacts of uncertainty in decision-making. Hindsight.
* Use of adjuvants. How similar is the same? Can/Should be aim at zero risk? Benefit-risk comparisons.
* Acceptability
* Value of information
 
 
CASE 8: Secret connections to drug industry?
 
Case 9: Narcolepsy
 
Trialogue, collective learning
* Justified true belief and its problems
* Inference rules: how do we know what we know?
* Shared information vs. private information
* Actions by a group based on shared information
* Shared belief systems
* Risk assessment as a collaborative project of information production.
 
Scientific method, falsification.
* Do we need pre-peer-review?
 
Discussion section here?
 
Opasnet section here?
CASE 10: Vaccination campaign halted.
 
CASE 10b: THL remains silent
 
CASE 11: Narcolepsy analysis


Why openness is needed
For a draft synopsis of the course, see a [http://en.opasnet.org/en-opwiki/index.php?title=Talk:Decision_analysis_and_risk_management&oldid=18242 previous version].
* Impacts of openness on topics discussed.


Opasnet and other web tools for risk assessment.
== Case study exercise ==
* Practical and technical things to get started.
* Aim: to learn skills that are needed to perform and participate in practical training.


Discussion section here?
Case study exercise description and instructions can be found at: [[:Category:DARM exercise]].
CASE 12: Publishing of narcolepsy results and apologies


CASE 13: Searching for the quilty.
== Lectures related to uncertainty ==


Discussion: Lectures of concepts
I was briefly discussing with Jouni from the uncertainty, Monte-Carlo and Value of Information path of the lectures.
* Pragma-dialectic argumentation theory.
* Parts of argumentation.


Structuring of discussions in practice and theory
I think this “path” has three main pieces:
* Ready-made texts: discussions and organisation
* Homework: evaluate relevance - validity of arguments. Discuss in groups the next day.
* Build a structured discussion out of this in groups.
* Possibly utilise group writing tools?


Case study: practical work
* What is uncertainty, how to identify uncertainty etc. This could include all kind of issues like expert elicitation, data analysis, literature search, expert estimates, modelers estimate etc.
* Decision analysis
* How to implement uncertainty to the calculations? Thus, how to actually combine uncertainties from different sources and/or how to create uncertainty functions. This could be divided to two different parts, Bayes and Monte-Carlo, since these two are the two most common ways of implementing uncertainty to the assessment.
* Revisiting of policy question
* Third phase consist sensitivity analysis and value of information. Thus, these are methods that are applied after the model is done and after the uncertainties have been implemented.
* Impact of closedness/openness in this case (opportunity or threat?)
* Risk communication: how should it have been done?


=== Case study exercise idea ===
I think the basic idea of this is already included in the course structure, but we could try to make it clearer in the lecturing structure?


Imagine that the Ministry of Social and Health affairs of Finland has given you an assignment to assess the case of AH1N1 (aka swine flu) vaccinations and the sudden increase in narcolepsy among young people in Finland. Consider yourself as an expert in protecting and promoting public health. The minister expects your assessment to shed light on e.g following aspects of the issue:


* Did something go wrong? If so, what, when, and why?
== Responsibilities among organizers ==
* How could have things been done better? What, when, and why?
* With the knowledge we have now in this situation, what could/should be done?
* What can be learned about this case regarding possible similar urgent public health risk management situations in the future?
* Are there any more general risk management or other lessons to learn from this case?


The case study exercise is done in two parts; the first pertaining more to decision analysis (DA), the second more to risk management (RM). The first part is group work, while the second is individual work (can also be agreed otherwise if needed).
The course basically consists of three temporal/contentual sections:


'''Part one: Decision analysis study plan
# Introduction 28.2.-4.3.
# DA 8.3.- 29.3.
# RM 30.3.-12.4.


''group work (~3-5 people/group)
The two main teaching/studying methods are employed in parallel more or less throughout the course:


Based on the lecture and exercise contents, the materials and discussions regarding the swine flu/narcolepsy case on the course, as well as all your own expertise and opinions, work out a plan how a DA study could/should be made. The plan should address at least the following:
* Theory lectures and exercises (classroom)
* Case discussions and group/independent work (classroom and outside)


# Background description
In order to allow us to concentrate on certain things below is suggestion for distribution of (primary) responsibility on certain issues along the course
#* purpose of the study
#* main question(s) related to the case
#* relevant actors related to the case
#* roles of different actors related to the case
#* sources of information
#* timeline of major events
# Decision analysis study plan
#* decisions considered
#* outcomes of interest that the decisions (are intended to) have influence on
#* the factors ([[variable]]s) that link decisions to their (intended) outcomes
#* all the above at certain different timepoints along the progress of the case
#** which timepoints?
#** what knowledge emerged between different timepoints?
#** how does the model change from a timepoint to another?
#* analyses over the model and its parts


The DA study plans are intended to be worked on gradually alongside the lectures and exercises, and progress will be presented to and discussed with other students as well as lecturers in classroom a few times during the course. In the end of exercice part 1, the group will present the final plan to other students as well as lecturers in classroom.  If possible, the final DA study plans will be, at least partly, executed by means of the demonstrator model that will be developed for demonstration purposes on the course. At least the possible results that could be achievable according to the plan will be discussed at the presentation of the plan. Results (actual or anticipated) are added to the plan.
'''Intro


''(part 1 can be considered as corresponding roughly to the introduction and methods sections of a scientific article, or to the scope and definition attributes of an [[assessment]] object in [[open assessment]])
Jouni and Mikko to agree on who leads which parts, and prepare corresponding teaching materials. Basically exercise introduction and instruction prepared by Mikko, general course arrangement basically a responsibility of Jouni. Marko to participate and help as available/needed.


'''Part two: Discussion and conclusions regarding risk management options and actions
'''DA


''Planned as individual work, but can also be combined as a part of the group work if so desired
Marko and Jouni to agree on who leads which parts, and prepare corresponding teaching materials. Also developing the model for demonstrations a responsibility of Jouni and Marko (Can the study plans also be executed with the demonstrator model (updated according to needs of the plans)?). Jouni to create and keep up the linkage between case study/exercise and theory. Mikko will be back to participate in the exercise part 1 presentations and discussions. Also Mikko is available if assessment performance is wanted to be taken up in DM under uncertainty.


Following the work done in exercise part 1, and taking account of the discussions regarding the plans by different groups in, consider the use of the DA results.
'''RM


# What does the analysis tell?
Mikko to plan the story-/timelines for both theory and case for the last section. More detailed distribution of lead responsibilities to be agreed upon by Jouni and Mikko. Follow-up and assistance for post-seminar improvement work on exercises needs to arranged. Also exercise output checks and student evaluations need to be agreed upon.
#* were the right decisions made?
#* what decisions should have been made?
#* could things have gone in a different way?
#* what implications other courses of events would have had?
#* what would it have required?
#* is it even possible that such could have happened in reality?
# What can be concluded?
#* if anything, what went wrong? why?
#* if a somewhat similar situation occurred, what should be done?
#* if possible, what should be done in preparation?


'''Basis for evaluation
'''Other


The main point is not to write long and detailed texts of any specific topic within this course. Instead the idea is to try to make use of what has been taught on the course by combining them in relation to a practical question. Most important issues in evaluating the exercises are:
In addition there are certain general issues that also need to be agreed upon and taken care of, e.g:
* collection and preparation of study and background materials for both theory and case to be done by whom?
* '''classroom reservations to be done by whom? (us or UEF?)
** '''needs for specific equipment or space needs must be clearly indicated in the course schedule
* video recording / streaming of lectures? (which lectures are suitable?)
* is there a need for an increased continuous Opasnet up-keeping or help desk -service during the course?
* requirements/needs for keeping up participation lists? how is this to be done?


* general clarity of thought
== theory vs. story vs. case study exercise ==
* comprehension and description of the big picture
* meaningful interrelations between the aspects of the case
* application of the knowledge and methods provided in lectures, exercises and discussions along the course
* ability to argue for or against different statements or actions


''For non-ToxEn participants it could be considered that the lectures and exercises, the DA study plan, and the Discussion and conclusions regarding RM options are each worth 2 study credits. (or should it be 3, 2, and 1 respectively?)
An idea for creating and keeping up a tight linkage between the swine-flu story and the theory lectures as well as case study exercises: Let us make the detailed theory lecture/exercise plans, so that in every possible theory topic there is a connection (at least a weak one) to the swine-flu story. It can be done e.g. by:


=== Suunnittelussa huomioonotettavaa ===
* taking lecture/exercise examples from the case to demonstrate the theory
* after presenting the theory, making students work out what the particular part of theory would mean in the case context
* making students explore and discuss a certain aspect(s) of the case as an introduction to the particular part of theory, presented afterwards.


* aikataulurajoitteet
The theory and the exercise are now implemented in the schedule to progress in unison. If a good linkage between the case story and theory lectures/exercises is achieved, it also guarantees contentual support for progressing the case study exercises. It is not necessary to be strictly bound to the chronological order of events in the story (the story parts will probably naturally fall into a nearly chronological order if in line with the theory and cases study exercise plans).
* tila-, väline- yms. vaatmukset
* tilavaraukset
* videointi, koneet, verkko?
* DA-mallin kehikko
* infomateriaali omatoimiseen työhön
** case
** käsitteistö ja teoria
* perusasialuennot ? työnjako, suunnitelmat, tausta- ja esitysmateriaalit
* harjoitustehtävät ? peruskäsitteistö yms. / DA-case / kyselyt kurssin kuluessa

Latest revision as of 10:12, 6 April 2011

Comments about DARM course (18.3.2011):

  • Are these methods used in the actual decision situation/in real world?
  • Some of issues too complicated (especially Bayes lectures);
  • Links of this course to other issues taught in environmental risk assessment;
  • More concrete examples would be useful;

Some things to take into account in planning/arranging/conducting the course

  1. Harkkatyön ohjeistukset: Category:DARM exercise. Ainakin RM harkan (osa 2) ohjeistuksessa lienee täsmentämistä. Muutenkin saa parannella niin paljon kuin osaa.
  2. Kunhan tiedostojen jakelun systeemi saadaan taas tavalla tai toisella pelaamaan, N:\YMAL\Presentations\2011\0228_DARM\materials sisältää jakeluun sopivaa tavaraa. Ainakin NRC:n kirjojen executive summaryt minulla oli alunperin tarkoituksena panna heande-sivulle (m-filesiin).
    • Understanding risk –kirjasta en saanut executive summarya pdf:na ladattua nap.edu:sta. Voisi vaikka skannata, jos ei muuten onnistu.
  3. Luento-/harkka-aikataulu on vielä aika epämääräinen ja sekava, etenkin loppupäästään (RM-osio). Jos kokonaiskuva alkaa kirkastumaan (esim. tämänpäiväisten kokemusten pohjalta), saa täsmentää kurssisivun kalenteriin vaikkapa sen kummemmin minulta kyselemättä.
  4. Sikaflunssatarinaakin voisi vähän miettiä. Esim.:
    • Taustamateriaalilinkkejä Decision analysis and risk management#General information sources on AH1N1 influenza (Swine_flu) voisi tarkistella. Kirsti on niitä keräillyt ja varmasti ovat tolkullisia, mutta olisihan se hyvä jonkun muunkin tietää mitä kaikkea sieltä löytyy. Olisiko Teemu sopiva henkilö tähän?
    • Aikataulun ja linkkilistan aihe-/vaihejaottelu eivät ole aivan yhdenmukaiset. Sikaflunssastoorituokioiden rajauksia ja niiden sijoittelua aikatauluun voisi varmaankin parantaa/tarkentaa nykyisestä. Jouni & Marko, jos ehditte?
    • Edellisen yhteydessä voisi myös miettiä, jos sikaflunssalinkkejä nostelisi sivun lopusta aikataulun yhteyteen sopiviin kohtiin.
  5. Kunhan alkaa selviämään kuinka monta ryhmää ja yksilöä tulee harkat tekemään, voidaan miettiä tarvitaanko kahta päivää loppuseminaariin. Nyt seminaarivaraus on aikataulussa sekä 11.4. että 12.4.
  6. En vielä kovin hyvin tiedä miten RM osio tulee kurssin loppupuolella järjestymään, mutta olisiko ideaa varata 1-2 päivää lopusta koko kurssin kokonaiskuvan kertaamiseen ja esille nousevien/nousseiden erityiskysymysten lisätarkasteluun? Esim. DA summary & specific topics 7.4. ja RM summary & specific topics 8.4. Tai jotain sinne päin.
    • ----#: . Ehkä yksikin sessio riittäisi tähän. 8.4. tai 11.4.? --Mikko Pohjola 13:16, 4 March 2011 (EET) (type: truth; paradigms: science: comment)

--Mikko Pohjola 13:08, 4 March 2011 (EET)

Draft synopsis

For a draft synopsis of the course, see a previous version.

Case study exercise

Case study exercise description and instructions can be found at: Category:DARM exercise.

Lectures related to uncertainty

I was briefly discussing with Jouni from the uncertainty, Monte-Carlo and Value of Information path of the lectures.

I think this “path” has three main pieces:

  • What is uncertainty, how to identify uncertainty etc. This could include all kind of issues like expert elicitation, data analysis, literature search, expert estimates, modelers estimate etc.
  • How to implement uncertainty to the calculations? Thus, how to actually combine uncertainties from different sources and/or how to create uncertainty functions. This could be divided to two different parts, Bayes and Monte-Carlo, since these two are the two most common ways of implementing uncertainty to the assessment.
  • Third phase consist sensitivity analysis and value of information. Thus, these are methods that are applied after the model is done and after the uncertainties have been implemented.

I think the basic idea of this is already included in the course structure, but we could try to make it clearer in the lecturing structure?


Responsibilities among organizers

The course basically consists of three temporal/contentual sections:

  1. Introduction 28.2.-4.3.
  2. DA 8.3.- 29.3.
  3. RM 30.3.-12.4.

The two main teaching/studying methods are employed in parallel more or less throughout the course:

  • Theory lectures and exercises (classroom)
  • Case discussions and group/independent work (classroom and outside)

In order to allow us to concentrate on certain things below is suggestion for distribution of (primary) responsibility on certain issues along the course

Intro

Jouni and Mikko to agree on who leads which parts, and prepare corresponding teaching materials. Basically exercise introduction and instruction prepared by Mikko, general course arrangement basically a responsibility of Jouni. Marko to participate and help as available/needed.

DA

Marko and Jouni to agree on who leads which parts, and prepare corresponding teaching materials. Also developing the model for demonstrations a responsibility of Jouni and Marko (Can the study plans also be executed with the demonstrator model (updated according to needs of the plans)?). Jouni to create and keep up the linkage between case study/exercise and theory. Mikko will be back to participate in the exercise part 1 presentations and discussions. Also Mikko is available if assessment performance is wanted to be taken up in DM under uncertainty.

RM

Mikko to plan the story-/timelines for both theory and case for the last section. More detailed distribution of lead responsibilities to be agreed upon by Jouni and Mikko. Follow-up and assistance for post-seminar improvement work on exercises needs to arranged. Also exercise output checks and student evaluations need to be agreed upon.

Other

In addition there are certain general issues that also need to be agreed upon and taken care of, e.g:

  • collection and preparation of study and background materials for both theory and case to be done by whom?
  • classroom reservations to be done by whom? (us or UEF?)
    • needs for specific equipment or space needs must be clearly indicated in the course schedule
  • video recording / streaming of lectures? (which lectures are suitable?)
  • is there a need for an increased continuous Opasnet up-keeping or help desk -service during the course?
  • requirements/needs for keeping up participation lists? how is this to be done?

theory vs. story vs. case study exercise

An idea for creating and keeping up a tight linkage between the swine-flu story and the theory lectures as well as case study exercises: Let us make the detailed theory lecture/exercise plans, so that in every possible theory topic there is a connection (at least a weak one) to the swine-flu story. It can be done e.g. by:

  • taking lecture/exercise examples from the case to demonstrate the theory
  • after presenting the theory, making students work out what the particular part of theory would mean in the case context
  • making students explore and discuss a certain aspect(s) of the case as an introduction to the particular part of theory, presented afterwards.

The theory and the exercise are now implemented in the schedule to progress in unison. If a good linkage between the case story and theory lectures/exercises is achieved, it also guarantees contentual support for progressing the case study exercises. It is not necessary to be strictly bound to the chronological order of events in the story (the story parts will probably naturally fall into a nearly chronological order if in line with the theory and cases study exercise plans).

Retrieved from "https://en.opasnet.org/index.php?title=Talk:Decision_analysis_and_risk_management&oldid=19987"