Category:DARM exercise: Difference between revisions

This category relates to Decision analysis and risk management course 2011, and it is not used for newer courses. For more recent material, see the newest Decision analysis and risk management course.

Instructions for the case study exercise

The global AH1N1 (swine flu) pandemic, vaccinations to fight the pandemic, and the side-effects of the vaccines have been topics of much debate during the last couple of years in many parts of the world. In Finland the hottest debates have recently related to the suspected connection between the Pandemrix vaccine and the unexpected increase in the prevalence of narcolepsy among young people. The swine flu case provides an example of a complex, multifaceted risk management problem in which there are multiple interrelated decisions to be made by multiple different decision makers, uncertainty about the outcomes of the decisions, and many possibilities for value conflicts.

Various different points of views have been taken both to criticize as well as defend the decisions and actions that have been taken to manage different risks in the swine flu pandemic and the vaccination campaign. Of course history could have taken other courses if different decisions and actions had taken place. With hindsight, i.e. when we already know what actually happened, it is of course easier to judge past decisions. Many of the things we know now, were not known in the situations when the decisions e.g. on launching a general vaccination campaign in Finland or choosing the vaccine to be used were made.

The discourse on what could or should have been done and why goes on and opinions fly about, but not much systematic analysis exists yet. Possible questions to be address in such a systematic analysis are e.g.:

• Based on the knowledge that existed at the time of the decisions, can it be considered that the right decisions made?
• If not, can we know which decisions should have been made instead?
• Would the historical counterfactuals that we can now picture have been realistic and feasible alternatives?

Imagine that the Ministry of Social and Health affairs of Finland is developing new approaches and increased capacity to managing public health risks, and wants to learn from the swine flu case. That is why they are asking these questions from you, an (future?) expert in protecting and promoting public health. In order to work out a response to the Ministry, you need to familiarize yourself with the swine flu case and analyze the decisions and actions taken in the case. Also you must consider and reason where, how, and why different decisions and actions could or should have been taken. Eventually the work needs to described and explained in a report to the Ministry. This is the case study exercise.

The case study exercise is done in two parts:

1. Plan of a decision analysis (DA) study to shed light on the questions asked by the Ministry (group work)
2. Report on risk management (RM) actions and options, which considers, points out, and argues how things could or should have been managed otherwise (individual work)

It is probably impossible to provide a thoroughly conclusive answer to any of the above mentioned questions, but a lot can still be learned by trying to answer them with a systematic analysis. Feel free to focus on those aspects of the swine flu case that are of most interest to you and your group members. There are no single right solutions, and it is only good if different groups/individuals come out with quite different kinds of plans and considerations.

In order to successfully accomplish the exercise, consider making use of e.g.:

• the theory lectures and classroom exercises on decision analysis (DA) and risk management (RM) on this course
• classroom discussions on the swine flu case as a DA and RM problem
• related materials listed and linked to on the course web-page
• the demonstrator DA model
• other assessments in Opasnet
• descriptions of assessment and variable objects
• all other related information e.g. on the web and libraries
• your own expertise and opinions
• other groups'/individuals' exercise works

The two parts of the exercise are explained in more detail below. Both parts of the exercise will be made in the Opasnet web-workspace. Using Opasnet allows e.g. convenient follow-up and commenting of exercise progress, commenting, discussing, and learning from other groups/individuals works, as well as dissemination of results to a broader audience. The basics of using Opasnet will be taught in the lecture on 4.3. 9-12 in computer classroom MC9 (also see below for more information on that).

(NOTE: It is smart to check the times and locations mentioned above and below from the schedule on the course web-page in case of possible changes.)

If in need of assistance, check the help/guidance contacts on the course web-page.

Part 1: Decision analysis study plan

This part is intended as group work (~3-4 people/group). It is recommended that there would be at least one person with fluency in Finnish in every group as some materials related to the swine flu case are available only in Finnish. Otherwise there are no constraints to group formation.

Work out and write down a plan for a decision analysis study to assess the goodness of actual or counterfactual decisions in the swine flu case. Consider your role as making and reporting the plan so that a skilled assessor can build a model, run the model, and produce meaningful results according to it. You may focus on certain specific aspect(s) case, but also remember to relate your analysis to the complex big picture of the overall swine flu case as well. Each DA study plan is written (containing text, tables, diagrams, images, etc.) on its own page in Opasnet. The plans can be structured e.g. according to the following example, which can be applied within the Opasnet assessment structure:

1. Brief background description
   • overview of the swine flu case
   • different decisions and actions in the swine flu case
2. Scope (vaccination decisions) (→ Scope)
   • purpose of the DA study
   • question(s) addressed in the study
   • the relation of the study to the overall swine flu case
   • relevant actors related to the study
   • roles of different actors related to the study
   • spatial and temporal boundaries the study
   • expected and possible impacts of the study
   • intended (even if imaginary) use of the study
3. Decision analysis study plan (→ Definition)
   • decisions and decision options considered in the study
   • outcomes of interest that the decisions (are considered to) have influence on
   • the relationships between the decisions and outcomes of interest (e.g. as a network of variables)
   • different sources of information needed/used in the study
   • means, methods, and tools (e.f. software) needed for the study
   • description of a (executable) calculative DA model
   • description of the execution of the model
   • analyses on the model, its parts, and its results (e.g. uncertainty, sensitivity, VOI, applicability, ...)
4. (Expected) results (→ Result)

The groups are recommended to start working on the plan gradually alongside the lectures and classroom exercises already from the very beginning of the course. The task may appear difficult to grasp in the beginning, but it should become clearer and more comprehensible as the course progresses. Try to express your plan in a quantitative and calculative form. It is recommended that you test the executability of the plan by trying to calculate it yourself e.g. by means of Excel, Open Office spredsheet, or pen, paper, and calculator.

Important milestones for the DA study plan:

Guidelines for presenting the DA study plans will be provided later on this page.

Part 2: Risk management exercise

This part is planned as individual work, but it builds on the group works done in the part 1 of the exercise. Collaboration between students is not only allowed, but recommended.

Take the perspective of the Ministry of Social and Health affairs. Consider yourself managing a project of developing capacity to manage major public health risks. In your project you want to take account of the lessons that could be learned from the swine flu case. In this exercise your task is to:

1. Evaluate all four DA study plans from the use/r point of view:
   • Of what value would each of the planned analysis be for you?
   • Make use of the properties of good assessment framework, particularly:
     • Relevance: Is content of the plan/analysis relevant in relation to the stated purpose of the analysis?
     • Pertinence: Is the purpose of the analysis relevant in relation your needs?
     • Usability: Can you grasp the idea of the plan/analysis? Does it increase your understanding of the swine flu case?
     • Acceptability: Would results/conclusions be acceptable to you? Why or why not?
2. Give an overall statement: How could/should the results of these analyses be taken into account in your project?
3. Choose (one) another perspective and repeat the evaluation of the DA study plans from that perspective
   • E.g. common citizen, medical superintendent in a health care center, health researcher, journalist, nurse in public health care, principal of an elementary school, …
   • Focus on the differences in comparison to the above evaluation
4. Write an (freely formatted) evaluation report on your own RM analysis page (see the list of links at the bottom of the page)
   • If you do not yet have a page, create. Advice, if needed, may be asked e.g. from fellow students or the lecturers
   • Aim for a clear and concise report.
   • Active commenting of of other groups individuals works can earn you pluses that will be considered in the overall grading of the course
5. Present your main findings in the final seminar 11.-12.4.
   • Improvements on the report page can be made up to the final evaluation in the end of April

Presentation guidelines for the final seminar

DA study plans

• Focus on the major changes since the presentation of full drafts
• The DA study plan can be used for illustrating the presentation
  • if so desired, the highlights of the plan/study can be summarized briefly to the beginning of the page
• Presentation time is ~5 minutes + ~10 minutes for discussion

RM exercise

• List the main points from your report in the beginning of the page e.g. as bullet points
  • in a similar manner as you could summarize the main points on slides
• Use the bullet point list as illustration to back up your presentation
• The issues considered in a presentation may include e.g.:
  • general comparison of different DA study plans from the Ministry point of view
  • a more detailed look into the evaluation of one particular DA study plan from the Ministry point of view
  • general conclusions on the potential effectiveness (see heande:Properties of good assessments) of the planned DA studies from the Ministry point of view
  • Round-up of the main differences between the evaluations of the DA study plans from the Ministry point of view and the other chosen point of view
    • you may also pick up some interesting details to illustrate the differences arising from the different points of view for evaluation
• Presentation time is ~5 minutes + ~10 minutes for discussion

See the suggested order of presentations in the course contents table. If someone has a pressing need to adjust the order, please propose a change early ahead of the beginning of the seminar.

Introduction to working in Opasnet

Both parts of the exercise will be made in Opasnet, i.e. this web-workspace. The necessary skills to get started will be taught in the lecture on 4.3. 9-12 in computer classroom MC9. More can be learned along the course, and some additional exercises on discussion and argumentation will be done in the lecture on 1.4. 9-12 in computer classroom MC9.

The content of the introductory lecture/exercise is:

1. Introduction to Opasnet
   • purpose and history of Opasnet
   • technical foundation: Mediawiki, same as in Wikipedia
   • openness, transparency, collective knowledge creation/learning
2. Guided tour to structure, content and functionalities of Opasnet
   • browsing, searching and reading Opasnet content
   • structure of Opasnet
     • English (open), Finnish (open), password protected Heande (project use)
   • content: text, styling, tables, images, documents
   • page history (no need to fear making mistakes!)
   • creating a user account / logging in
   • creating a new page
   • page (object) types? → templates
   • categorization
   • ways of contributing to Opasnet
   • commenting (no login required)
   • discussion on talk page (login required)
   • structured argumentation on talk page (login required)
   • editing of the article page (login required)
   • uploads (login required)
   • help & examples
   • edit conflict
     • what is it?
     • how to avoid?
     • what to do if happens?
   • wiki-markup vs. wikiwyg
   • different browsers
   • special pages
3. Practical exercises
   • create a user account for yourself
     • think of a good, descriptive user name for yourself
     • add user information
     • adjust your "my preferences"
   • create a DA study plan page with your group
     • choose a page type?
     • give a good descriptive name for your page
     • categorize your page to DARM exercise category
     • page structure
     • content?
   • create an individual RM analysis page for yourself
     • page type?
     • name
     • categorization
     • page structure
     • content?
   • Browsing, searching, commenting, and discussing information in Opasnet
     • find an assessment on "Farmed salmon" in Opasnet and see what it contains and how it is structured
     • find out how many "variables" there are in Opasnet
     • find out what kind of objects "methods" in Opasnet are, find at least one (representative) example
     • find an interesting page in Opasnet and comment on it
       • with the comment box
       • by editing the discussion page
4. Problems? Questions? Comments?

In the lecture/exercise on 1.4., the focus is on specifics of discussion and structured argumentation in wiki-mediated broad collaboration. The course participants are encouraged to actively discuss their own and others exercise works in Opasnet.

Evaluation principles for the case study exercises

The case study exercises are considered to be worth 3 ECTS in total, of which the decision analysis study plan is 2 ECTS, and the risk management analysis is 1 ECTS. The corresponding maximum scores are 20 points for the DA study plan and 10 points for the RM exercise. Activity in commenting and discussing the exercises in Opasnet will be considered as a positive factor in deciding upon the overall course grades (e.g. rounding up).

The main point is not to write long and detailed reports. Instead, the idea is to try to make good use of the theoretical and practical issues taught on the course in the context of a real-world risk management and decision analysis problem. When evaluating exercises, following general qualities will be expected:

• clear and focused scoping
• clear connections to the "big picture"
• coherent connections between different aspects of the the issues considered
• general clarity of thought and its expression
• innovative application of (at least some of) the knowledge and methods provided in lectures, exercises and discussions along the course
• well-founded reasoning (e.g. arguing for or against) for the statements made
• hard work and sincere efforts
• collaboration within and between groups and between individuals

As the course is being taught for the first time, and there is no strong prior experiences on having students make such exercises, the evaluation criteria are quite flexible. As clear grading principles can not yet be provided, the grading will be made quite much by relating the quality of the exercise reports to each other, rather to any predefined golden standard.

References

Related files

<mfanonymousfilelist></mfanonymousfilelist>