RM analysis Jukka-Pekka Männikkö

This page is a encyclopedia article. The page identifier is Op_en4976
Moderator:Jpmannikko (see all)
Upload data {{#opasnet_base_link:Op_en4976}}

Take the perspective of the Ministry of Social and Health affairs. Consider yourself managing a project of developing capacity to manage major public health risks. In your project you want to take account of the lessons that could be learned from the swine flu case. In this exercise your task is to:

1. Evaluate all four DA study plans from the use/r point of view:
   • Of what value would each of the planned analysis be for you?
   • Make use of the properties of good assessment framework, particularly:
     • Relevance: Is content of the plan/analysis relevant in relation to the stated purpose of the analysis?
     • Pertinence: Is the purpose of the analysis relevant in relation your needs?
     • Usability: Can you grasp the idea of the plan/analysis? Does it increase your understanding of the swine flu case?
     • Acceptability: Would results/conclusions be acceptable to you? Why or why not?
2. Give an overall statement: How could/should the results of these analyses be taken into account in your project?
3. Choose (one) another perspective and repeat the evaluation of the DA study plans from that perspective
   • E.g. common citizen, medical superintendent in a health care center, health researcher, journalist, nurse in public health care, principal of an elementary school, …
   • Focus on the differences in comparison to the above evaluation
4. Write an (freely formatted) evaluation report on your own RM analysis page (see the list of links at the bottom of the page)
   • If you do not yet have a page, create. Advice, if needed, may be asked e.g. from fellow students or the lecturers
   • Aim for a clear and concise report.
   • Active commenting of of other groups individuals works can earn you pluses that will be considered in the overall grading of the course
5. Present your main findings in the final seminar 11.-12.4.
   • Improvements on the report page can be made up to the final evaluation in the end of April

EXTRA: also include consideration/evaluation of the example swine flu/narcolepsy model (discussed in 8.4. lecture) in your report/presentation.

Decision analysis exercise Group 1

Point of view: Ministry of Social Affairs and Health

Relevance

Group 1 states evaluating the impact of H1N1 vaccination in Finland as the purpose of their decision analysis. The questions to be addressed are:

1. Was the decision to vaccinate everybody the right one?

and

2. Would it have been better if vaccinations were not done?

Against this background the groups analysis seems relevant. Group 1 evaluates the possible effects and outcomes of the decision to vaccinate the whole population, risk groups only or the possible decision not to vaccinate anyone. Group 1 suggests use of DALY:s to evaluate the impacts of the vaccination decisions.

Pertinence

Decision analysis of group 1 is relevant for the Ministry of Social Affairs and Health. Evaluation of side effects against benefits of the vaccinations is important. It should be noted, however, that no such data would have been available at the time of the actual vaccination decisions.

Usability

Group 1 decision analysis contains much useful information. The calculation of DALY:s requires more work to provide actual quantitative information.

Acceptability

Conclusion of group 1 DA is that the benefits of vaccinations exceed the harmful side-effects caused by the vaccination. The conclusion is drawn based on relevant information about narcolepsy and about the number of cases in Finland. This is acceptable for the Ministry of Social Affairs and Health.

Overall statement

Group 1 DA contains much useful information and seems to be objective and based on facts. The quantitative information provided by the analysis requires more work to provide information which could be used in risk calculations.

Point of view: Common citizen

Relevance

Content of the analysis seems relevant in comparison to questions the DA is seeking to address.

Pertinence

The analysis is somewhat interesting. It may increase average citizen's trust towards health officials to know that expert analysis shows that the vaccination decision was the right one.

Usability

The analysis is not fully understandable to average citizen. Use of DALY:s to calculate effects of different outcomes may need further explanations. The use of the analysis in practice may be that it will affect the individual vaccination decisions in future.

Acceptability

The analysis is acceptable for average citizen because it seems to be based on facts and seems to be objective.

Overall statement

Group 1 DA is perhaps not very interesting to average citizen, but the conclusion is clear. The vaccination decision seems to be the right one, because the result caused least harm even with narcolepsy side effects. Understanding the reasoning behind the result may require clarification.

Decision analysis exercise Group 2

Point of view: Ministry of Social Affairs and Health

Relevance

Group 2 states evaluation of impacts of H1N1 vaccination in Finland as the purpose of their DA. Outputs which they will consider are number of new cases of swine flu before and after the vaccination and side effects likely to arise from vaccination. Boundaries are set to priority and target groups. The actual analysis however considers the vaccination of whole population and risk groups. No estimations about number of cases before and after the vaccination is provided. It seems that purpose and boundaries of this DA are not up to date. (10.04.2011)

Pertinence

Group 2 DA contains much useful information for the Ministry of Social Affairs and Health. Decision analysis contains a broad causal diagram which lists many of the affecting factors related to outcomes of vaccinations. Information about calculation of DALY:s is provided.

Usability

Actual quantitative information and results are missing, and thus this DA is not fully usable for the Ministry of Social Affairs and Health. Still, the analysis contains valuable information for the Ministry.

Acceptability

Results of the DA are not provided, although the content so far seems relevant. Group 2 states in their DA background chapter, that their aim is to "support the decision to vaccinate by weighing the rationale at various levels of the decision chain". This raises a question about the objectivity of this kind of analysis which is aiming to a certain result from the beginning.

Overall statement

Group 2 DA provides much useful information but remains incomplete. Purpose and boundaries of the study plan should be updated.

Point of view: Common citizen

Relevance

Group 2 DA seems relevant but boundaries and content of the analysis are a bit confusing. Boundaries of the DA define target group as:

"health care professionals, pregnant women and persons aged from 6 months to 64 years who may belong to a risk group due to some other illness"

Despite this definition the DA also evaluates vaccinations of the whole population.

Pertinence

The analysis is relevant for a common citizen because it discusses the risks involved in swine flu vaccinations for the whole population. It is possibly not very interesting for a common citizen due to its complexity except for the main conclusions which are that harmful effects (DALY) are expected to be lower if only risk groups are vaccinated.

Usability

Group 2 DA is not very usable for a common citizen. It is hard to grasp the reasoning behind the conclusion that harmful effects would be lower if only risk groups are vaccinated. The result could be applied to individual decisions like wether to take a swine flu vaccination or not.

Acceptability

The results are acceptable because they are based on an expert opinion and common citizen does not have the required knowledge to question those opinions. Results would probably not have a very strong effect because the reasoning behind the conclusion is so hard to grasp.

Overall statement

Group 2 DA contains much information for a common citizen but it also has many complicated parts which would not be possible to understand without further clarifications. The result of the DA is that harmful effects of mass coverage vaccinations exceed the harmful effects of vaccinating only the risk groups. It is not clear on which grounds this argument is based. No calculations about the DALY:s in each case are presented. Common citizen may or may not believe the result and it may affect their individual decisions about taking vaccinations.

Decision analysis exercise Group 3

Point of view: Ministry of Social Affairs and Health

Relevance

Purpose of Group 3 DA is to answer the following question:

Can the use of thermal scanners combined with PCR tests prevent the spreading of swine flu to Finland if all passengers arriving from abroad will be scanned at the border control points?

The DA contains relevant information about scanners and the factors affecting the spread of swine flu.

Pertinence

Group 3 DA might be relevant to the Ministry of Social Affairs and Health, if the threat caused by swine flu to population of Finland would seem very high. The content of the analysis is an interesting approach to swine flu problem. However, there are many factors which perhaps make it less interesting for the Ministry as an actual measure to be considered. Use of thermal scanners at all border points is a costly and complicated procedure and from a subjective point of view it does not seem likely that it would be effective. Swine flu can spread in the airplane and the newly infected people would not have fever yet. There are also considerations about limiting individual freedoms which must be taken in account.

Usability

Group 3 DA contains relevant information about swine flu and the analysis overall is impressive. Calculations are very interesting but it seems likely that they have too many uncertainties to be usable by the Ministry of Social Affairs and Health.

Acceptability

The quality of information in the group 3 DA seems acceptable for the Ministry of Social Affairs and Health. It is, however, very unlikely, that the use of thermal scanners and quarantine of travelers with fever would be acceptable for the Ministry or the people subjected to these measures. It does not seem to be in proportion to the threat caused by swine flu.

Overall statement

DA study of group 3 has useful information for the Ministry and the quality of work is impressive. The calculations require so many uncertain estimates that it does not seem likely that they would be accurate, despite the high quality of the analysis. One factor which has not been considered in the analysis are the limitations to individual freedoms caused by such drastic quarantine measures. It does not seem to be in proportion to the threat caused by swine flu in a country of high quality medical care such as Finland. Effectiveness of thermal scanners and quarantine to control the spread of swine flu is questionable. It is also not likely that passengers would accept being quarantined for days because of a fever. Cost and feasibility of thermal scanning and quarantine procedures is not estimated in the analysis and it would be an important factor for the Ministry.

Point of view: Common citizen

Relevance

The content of Group 3 DA seems to be relevant in comparison to stated purpose of the analysis.

Pertinence

From the point of view of a common citizen, using thermal scanners to detect swine flu patients might not be very relevant. To someone in Finland who is very afraid of the epidemic it might be relevant that all measures to stop the epidemics are being used.

Decision analysis exercise Group 4

Point of view: Ministry of Social Affairs and Health

Relevance

Group 4 states that purpose of their DA is to answer the following question:

Would the consequences of the swine flu epidemic be less harmful if the vaccination of the population would be postponed and reconsidereded after proper testing and extensive hygiene campaign, than if the whole population would be vaccinated immediately?

The analysis is relevant in relation to this purpose set by the group. The DA contains much information about effects of decision variables and their relations. Still, group 3 DA lacks some critical points of view like the typical epidemic "peak" of new infections which is important to take in account when considering postponing the vaccinations.

Pertinence

Group 3 DA has much information which is relevant to the Ministry of Social Affairs and Health. There are some points of view missing, which would be relevant, like the seasonal variation of epidemics.

Usability

Although there is much usable information about DALY:s and possible calculations, this DA lacks the actual calculations. Hygiene campaign is not determined accurately, and no actual actions are suggested. Quantitative information which would be important for actual decisions is not provided.

Acceptability

Issues considered in group 3 DA are relevant and there is much information about DALY:s. The group suggests as a result that postponing the vaccination decision would result in a better outcome than if the whole population would be vaccinated immediately. Group 3 states they "believe" that the number of DALY:s would be greater in case of immediate vaccinations. No calculations to back this belief up are provided. It seems that the group has made its conclusions without any actual calculations and thus the result is not acceptable as such.

Overall statement

The evaluation of the effects of postponing the vaccination decision cannot be made without considering the typical peaking of the epidemics after which they fade away, making further vaccinations useless for the epidemic at hand. The group has not considered this point at all which would be critical for the actual decisions about vaccinations. Suggested result is not backed up by any calculations. Despite these faults the analysis contains much valuable and relevant information about swine flu and DALY:s.