Common currency in health assessments

This page is a knowledge crystal of subtype method. The page identifier is Op_en4692
Moderator:Olli (see all)
Upload data {{#opasnet_base_link:Op_en4692}}

Scope

This page reviews identifying a common metric to be used in benefit-risk assessments. Several European research projects have already been dealing with this issue, therefore their conclusion are presented first. In the end we select the common currency most suitable for PlantLibra project.

We may qualitatively compare things very different in nature, apples and oranges. In cases with very large and clear differences between the choices, qualitative comparisons might be enough. However, the question easily becomes value driven. Common measure is an answer to this problem.

Most established options for common currency

There are several common currency measures available. Here we briefly present six of them.

Incidences

Incidence information as a health measure is the absolute number of cases during a give time. Commonly, it is expressed as a proportion or a rate, where number of new cases is divided by the population initially at risk. It can also be used for presenting person-time incidence rate, where the denominator is the sum of the person-time of the at risk population. Incidence should not be confused with prevalence which is a measure of the total number of cases of disease in a population rather than the rate of occurrence of new cases.

Disability adjusted life years (DALY)

DALY is a measure of overall burden of disease. It contains both mortality and morbidity, which makes it useful for comparing the health effects of different nature, and it has become increasingly common in the field of public health. Methodologically, it summarizes years of life lost (YLL) and years lived with disease. The less DALY, the better. One DALY is equal to one year of healthy life lost. For diseases, there are severity weights which have been derived in combination of expert panels and patient reports. Severity weight of death is assigned the value of 1.0, and perfect health 0.

Quality Adjusted Life Years (QALY)

QALY is very much like DALY. The greatest difference is that it measures the positive health state, apart from DALY. Similarly, it includes both the quality and the quantity of life lived. The more QALYs, the better. Using QALY requires utility independent, risk neutral, and constant proportional tradeoff behaviour. Each year in perfect health is assigned the value of 1.0, and value of 0.0 for death. Similarly to DALY, there are weights for health states. Determining the weight associated with a particular health state can be done using standard descriptive systems which categorises health states according to the following dimensions: mobility, self-care, usual activities (e.g. work, study, homework or leisure activities), pain/discomfort and anxiety/depression.

Days of work lost

This measure tries to capture the disease burden of working age population to be further analyzed in more sophisticated measures, such as cost.

Cost in money

EFSA 2006

In 2006 EFSA suggested that the assessments with both risk and benefit ideally should be performed under the same criteria for weighing the evidence and identifying the uncertainties. The presentation of the results of the risk-benefit assessment must fit the predefined purpose of the request and make clear where the certainties and uncertainties are in order to compare the relative confidence on the benefits with the risks. The following possible common scale measures were mentioned:

• Incidences
• Disability Adjusted Life Years DALY
• Quality Adjusted Life Years QALY
• Days of work lost
• Costs in money

Further EFSA 2006 concluded that QALY are, nevertheless, based on a number of assumptions, and are more difficult to quantify than DALY. The difficulty in expressing results from toxicological studies in experimental animals as DALY needs to be overcome. Using costs requires equal cost structures across countries/world and is difficult to communicate. It was agreed that more research and experience with different approaches are needed.

Advantages using DALY

• The advantages of using DALYs are that they represent an established procedure to compare risks of different nature
• DALYs may provide guidance to the risk manager on how to prioritize the direction of targeted intervention measures
• DALYs have a time-scale

Challenges using DALY

• DALYs are applied at the societal, rather than the individual, level. It is possible to apply DALYs in risk-benefit assessment, but appropriate data may seldom be available.
• The difficulties in using DALYs are that clear messages are needed so that the numbers generated are not taken out of context.
• It also seems difficult to include preventive aspects (such as effects of preservation) or absence of risk rather than benefit.
• The difficulty in expressing results from toxicological studies in experimental animals as DALYs needs to be overcome.
• Because the scale is likely to differ for different analysis, no generally applicable measurement scale is likely to be developed.

EFSA 2010

Scientific committee, appointed by the EFSA 2010, recommended a stepwise approach for assessing the benefits and risks of food. The last of the three steps utilizes common currency approach to combine two or more of the following elements: increases or decreases in morbidity, mortality, disease burden, and quality of life. The choice of composite metrics should be made on a case by case basis, based on the specific risk-benefit question, identified hazards and positive health effects. Health effects can be assessed in a number of different dimensions, such as incidence of effect, severity of effect, morbidity and mortality rate, and in the case of positive health effects also quality of life. More than one metric will be needed to capture all dimensions of health for a risk-benefit assessment.

Advantages using DALY

• Composite metrics, such as DALY or QALY, can be used for direct comparison of effects
• Effects expressed in a common metric can be compared, but care must be exercised in the interpretation of the comparison.

Challenges

• It is important to recognize that not all relevant dimensions are captured in these metrics, for example, whether the effect is in children or adults. This is because these metrics combine incidence with life years to obtain an estimate of years saved or lost respectively, so that a few young people with many years of potential life can give an equivalent value as a larger number of elderly people with far fewer years of potential life.
• Some of the DALY or QALY weightings are open for discussion.
• When reporting to the risk-benefit manager the risk benefit assessor needs to consider that the result “is more than a number” and should be considered together with the outcome of the Step 2 assessment.
• Currently, generally agreed metrics for positive health effects and well being are lacking, in part because there are no agreed weighting factors for positive health effects.

The Scientific Committee recommends that composite metrics are used to combine two or more of the following elements: increases or decreases in morbidity, mortality, disease burden, and quality of life. The choice of composite metrics should be made on a case by case basis, based on the specific risk-benefit question, identified hazards and positive health effects. The Scientific Committee recommends however, when reporting to the risk-benefit manager on the outcome of the risk-benefit assessment, to provide as well the respective health impact values expressed in the selected composite metric for each relevant health effect and each relevant sub population with their respective uncertainties.

Brafo

Brafo project tried to develop a framework that allows quantitative comparison of human health risks and benefits of foods and food compounds based on a common scale of measurement. It will be based on the evaluation of changes in the quality/duration of life using a system that allows weighting of data quality and severity of effect, with quantification by QALY or DALY-like methodology.

The framework will take into account how risks and benefits interrelate. It is intended that the methodology developed is sufficiently transparent to serve as a reference for the harmonisation of the evaluation methods used within the European Union and more widely in international evaluations.

Options others than to combine risk and benefit in a common scale were suggested, e.g. (1) to give a detailed risk-benefit description and leave any decisions to the risk-benefit manager, or (2) to express the assessment results as changes in risk or benefit (increments) and calculate the risk + benefit difference. It was agreed that more research and experience with different approaches are needed.

QALIBRA

In many cases, conventional risk assessment may show that adverse effects are unlikely. In other cases, a qualitative evaluation may be sufficient to conclude that either the risks or the benefits dominate. When this is not the case, it may be necessary to quantify not only the incidence of adverse and beneficial effects, but also the magnitude of their impact on health, their duration, and their impact on life expectancy. In addition, it may be helpful to combine these different dimensions of health impact into a single integrated measure such as disability-adjusted life years (DALYs).

Qalibra project developed a tool for the higher (quantitative) tiers of tiered approaches to risk-benefit assessment, such as those being considered by the EFSA and the related EU project Brafo.

Bepraribean

Bepraribean project produced seven peer reviewed publications (see Bepraribean papers in the end) on how to manage benefit-risks in different fields, and found that the most used integrated health measures in food-related benefit-risk analysis are DALY and QALY. QALY’s are traditionally mostly used to measure health gains at micro scale, for example to compare two interventions. For many diseases, disability weights, which are currently being revised in the Global Burden of Disease 2010 study, are available at the WHO website. Further, often the choice for DALY or QALY is a pragmatic one, based on data availability or experience of use rather than on a fundamental choice.

Alternative options for common currency

Utility

In utility the worst result for a decision-maker is defined as value 0 and the best as value 1. Utility of other results is defined as follows: If the decision-maker is indifferent (can not choose over options) about options a) the best option happens with probability p, and b) option under consideration happens with probability 1, then the utility of option under consideration is given value p.

Definition of benefits, the role of placebo effect

The concept of benefits is often not as clear as it might first seem. The clearest way of presenting health benefits is measuring improved health status due to intake of a substance which has a plausible toxicological mechanism of action in the human body. Modern medicinal treatments should fall into this category. However, often the evaluation of benefits is not that clear. When the effects are possible but small, which often is the case for example in herbal products and food supplements, placebo factor comes relevant (Kaptchuk 2008). Actually, it is a strong factor of various health benefits (Margo 1999). How to take into account this effect in benefit-risk assessments, is an open question (Hunsley and Westmacott 2007) Should the definition of benefits be broadened in a way that also placebo effects are included, and how to do this in a coherent and an objective way?

Comparison between alternative common currencies

Parameter	Incidences	Disability Adjusted Life Years (DALY)	Quality Adjusted Life Years (QALY)	Days of work lost	Costs in money	Utility
Does it sum up to a single metric?	No	Yes	Yes	Yes	Yes	Yes
Can it be used for non-health endpoints?	No	No	No	No	Yes	Yes
What is the basis for weighting?	No weighting	Severity weight, which is specific to a disease diagnosis.	Quality indices such as pain, cognitive capacity, orientation, anxiety, performance of daily routines. Specific to the condition of a patient.	Reduced capacity of a patient in the labour market.	Costs of treatment, opportunity loss due to a disease, and willingness to pay to avoid a disease.	Decision maker expresses his/her values about all possible outcomes on a scale from 0 to 1.
Where can the weights be used?	Universally; however, background rates vary from country to country and between sub-populations	Universally, although different cultures might have somewhat different severity weights.	Universally, although different cultures might have somewhat different severity weights.	Within a country, because different treatment practices and social security systems may lead to different loss of work for the same disease.	Within a country, because treatment costs and willingness to pay vary depending on economic wealth.	Only for a specific case. The weights depend on the value judgements of a specific decision maker. However, it is also possible to derive more generic weights for standard decision situations.
Who decides about the weights?	No weighting	An international panel of medical doctors (organised by WHO) using person trade-off technique (Salomon 2010).	Large panels of patients and doctors.	Can be derived from medical statistics for a country.	Contingent valuation studies performed in a country.	The decision maker of each decision.
What is the variable that is weighted?	Number of patients with specific diagnoses.	Number of patients with specific diagnoses, duration of each disease.	Number of people with different levels of health loss, duration of health loss.	Number of patients with specific diagnoses.	Number of patients with specific diagnoses, costs of treatment.	Any outcomes that the decision maker considers important.
Are the weights easily available?	Yes	Yes, provided by WHO.	No	Yes for some countries	Rather easy for some countries.	Only if the decision maker is involved in the assessment.
Is the summary indicator easy to understand without further explanations?	Yes	Fairly easy, deeper understanding requires insight	Fairly easy, deeper understanding requires insight	Yes	Yes, although readers may not understand that there are hidden value judgements of WTP.	No
Can the outcome results be compared to other cases?	Yes	Yes	Yes	Yes within a country, between countries only with caution	Yes within a country, between countries only with caution	Only if the decision maker is the same or his/her values are shared with in the other case.
Overall conclusion	Incidences are calculated anyway in a health assessment, and they should be published although it is not actually a summary measure.	DALY is the preferred method and better than QALY because the easy availability of weights and diagnoses.	QALY is also a good method and arguably more precise than DALY but information about health conditions (compared with diagnoses in DALY) and their weights is not easily available.	In many cases a practical metric, but ignores diseases in the young and elderly, and also varies between countries.	If non-health endpoints are considered, money should be considered as DALYs don't work. It is widely used and thus makes comparison easy (at least within a country).	Utility is theoretically the best alternative, and any of the other methods can be seen as a sub-method for utility. However, using utilities requires extensive cooperation with decision maker which is not always possible. On the other hand, this kind of cooperation is recommended anyway because of other reasons.
Recommendation: Which metrics should be used? (Note: there is no need to limit to only one.)	Use always	Use always	Use if data on health conditions and weights is available	Do not use routinely	Use always when non-health endpoints should be considered.	Should be used if there is a possibility for cooperation with the decision maker.

Conclusions

Based on the table, we identified DALYs, QALYs as potential measures for common currency. DALY is the most widely used measure and therefore the first choice for the work. Incidences, as the most transparent indication of the health effect should be presented along with common currencies. If one prefers to present something else than changes in health statuses, cost and utility are the best options for this.

Because calculating common currency often demands special data and some understanding about the methods, developing common currency calculation tools, such which has been developed in the EU-projects INTARESE and QALIBRA, and Health and Safety Executive (HSE), is highly appreciated.

References

Hunsley, J., and Westmacott, R. 2007. "Interpreting the magnitude of the placebo effect: mountain or Molehill?". J Clin Psychol 63 (4): 391–9.

Kaptchuk, JT., Kelley, JM., Deykinc, A., Waynea, PM., Louis C. Lasagnad, LC., Epsteine, IO., Kirschf, I., and Wechsler, ME. Do “placebo responders” exist? Contemporary Clinical Trials Volume 29, Issue 4, July 2008, Pages 587-595

Margo, CE. 1999. The Placebo Effect. Survey of Ophthalmology Volume 44, Issue 1, 8 July 1999, Pages 31-44

Salomon 2010. New disability weights for the global burden of disease. Bulletin of the World Health Organization. Volume 88, Number 12, December 2010, 877-953.

Bepraribean papers:

Verhagen, H., Tijhuis, MJ., Gunnlaugsdόttir, H., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Pohjola, MV., Tuomisto, JT., Ueland ,O., White, BC., and Holm, F. 2011. State of the art in benefit-risk analysis: Introduction. Food Chem Toxicol. 2011 Jun 12. Introduction paper

Tijhuis, MJ., de Jong, N., Pohjola, MV., Gunnlaugsdóttir, H., Hendriksen, M., Hoekstra, J., Holm, F., Kalogeras, N., Leino, O., van Leeuwen, FX., Luteijn, JM., Magnússon, SH., Odekerken, G., Rompelberg, C., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011a. State of the art in benefit-risk analysis: Food and nutrition. Food Chem Toxicol. 2011 Jun 12. Food and nutrition paper

Ueland, O., Gunnlaugsdottir, H., Holm, F., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Consumer perception. Food Chem Toxicol. 2011 Jun 12. Consumer perception paper

Magnússon, SH., Gunnlaugsdóttir, H., van Loveren, H., Holm, F., Kalogeras, N., Leino, O., Luteijn, JM., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Food microbiology. Food Chem Toxicol. 2011 Jun 12. Food and microbiology paper

Kalogeras, N., Odekerken, G., Pennings, JM., Gunnlaugsdόttir, H., Holm, F., Leino, O., Luteijn, JM., Magnússon, SH., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011. Food Chem Toxicol. 2011 Jun 12. Economics and marketing-finance paper

Luteijn, JM., White, BC., Gunnlaugsdóttir, H., Holm, F., Kalogeras, N., Leino, O., Magnússon, SH., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., McCarron, PA., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Medicines. Food Chem Toxicol. 2011 Jun 12. Medicines paper

Pohjola, MV., Leino, O., Kollanus, V., Tuomisto, JT., Gunnlaugsdóttir, H., Holm, F., Kalogeras, N., Luteijn, JM., Magnússon, SH., Odekerken, G., Tijhuis, MJ., Ueland, O., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Environmental health. Food Chem Toxicol. 2011 Jun 12. Environmental health paper

Tijhuis, MJ., Pohjola, MV., Gunnlaugsdóttir, H., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Potof, M., Tuomisto, JT., Ueland, O., White, BC., Holm, F., and Verhagen, H. 2011b. Looking beyond Borders: Integrating Best Practices in Benefit-Risk Analysis into the Field of Food and Nutrition. xxxx xxxx xxxx To be submitted in September 2011.

Projects

<mfanonymousfilelist></mfanonymousfilelist>