Common currency in health assessments

This page is a knowledge crystal of subtype method. The page identifier is Op_en4692
Moderator:Olli (see all)
Upload data {{#opasnet_base_link:Op_en4692}}

Scope

PlantLIBRA projects aims to foster the safe use of food supplements containing plants or botanical preparations, by increasing science-based decision-making by regulators and food chain operators.
 PlantLIBRA is structured to develop, validate and disseminate data and methodologies for risk and benefit assessment. Common currency methodology presented in this document answers to the need to develop food supplement benefit-risk methodology.

We may qualitatively compare things very different in nature, apples and oranges. In cases with very large and clear differences between the choices, qualitative comparisons might be enough. However, the question easily becomes value driven. Common measure is an answer to this problem. This document reviews identifying a common metric to be used in food supplement benefit-risk assessments. Several European research projects have already been dealing with common currency issue, therefore their conclusion are also presented here. In the end we select the most suitable common currency.

Most established options for common currency

There are several common currency measures available for estimating benefits and risks of food supplements. Therefore, the list of options in this document is not supposed to be all-inclusive but we briefly introduce six of the potential ones. Also example plants are presented and how the common currencies can be applied to estimate the effects. Health benefits refer to ones described in the PlantLIBRA Description of Work, and risks refer to ones presented in the PlantLIBRA milestone 4.1.

Incidences

Incidence information as a health measure is the absolute number of cases during a give time. Commonly, it is expressed as a proportion or a rate, where number of new cases is divided by the population initially at risk. It can also be used for presenting person-time incidence rate, where the denominator is the sum of the person-time of the at risk population. Incidence should not be confused with prevalence which is a measure of the total number of cases of disease in a population rather than the rate of occurrence of new cases. Incidence is the standard quantitative measure in public health.

Data needs:

Incidence data requires good register of diseases, which are usually readily available in all countries. International Statistical Classification of Diseases and Related Health Problems (ICD) is the preferred indicator of incidence data (WHO 2007).

Example of use:

Echinacea purpurea

Benefit: Decreased incidence of lower esophageal pressure

Risk: Incidence of genotoxicity induced tumors in descendants, according to ICD codes (Huntley et al, 2005; Jacobsson et al, 2009; Kocaman et al, 2008; Lee and Werth, 2004; Ondrizek et al, 1999; Soon and Crawford, 2001; Taylor et al, 2003)

Unit: Number of new cases for two different health end points

Example of use:

Camelia sinensis

Risk: Hepatotoxicity induced incidence of several liver diseases.

Unit: Number of new cases, according to ICD codes

Disability adjusted life years (DALY)

DALY is a measure of overall burden of disease. It contains both mortality and morbidity, which makes it useful and comprehensive for comparing the health effects of different nature, and it has become increasingly common in the field of public health. Methodologically, it summarizes years of life lost (YLL) and years lived with disease (YLD). One DALY is equal to one year of healthy life lost. The less DALY, the better. Each disease is assigned with severity weights which have been derived in combination of expert panels and patient reports. Severity weight of death is assigned value of 1.0, and perfect health 0. (WHO 2011, Murray and Lopez 1995).

Data needs:

Negative DALYs can be understood as avoided diseases. Disability weights can be found for most of the ICD-coded diseases (WHO 2004, Opasnet 2011b). Expert judgement must be used for estimating duration of disease if such data can not be found.

Example of use:

Serenoa serrulata Hook f.

Benefit: reduced disease burden of benign prostatic hyperplasia

Unit: avoided DALY (avoided disability due to disease)

Example of use:

Cinnamomun zeylanicum

Risk: Disease burden of allergic reactions (Endo and Rees 2007; Tremblay Avon 2008)

Unit: DALY for suffering from allergic reactions

Data need:

• DALY weight
• duration of disease (occurence)

Quality Adjusted Life Years (QALY)

QALY is very much like DALY. The greatest difference is that it measures the positive health state, apart from DALY. Similarly, it includes both the quality and the quantity of life lived. The more QALYs, the better. Using QALY requires utility independent, risk neutral, and constant proportional tradeoff behavior. The QALY is based on the number of years of life that would be added by the intervention, so it is clearly health oriented measure, apart from DALY. Each year in perfect health is assigned the value of 1.0, and value of 0.0 for death. Similarly to DALY, weights are assigned to health states. Determining the weight associated with a particular health state can be done using standard descriptive systems which categorizes health states according to the following dimensions: mobility, self-care, usual activities (e.g. work, study, homework or leisure activities), pain/discomfort and anxiety/depression. (Schlander 2010, Health Economics 2010)

Data needs:

Weights can be acquired several ways. By using time-trade-off, standard gamble or visual analogue scale. It can also be estimated by use of standard descriptive systems such as the EuroQol Group's EQ5D questionnaire, which categorises health states according to the following dimensions: mobility, self-care, usual activities (e.g. work, study, homework or leisure activities), pain/discomfort and anxiety/depression.

Example of use:

Cassia angustifolia Vahl.

Benefit: Improved quality of life in form of intestinal regularity

Risk: Reduced quality of life due to constipation

Unit: QALY

Days of work lost

This measure tries to capture the disease burden of working age population to be further analyzed in more sophisticated measures, such as cost. The information is derived from medical statistics, readily available in some countries. Another advantage is that it is a societal measure and can be used to assess association between productivity and health. The biggest shortage of this measure is that it totally ignores other than working population. Therefore using this measure to assess public health is misleading.

Data needs:

Occupational health data, which is often sparse. Moreover, short absence from work is often recorded without a specific disease. Longer sick leaves are therefore dominant.

Example of use:

Aloe ferox Mill.

Benefit: Decreased number of days of work lost due to improved intestinal regularity

Risk: Increased number of days of work lost due to ophthalmia (Fraunfelder 2004; Luyckx et al. 2002; Willems et al. 2003)

Unit: Number of days

Cost in money

One can also translate health outcomes into money, for example estimating the treatment costs for illnesses, and reduced labor. It can also be used for estimating willingness to pay to avoid a disease. The advantage is that it can easily be compared with other societal costs, and therefore it can be a convenient measure for decision-making. In some countries data requirements are quite easy to fulfill, however, in others very challenging. Additionally, willingness to pay vary depending on economic wealth, and the objectivity of the measure can also be questionable as hidden judgements might be involved in the process. (Holland et al. 2005)

Data needs:

Several sources can be used, e.g. data from hospitals, occupational health institutes, and health economics.

Example of use:

Ginkgo biloba L.

Benefits: Increased GDP due to enhanced productivity of the public

Risk: Cost of treatment of haemorrhage patients

Unit: €

Utility

Utility is a mathematical expression that assigns a value to all possible choices. The presumption is that the decision-maker has to be indifferent about options (can not choose over options). The worst result for a decision-maker is defined as value 0 and the best as value 1. If the best option happens with probability p, and option under consideration happens with probability 1, then the utility of option under consideration is given value p. This approach is probably the most academic one from the list of options here but what it lacks in practicality, it benefits in applicability (Lintley 1998, Lintley 1991) .

Data need:

Ultimately, this is a value driven approach, and therefore needs weights assigned for a specific purpose. Summoning an expert panel for the project would be the easiest solution for acquiring weights.

Example of use:

Olea Europea L.

Benefit: Antispasmodic properties

Risk: Anaphylaxis (Alvarez-Eire Marimar et al, 2010; Florido et al, 2002)

Unit: Combined utility of benefits and risks (number from 0 to 1 range)

Previous experience on using common currency

Several projects have already assessed the use of common currency. The next sections contain background information and conclusions from these projects.

EFSA 2006

In 2006 EFSA suggested that the assessments with both risk and benefit ideally should be performed under the same criteria for weighing the evidence and identifying the uncertainties. The presentation of the results of the risk-benefit assessment must fit the predefined purpose of the request and make clear where the certainties and uncertainties are in order to compare the relative confidence on the benefits with the risks. The following possible common scale measures were mentioned:

• Incidences
• Disability Adjusted Life Years DALY
• Quality Adjusted Life Years QALY
• Days of work lost
• Costs in money

Further EFSA 2006 concluded that QALY are, nevertheless, based on a number of assumptions, and are more difficult to quantify than DALY. The difficulty in expressing results from toxicological studies in experimental animals as DALY needs to be overcome. Using costs requires equal cost structures across countries/world and is difficult to communicate. It was agreed that more research and experience with different approaches are needed.

Advantages using DALY

• The advantages of using DALYs are that they represent an established procedure to compare risks of different nature
• DALYs may provide guidance to the risk manager on how to prioritize the direction of targeted intervention measures
• DALYs have a time-scale

Challenges using DALY

• DALYs are applied at the societal, rather than the individual, level. It is possible to apply DALYs in risk-benefit assessment, but appropriate data may seldom be available.
• The difficulties in using DALYs are that clear messages are needed so that the numbers generated are not taken out of context.
• It also seems difficult to include preventive aspects (such as effects of preservation) or absence of risk rather than benefit.
• The difficulty in expressing results from toxicological studies in experimental animals as DALYs needs to be overcome.
• Because the scale is likely to differ for different analysis, no generally applicable measurement scale is likely to be developed.

EFSA 2010

Scientific committee, appointed by the EFSA 2010, recommended a stepwise approach for assessing the benefits and risks of food. The last of the three steps utilizes common currency approach to combine two or more of the following elements: increases or decreases in morbidity, mortality, disease burden, and quality of life. The choice of composite metrics should be made on a case by case basis, based on the specific risk-benefit question, identified hazards and positive health effects. Health effects can be assessed in a number of different dimensions, such as incidence of effect, severity of effect, morbidity and mortality rate, and in the case of positive health effects also quality of life. More than one metric will be needed to capture all dimensions of health for a risk-benefit assessment.

Advantages using DALY

• Composite metrics, such as DALY or QALY, can be used for direct comparison of effects
• Effects expressed in a common metric can be compared, but care must be exercised in the interpretation of the comparison.

Challenges

• It is important to recognize that not all relevant dimensions are captured in these metrics, for example, whether the effect is in children or adults. This is because these metrics combine incidence with life years to obtain an estimate of years saved or lost respectively, so that a few young people with many years of potential life can give an equivalent value as a larger number of elderly people with far fewer years of potential life.
• Some of the DALY or QALY weightings are open for discussion.
• When reporting to the risk-benefit manager the risk benefit assessor needs to consider that the result “is more than a number” and should be considered together with the outcome of the Step 2 assessment.
• Currently, generally agreed metrics for positive health effects and well being are lacking, in part because there are no agreed weighting factors for positive health effects.

The Scientific Committee recommends that composite metrics are used to combine two or more of the following elements: increases or decreases in morbidity, mortality, disease burden, and quality of life. The choice of composite metrics should be made on a case by case basis, based on the specific risk-benefit question, identified hazards and positive health effects. The Scientific Committee recommends however, when reporting to the risk-benefit manager on the outcome of the risk-benefit assessment, to provide as well the respective health impact values expressed in the selected composite metric for each relevant health effect and each relevant sub population with their respective uncertainties.

Brafo

Brafo project tried to develop a framework that allows quantitative comparison of human health risks and benefits of foods and food compounds based on a common scale of measurement. It will be based on the evaluation of changes in the quality/duration of life using a system that allows weighting of data quality and severity of effect, with quantification by QALY or DALY-like methodology.

The framework will take into account how risks and benefits interrelate. It is intended that the methodology developed is sufficiently transparent to serve as a reference for the harmonisation of the evaluation methods used within the European Union and more widely in international evaluations.

Options others than to combine risk and benefit in a common scale were suggested, e.g. (1) to give a detailed risk-benefit description and leave any decisions to the risk-benefit manager, or (2) to express the assessment results as changes in risk or benefit (increments) and calculate the risk + benefit difference. It was agreed that more research and experience with different approaches are needed.

QALIBRA

In many cases, conventional risk assessment may show that adverse effects are unlikely. In other cases, a qualitative evaluation may be sufficient to conclude that either the risks or the benefits dominate. When this is not the case, it may be necessary to quantify not only the incidence of adverse and beneficial effects, but also the magnitude of their impact on health, their duration, and their impact on life expectancy. In addition, it may be helpful to combine these different dimensions of health impact into a single integrated measure such as disability-adjusted life years (DALYs).

Qalibra project developed a tool for the higher (quantitative) tiers of tiered approaches to risk-benefit assessment, such as those being considered by the EFSA and the related EU project Brafo.

Bepraribean

Bepraribean project produced seven peer reviewed publications (see Bepraribean papers in the end) on how to manage benefit-risks in different fields, and found that the most used integrated health measures in food-related benefit-risk analysis are DALY and QALY. QALY’s are traditionally mostly used to measure health gains at micro scale, for example to compare two interventions. For many diseases, disability weights, which are currently being revised in the Global Burden of Disease 2010 study, are available at the WHO website. Further, often the choice for DALY or QALY is a pragmatic one, based on data availability or experience of use rather than on a fundamental choice.

Definition of health benefits

There are several definitions for benefits, depending on scope and viewpoint. EFSA uses the following general definition "The probability of a positive health effect and/or the probability of a reduction of an adverse health effect in an organism, system, or (sub)population, in reaction to exposure to an agent" (EFSA 2010).

In the food supplement field we can use the following definition outlined in the PlantLIBRA work package 2: "The attainment of specific physiological objectives, such as reduction of risk factors for chronic diseases and the maintenance of the human homeostasis, which is the body’s capability to physiologically regulate well-being and ensure stability and balance in response to changes in the external environment." This definition includes variety of different health related end points, some of which are not directly reducing diseases but rather improving quality of life, such as:

• improvement of visual adaptation to the dark
• enhancement of mood
• improved defense against bacterial pathogens in the lower urinary tract

These benefits may be more difficult to evaluate quantitatively because of their broader definition of health benefits. Disease driven common currencies, such as incidences, days of work lost, and DALY, are often less adequate for evaluating these health effects, whereas more general approaches, such as cost in money, utility, and QALY are more suitable for this but in turn, lack in background data availability and rely more on expert judgements. Therefore, the selecting process is not a simple task. The challenges and examples of these health benefits are presented in Tables 1, 2, and 3. Importantly, the examples given in these tables are not the only options how to estimate the health benefits but there are often several other ways of estimating the outcomes. In the end, the question is to balance with data available and common currency best suitable for a food supplement benefit-risk assessment. For example, DALY is a well established method for estimating the effects in common currency but for some unconventional health endpoints disability weights are not readily available, and they have to be evaluated by an expert panel or be interpolated from other disease weights.

Table 1. Improvement of visual adaptation to the dark

Parameter	Applicable	Challenges	Example
Incidence	No	Incidence is difficult to define due to sliding scale of the benefit	If such incidence data existed, comparison between subsequent year incidences
DALY	To some extent	Severity weight surrogate must be found (e.g. decreased severity of night blindness)	Avoided DALYs due to recovery from night blindness
QALY	To some extent	Weight surrogate must be found	Improved quality of life, expert panel assigned weight needed for this particular health benefit
Days of work lost	Yes	Data for cause of retirement not be found in all countries	Avoided days of work lost due to vision enhancement
Cost in money	Yes	Calculation needs numerous assumtions and estimations	Treatment costs of night blindness
Utility	Yes	Assigning weight for the health effect is ultimately an expert judgement	Utility gained because of health effect

Table 2. Enhancement of mood

Parameter	Applicable	Challenges	Example
Incidence	To some extent	Incidence is difficult to define due to sliding scale of the benefit	Reduced incidence of mild depression
DALY	To some extent	Severity weight surrogate needed (reducing mild depression)	Avoided DALYs due to reduced mild depression cases
QALY	To some extent	Assigning weight needs expert judgement	Improved quality of life due to acquired increased QALYs
Days of work lost	Yes	Does not always show as days of work lost	Reduced days of work lost due to depression (clinically diagnosed)
Cost in money	Yes	Numerous assumption and estimations must be used, monetary value of anxiety	Saved direct costs due to treatment of mild depression
Utility	Yes	Assigned weight might differ greatly between individuals	Utility of better psychological condition

Table 3. Improved defense against bacterial pathogens in the lower urinary tract

Parameter	Applicable	Challenges	Example
Incidence	To some extent	Incidence data for urinary tract infections may not be available for all countries	Decreased incidence of infection in the lower urinary tract
DALY	Yes	Severity weight surrogate probably needed	Avoided DALYs due to infection in the lower urinary tract
QALY	Yes	Weight of the health effect	Gained QALYs due to healthy urinary tract
Days of work lost	Poorly	Occupational data hard to find, may not require absence from work	Reduced absence from work
Cost in money	Yes	Various different treatments, monetary value of pain	Saved medical costs of urinary tract infections
Utility	Yes	Assigning weight	Utility gained due to healthy urinary tract

There is also a strong psychological factor on health benefits (and health risks). This is called benefit (or similarly risk) perception. This does not relate only to personal satisfaction, but also to the placebo effect which in case of food supplements might be significant and clearly has impact on public health. The subject, however, is controversial and not quantitatively taken into consideration here as a health benefit but it is acknowledged as a potential factor in estimating health effects.

Comparison between alternative common currencies

Table 4 presents how and where the proposed common currencies can be used.

Table 4. Common currencies and their fundamental properties

Parameter	Incidences	Disability Adjusted Life Years (DALY)	Quality Adjusted Life Years (QALY)	Days of work lost	Costs in money	Utility
Does it sum up to a single metric?	No	Yes	Yes	Yes	Yes	Yes
Can it be used for non-health endpoints?	No	No	No	No	Yes	Yes
What is the basis for weighting?	No weighting	Severity weight, which is specific to a disease diagnosis.	Quality indices such as pain, cognitive capacity, orientation, anxiety, performance of daily routines. Specific to the condition of a patient.	Reduced capacity of a patient in the labour market.	Costs of treatment, opportunity loss due to a disease, and willingness to pay to avoid a disease.	Decision maker expresses his/her values about all possible outcomes on a scale from 0 to 1.
Where can the weights be used?	Universally; however, background rates vary from country to country and between sub-populations	Universally, although different cultures might have somewhat different severity weights.	Universally, although different cultures might have somewhat different severity weights.	Within a country, because different treatment practices and social security systems may lead to different loss of work for the same disease.	Within a country, because treatment costs and willingness to pay vary depending on economic wealth.	Only for a specific case. The weights depend on the value judgements of a specific decision maker. However, it is also possible to derive more generic weights for standard decision situations.
Who decides about the weights?	No weighting	An international panel of medical doctors (organised by WHO) using person trade-off technique (Salomon 2010).	Large panels of patients and doctors.	Can be derived from medical statistics for a country.	Contingent valuation studies performed in a country.	The decision maker of each decision.
What is the variable that is weighted?	Number of patients with specific diagnoses.	Number of patients with specific diagnoses, duration of each disease.	Number of people with different levels of health loss, duration of health loss.	Number of patients with specific diagnoses.	Number of patients with specific diagnoses, costs of treatment.	Any outcomes that the decision maker considers important.
Are the weights easily available?	Yes	Yes, provided by WHO.	No	Yes for some countries	Rather easy for some countries.	Only if the decision maker is involved in the assessment.
Is the summary indicator easy to understand without further explanations?	Yes	Fairly easy, deeper understanding requires insight	Fairly easy, deeper understanding requires insight	Yes	Yes, although readers may not understand that there are hidden value judgements of WTP.	No
Can the outcome results be compared to other cases?	Yes	Yes	Yes	Yes within a country, between countries only with caution	Yes within a country, between countries only with caution	Only if the decision maker is the same or his/her values are shared with in the other case.
Overall conclusion	Incidences are calculated anyway in a health assessment, and they should be published although it is not actually a summary measure.	DALY is the preferred method and better than QALY because the easy availability of weights and diagnoses.	QALY is also a good method and arguably more precise than DALY but information about health conditions (compared with diagnoses in DALY) and their weights is not easily available.	In many cases a practical metric, but ignores diseases in the young and elderly, and also varies between countries.	If non-health endpoints are considered, money should be considered as DALYs don't work. It is widely used and thus makes comparison easy (at least within a country).	Utility is theoretically the best alternative, and any of the other methods can be seen as a sub-method for utility. However, using utilities requires extensive cooperation with decision maker which is not always possible. On the other hand, this kind of cooperation is recommended anyway because of other reasons.
Recommendation: Which metrics should be used? (Note: there is no need to limit to only one.)	Use always	Use always	Use if data on health conditions and weights is available	Do not use routinely	Use always when non-health endpoints should be considered.	Should be used if there is a possibility for cooperation with the decision maker.

Conclusions

The selection of common currency is not an obvious task because they all provide their own strengths and weaknesses. We identified DALYs, and QALYs as potential measures for common currency. DALY is the most widely used measure and therefore the first choice for the work. Incidences, as the most transparent indicator of health effect should be presented along with the selected common currency measure. If one prefers to present something else than changes in health statuses, cost and utility are the best options for this. Further, tables 1-3 present a practical challenge in case of health benefits that rather promote health than prevent from an adverse health effect. In this case, selecting the appropriate common currency is a trade off between practicality and data availability.

Because calculating common currency often demands special data and some understanding about the methods, developing common currency calculation tools, such which has been developed in the EU-projects INTARESE (INTARESE 2011, Opasnet 2011a) and QALIBRA (QALIBRA 2011, Hoekstra et al. 2011), is highly appreciated.

See also

http://en.opasnet.org/w/Common_currency_in_health_assessments

References

Alvarez-Eire Marimar G, Pineda F, Varela Losada S, González de la Cuesta C, Menéndez Villalva M, Palacios R. Anaphylaxis to olive fruit due to lipoproteins sensitization. 29th EAACI Congress. Poster Group 1 - Molecular chemistry of novel allergens European Academy of Allergy and Clinical Immunology, 2010. http://www.postersessiononline.com/173580348_eu/congresos/ 29EAACI/aula/-P_864_29EAACI.pdf

Endo H, Rees TD. Cinnamon products as a possible etiologic factor in orofacial granulomatosis. Med Oral Patol Oral Cir Bucal 2007; 12: E440-4.

Florido JF, Quiralte EJ, Arias SAJM, Saenz SPB, Martin CE. An allergen from Olea europaea pollen (Ole e 7) is associated with plant-derived food anaphylaxis. Allergy 2002; 57: 53-9.

Fraunfelder FW. Ocular Side Effects From Herbal Medicines and Nutritional Supplements. Am J Ophthalmol 2004; 138: 639-47.

Health Economics. 2010. What is QALY? http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/QALY.pdf Accessed September 2011.

Hoekstra, J., Hart, A., Owen, H., Zeilmaker, M., Bokkers, Bas., Thorgilson, B., Gunnlaugsdottir, H. 2011. Fish, contaminants and human health; Quantifying and weighing benefits and risks. Submitted manuscript.

Holland, M., Watkiss, P., Pye, S., de Oliveira, A., and Van Regemorter, D. 2005. Cost-Benefit Analysis of Policy Option Scenarios for the Clean Air for Europe programme. AEA Technology Environment.

Huntley AL, Thompson Coon J, Ernst E. The safety of herbal medicinal products derived from Echinacea species: a systematic review. Drug Saf 2005; 28: 387-400.

INTARESE. Integrated Assessment of Health Risks of Envirnonmental Stressors in Europe. Front page: http://www.intarese.org/ Accessed September 2011.

Kocaman O, Hulagu S, Senturk O. Echinacea-induced severe acute hepatitis with features of cholestatic autoimmune hepatitis. Eur J Intern Med 2008; 19: 148.

Jacobsson I, Jönsson AK, Gerdén B, Hägg S. Spontaneously reported adverse reactions in association with complementary and alternative medicine substances in Sweden. Pharmacoepidemiol Drug Saf 2009; 18: 1039-47.

Lee AN, Werth VP. Activation of autoimmunity following use of immunostimulatory herbal supplements. Arch Dermatol 2004; 140: 723-7.

LINDLEY, DV. 1991. SUBJECTIVE-PROBABILITY, DECISION-ANALYSIS AND THEIR LEGAL CONSEQUENCES. JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES A-STATISTICS IN SOCIETY. Volume 154, Pages 83-92

Lintley, DV. 1998. Decision analysis and bioequivalence trials. STATISTICAL SCIENCE. Vol 13, Issue 2, Pages 136-141

Luyckx VA, Ballantine R, Claeys M, Cuyckens F, Van den Heuvel H, Cimanga RK, Vlietinck AJ, De Broe ME, Katz IJ. Herbal remedy-associated acute renal failure secondary to Cape aloes. Am J Kidney Dis 2002; 39: E13.

Murray, CJL., and Lopez, AD. 1995. Global mortality, disability, and the contribution of risk factors: Global Burden of Disease Study. Vol 349, Issue 9063, 1436-1442

Ondrizek RR, Chan PJ, Patton WC, King A. Inhibition of human sperm motility by specific herbs used in alternative medicine. J Assist Reprod Genet 1999; 16 (2): 87-91.

Opasnet 2011a. Impact calculation tool. http://en.opasnet.org/w/Impact_calculation_tool Accessed September 2011.

Opasnet 2011b. Disability weights. http://en.opasnet.org/w/Disability_weights Accessed October 2011.

QALIBRA. 2011. Quality of life - integrated benefit and risk. Calculation tool: http://www.qalibra.eu/ accessed September 2011.

PlantLIBRA deliverable 2.2. Review of existing experimental approaches for benefit assessment, including critical definition of benefits and (bio)markers.

Salomon 2010. New disability weights for the global burden of disease. Bulletin of the World Health Organization. Volume 88, Number 12, December 2010, 877-953.

Schlander, M. 2010. Measures of efficiency in healthcare: QALMs about QALYs? Z. Evid. Fortbild. Qual. Gesundh. wesen (ZEFQ) 104 (2010) 209–226

Soon SL, Crawford RI. Recurrent erythema nodosum associated with Echinacea herbal therapy. J Am Acad Dermatol 2001; 44: 298-9.

Taylor JA, Weber W, Standish L, Quinn H, Goesling J, McGann M, Calabrese C. Efficacy and safety of Echinacea in treating upper respiratory tract infections in children: a randomized controlled trial. JAMA 2003; 290: 2824-30.

Tremblay S, Avon SL. Contact allergy to cinnamon: case report. J Can Dent Assoc 2008; 74: 445-61.

WHO. 2004. Disability weights, discounting and age weighting of DALYs. http://www.who.int/healthinfo/global_burden_disease/daly_disability_weight/en/index.html

WHO 2007. International Statistical Classification of Diseases and Related Health Problems 10th Revision http://apps.who.int/classifications/apps/icd/icd10online/ Accessed Oct 2011.

WHO. 2011. Health statistics and health information systems. http://www.who.int/healthinfo/global_burden_disease/metrics_daly/en/index.html Accessed September 2011.

Willems M, Van Buuren HR, de Krijger R. Anthranoid self-medication causing rapid development of melanosis coli. Neth J Med 2003; 61: 22-4.

Bepraribean papers:

Verhagen, H., Tijhuis, MJ., Gunnlaugsdόttir, H., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Pohjola, MV., Tuomisto, JT., Ueland ,O., White, BC., and Holm, F. 2011. State of the art in benefit-risk analysis: Introduction. Food Chem Toxicol. 2011 Jun 12. Introduction paper

Tijhuis, MJ., de Jong, N., Pohjola, MV., Gunnlaugsdóttir, H., Hendriksen, M., Hoekstra, J., Holm, F., Kalogeras, N., Leino, O., van Leeuwen, FX., Luteijn, JM., Magnússon, SH., Odekerken, G., Rompelberg, C., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011a. State of the art in benefit-risk analysis: Food and nutrition. Food Chem Toxicol. 2011 Jun 12. Food and nutrition paper

Ueland, O., Gunnlaugsdottir, H., Holm, F., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Consumer perception. Food Chem Toxicol. 2011 Jun 12. Consumer perception paper

Magnússon, SH., Gunnlaugsdóttir, H., van Loveren, H., Holm, F., Kalogeras, N., Leino, O., Luteijn, JM., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Food microbiology. Food Chem Toxicol. 2011 Jun 12. Food and microbiology paper

Kalogeras, N., Odekerken, G., Pennings, JM., Gunnlaugsdόttir, H., Holm, F., Leino, O., Luteijn, JM., Magnússon, SH., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011. Food Chem Toxicol. 2011 Jun 12. Economics and marketing-finance paper

Luteijn, JM., White, BC., Gunnlaugsdóttir, H., Holm, F., Kalogeras, N., Leino, O., Magnússon, SH., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., McCarron, PA., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Medicines. Food Chem Toxicol. 2011 Jun 12. Medicines paper

Pohjola, MV., Leino, O., Kollanus, V., Tuomisto, JT., Gunnlaugsdóttir, H., Holm, F., Kalogeras, N., Luteijn, JM., Magnússon, SH., Odekerken, G., Tijhuis, MJ., Ueland, O., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Environmental health. Food Chem Toxicol. 2011 Jun 12. Environmental health paper

Tijhuis, MJ., Pohjola, MV., Gunnlaugsdóttir, H., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Potof, M., Tuomisto, JT., Ueland, O., White, BC., Holm, F., and Verhagen, H. 2011b. Looking beyond Borders: Integrating Best Practices in Benefit-Risk Analysis into the Field of Food and Nutrition. xxxx xxxx xxxx To be submitted in September 2011.

Projects

<mfanonymousfilelist></mfanonymousfilelist>