Environmental burden of disease estimates

From Opasnet
Jump to: navigation, search
EBoDE project
This page is a product of the EBoDE project. The final report of the EBoDE project has been published as a report in 2011[1] and also as web pages in Opasnet. These links lead to parts of the report.

EBoDE project: main page | overview | contributors | data overview | Parma meeting | abbreviations | all pages

Methods: environmental burden of disease calculation | selection of exposures and health effects | data needed | impact calculation tool

Health effects in Europe: benzene | dioxins | formaldehyde | lead | ozone | particulate matter | radon | second-hand smoke | transport noise | environmental burden of disease | results by country

In the EBoDE project, we have calculated the environmental burden of disease for nine stressors in six countries, for the year 2004. The following paragraphs present and discuss the results of these calculations. The primary results are presented as undiscounted, un-age-weighted DALYs per million people. In addition to the results presented in this chapter, Appendix A presents the results per country. Calculations were based on the most recent scientific evidence concerning population exposure-response functions, national exposure data, and WHO burden of disease data and methods for estimating disease burden where available. Even though the most recent scientific knowledge and data were used, many uncertainties and controversies remain (see Chapter 5). Results give only a crude ranking of environmental burden of disease associated with the stressors and need to be interpreted with caution.[1]

Overall results

The results of the EBoDE project suggest that 3–7% of the standard WHO discounted age-weighted burden of disease in the participating countries is associated with exposure to the selected nine environmental stressors. The aggregate results for all stressors are shown in Figure Relative public health impact of the selected environmental stressors, which also indicates the relative scientific strength of the evidence underlying the estimates. The quantitative uncertainty ranges provided in this figure are based on qualitative and semi quantitative evaluations of the uncertainties and author judgment.[1]

Relative public health impact of the selected environmental stressors in undiscounted un-ageweighted DALYs per population of a million in the participating countries. Numerical ranges reflect quantitative uncertainty in the average estimate. Variability between countries is in many cases much larger. (* =numerical model used in estimating threshold exceedances).

Pie chart shows the burden of disease related to the nine stressors proportional to each other. As can be seen from this figure, particulate matter (using PM2.5 as indicator) is estimated to be the leading factor associated with 6.000 to 10.000 non-discounted DALYs per million people. Overall, PM2.5 is responsible for approximately two thirds of the environmental burden of disease related to the nine stressor evaluated in EBoDE. After PM, transportation noise, second hand smoke and radon contribute to the largest share of the environmental burden of disease. These four factors together are estimated to be responsible for over 90 % of the total studied environmental burden of disease.[1]

Relative contribution of the nine targeted stressors to the burden of disease (undiscounted, un-age- weighted DALYs) attributed to these stressors, average over the six participating countries.

The quantitative results of the environmental burden of disease calculations per stressor and health endpoint, averaged over the six countries, are presented in Table Relative public health impact of the selected environmental stressors. The total results aggregated per stressor are shown in table of aggregate burden of disease. When we look at effects on mortality, the studied nine factors are estimated to be associated with approximately 1.6 million years of life lost (YLL, non-discounted, not age-weighted) in the participating countries in the 2004, or 6 900 YLL per a million inhabitants (Table Mortality and morbidity components). For most of the stressors, the impacts are dominated either by morbidity (formaldehyde, lead and traffic noise) or by mortality (benzene, dioxins, and radon). We realize that the selection of health endpoints and further assumptions are partly responsible for this effect. For dioxin, for example, all DALYs are due to mortality (i.e. YLL), as all cancers are assumed to be fatal in our estimates.[1]

Burden of disease in DALYs per million people for each stressor and endpoint, averaged over the participating countries.

Stressor Endpoint Non-discounted Discounted with lag Difference%
Total DALY DALY per million Total DALY DALY per million
Benzene Leukemia 741 3 341 1 -54.00%
Dioxin Total cancer incidence 112332 482 42429 182 -62.00%
SHS Lung cancers in non smokers 19381 83 3989 17 -79.00%
Ishaemic heart disease 157919 678 68154 292 -57.00%
Asthma induction, children (>21 yr) 30363 130 23892 103 -21.00%
Asthma induction, children (<14 yr) 10481 45 9988 43 -5.00%
Lower respiratory infections (<2 yr) 1414 6 595 3 -58.00%
Otitis media (<3 yr) 654 3 270 1 -59.00%
Formaldehyde Asthma aggrevation (<3 yr) 20 0 16 0 -23.00%
Lead LQ loss 106621 457 31453 135 -70.00%
Increased blood pressure 2930 13 2736 12 -7.00%
Road traffic noise High sleep disturbance (HSD) 167916 720 167916 720
Ischaemic heart disease (IHD) 13105 56 5890 25 -55.00%
Railway noise High sleep disturbance (HSD) 12253 53 12253 53
Aircraft noise High sleep disturbance (HSD) 7202 31 7202 31
Ozone Total mortality (non-violent) 8365 36 8122 35 -3.00%
Minor restricted activity days 4484 19 4484 19
Cough days, children 6042 26 6042 26
LRS days in children 1470 6 1470 6
PM2.5 Cardiopulmonary mortality 1081750 4642 499063 2141 -54.00%
Lung cancer mortality 348623 1496 72512 311 -79.00%
Vhronic bronchitis (COPD) 289711 1243 151089 648 -48.00%
Restricted activity days (RAD) 61003 262 61003 262
Radon Lung cancers 194277 834 40563 174 -79.00%
  • In calculations according to method 2B (see Table 3-19 and section 2.1), discounting, when applicable, was used without age-weighing.

Aggregate burden of disease by stressors per country for 2004/2005, in undiscounted, un-age-weighted DALYs per million people.

Stressor Belgium Finland France Germany Italy Netherlands Average
Benzene 2.5 3 3.4 2.7 4.2 1.6 3.2
Dioxin 453 330 586 466 483 242 482
SHS 1110 891 550 1235 975 749 945
Formaldehyde 0.2 1.6 0.1 0.1 0 0 0.1
Lead 298 118 461 235 946 217 470
Traffic noise 437 371 1483 591 734 775 860
Ozone 52 47 81 73 138 34 87
PM2.5 10462 4602 4572 8384 9378 8322 7642
Radon 1078 926 1146 620 866 453 834
Total 13892 7289 8883 11608 13525 10793 11324

Mortality and morbidity components of the non-discounted burden of disease by stressor in the six participating countries per population of a million. Cases where over 90 % of the burden of disease is caused by either morbidity or mortality are highlighted in color blue and bold.

Stressor Deaths(a) per year YLD per year YLL per year DALY per year YLL %
Formaldehyde 0 0.1 0 0.1 0.60%
Lead 0 470 0.1 470 0.02%
Benzene 0.2 0.1 3.1 3.2 97.00%
Traffic noise 4.2 810 50.3 860 5.80%
Dioxin 30 36 446 482 93.00%
Ozone 36 51 36 87 41.00%
Radon 51 16 817 834 98.00%
SHS(b) 78 n/a n/a 945 n/a
PM2.5 516 2063 5580 7642 73.00%
Total 715 3446 6933 11324 61.00%

a=Numbers of deaths are not valid indicators for some effects (see Uncertainties and limitations).

b=YLL+YLD split not available for SHS due to the different method of calculation.

Results by stressor

Figure 4-3 at the end of this paragraph shows the results (not discounted, not age-weighted) per stressor and per country. TURHA/MUUALLE???


Based on the available information about leukaemia, the total impact of benzene on public health is estimated to be low (see Figure Relative public health impact of the selected environmental stressors and Figure Relative contribution of the nine targeted stressors). Benzene impacts are the highest in Italy and the lowest in the Netherlands. The great quantity of two-wheelers with two-stoke engines in Italy may partly explain the high benzene exposures. The low benzene related burden of disease in the Netherlands is due to the lowest exposures. In France, data reflect a large number of dwellings, while in other countries data are limited to a smaller number of monitored houses; thus the high burden estimate for France is based on more reliable data than the other estimates. In addition, the presence or absence of tobacco smoke in indoor environments is not always reported, making comparison more difficult. This at least partly explains the higher levels in Finland, where benzene from smoking was included and was estimated to contribute approximately one third of the exposures (see also Table Characteristics of benzene in section Health effects of benzene in Europe). Further sources of uncertainty in the benzene related burden of disease estimates relate mainly to the availability of exposure data, exclusion of other health effects than leukaemia, and the potential interaction of benzene with other components of tobacco smoke.[1]

Dioxins and dioxin-like PCBs

The relative burden of disease related to dioxins is estimated to be medium (see Figure Relative public health impact and Figure Relative contribution of the nine targeted stressors), however, uncertainties are large. Effects of dioxins cannot easily be distinguished from other occupational risk factors; low-dose effects are very difficult to assess; thresholds for effects are mostly unknown; and exposure data are often only indirectly available. Our estimates only include effects of dioxins on total cancer incidence. This is a rather crude aggregate end-point. In addition, each cancer case was assumed to be fatal during the first year, which may also have lead to overestimation. Also, numerous more specific health end-points (see Health effects of dioxins in Europe) were not modelled. Therefore, it is yet unclear whether our estimates over- or underestimate the total burden of disease. The burden of disease related to dioxin exposure is relatively low in Finland and the Netherlands, and highest in France. Potential explanations for this are different eating habits and differences in food contamination. But also the different methods to evaluate the daily intake may contribute to an unknown extent of uncertainty.[1]

Second hand smoke

The burden of disease related to SHS is estimated to account for 600-1200 DALYs per million people (medium impact). As outlined in Second-hand smoke, not all health effects could be included in the calculations due to unavailability of statistics. Besides, uncertainties in our estimates relate to e.g. survey-based exposure measurements (as opposed to measuring personal exposures), relative risks and the various assumptions made in the method (e.g. smokers are not susceptible to SHS). The provided range around the best estimate has been based on a sensitivity analysis varying the main assumptions made in the method. Nonetheless, most evidence for SHS-related impacts is fairly consistent, and the estimates of the burden of disease are considered relatively stable. Burden of disease from second hand smoke are remarkably low in France and high in Germany. Potential explanations for the higher levels in Germany are the slightly higher exposure levels combined with higher prevalence of the relevant diseases, such as ischaemic heart disease, which increases the susceptibility to the risk factor. The disease burden due to SHS is still substantial. Trends, however, are decreasing due to European-wide implementation of smoke-free policies to reduce SHS exposure. The predicted exposure reductions between 2004 and 2010 as presented in Tables Modelled exposure to SHS and Table Estimated exposures for 2010 for second-hand smoke are quite significant. 100% smoke-free policies have shown significant reductions in mortality and should be implemented in all indoor workplaces, public places and public transports. Complementary educational strategies may be necessary to extend the protection of children and adult non-smokers at home.[1]


The burden of disease related to formaldehyde, based on asthma incidence in children under 3 years of age, is estimated to be relatively low. However, the consistency of the knowledge base is low, with uncertainties related to the difficulty of establishing a threshold for effects, a lack of epidemiological data and a large discrepancy in widely used models. Formaldehyde exposures are remarkably high in Finland (see also Figure Estimated formaldehyde exposure distribution in Finland). The absolute average concentration levels do not vary so drastically between the countries. However, because a threshold of 100 μg m-3 was applied, the relative differences between countries increased. The formaldehyde levels in Finland are higher than in many other developed countries due to the types of construction materials used and the relatively tightly sealed buildings. The risk estimates for formaldehyde depend strongly on the chosen threshold level. Even though identified as a known human carcinogen, it is likely that cancer effects are negligible in Europe, due to the fact that almost all exposures are below the threshold level as proposed by WHO (2000a, 2011). The currently available exposure data cannot be used for reliably estimating the fraction of the population that is exposed to formaldehyde level exceeding the threshold levels. That is caused by the fact that current formaldehyde monitoring techniques are not suitable for detecting peaks of exposure, which may occur during some domestic tasks or soon after home refurbishment, especially in the absence of proper ventilation. Other common sources of formaldehyde, such as widespread use of fragrances and SHS, may also cause exceedance of the threshold value. This may cause underestimation of the true formaldehyde-related burden of disease.[1]


Lead is estimated to contribute to 100-500 DALYs per million people (medium impact, see Figure Relative public health impact of the selected environmental stressors and Figure Relative contribution of the nine targeted stressors). These estimates are based on a limited population representativity and partly older data with uncertain trend estimations (see Health effects of lead in Europe). Other uncertainties relate to the availability of dose-response functions over the complete exposure spectrum, and the aggregation of effects. Lead impacts have only been based on IQ loss and mild mental retardation. At least at higher exposure levels that prevailed during earlier decades, lead exposures were associated with a larger number of health endpoints, ranging from hearing impairment to kidney failures. However, the evidence for these effects at the prevailing low level exposures is very limited (WHO, 2007b). Nonetheless, our burden of disease estimates may underestimate the true lead-related burden of disease (see also Uncertainties and limitations). Lead exposures are the highest in Italy. One of the most important reasons for this may be the relatively old age group in which blood lead levels were measured. As was shown in Health effects of lead in Europe, the blood lead levels in Italy were presented for people aged 18–64 years. In the Netherlands, in contrast, the sample included children aged 1–6 years. Because lead accumulates in the body over the years, this is probably the most important reason for lead-related burden of disease in the Netherlands being relatively low and in Italy relatively high. In addition, the data for Italy (Apostoli et al., 2002) are quite old (from 2000). Lead levels are expected to be lower in 2004. Some other uncertainties that may affect the comparability of lead results among countries include the fact that data from Finland, Germany and the Netherlands are not representative for the age-group considered (let alone for the whole population); and that some countries provided a Geometrical Mean (GM) instead of an Arithmetical Mean (AM). The GM is expected to be lower than AM, because there are few high values within the samples.[1]

Transportation noise

Since so many people are exposed to noise, the total associated disease burden is substantial despite the relatively small disability weights (0.04-0.09) and is estimated to cause an undiscounted, un-age-weighted average of 860 DALYs per million people. Transportation noise plays a great role in each included country but there are numerous differences between the countries. DALYs range from about 371 per million inhabitants in Finland (less densely populated and highly urbanized but with only very few cities with more than 250 000 inhabitants) up to 1 483 DALYs per million people in France (due to the fact that only data for the greater Paris area were available). These differences point out some major limitation due to incomplete exposure data from environmental noise directive reporting (for Belgium e.g. only available for the Flanders region) and due to different population and traffic densities in the countries. Furthermore, a strong limitation results from the fact that exposure data reported in the first stage of END-reporting represent only “hot spots” where noise levels are supposed to be much higher than in the countryside. Therefore, the results from the calculations are an underestimation of the total burden of a country but may be overestimating the risks when applied for the whole population of a country (as done when normalized by million inhabitants). Country comparability consequently is affected by the variability of representativity (more representative in highly urbanized countries). In addition, only exposure levels above Lnight 50dB (Lden 55 dB) were available from the END database, so no health impacts could be calculated for the lower exposure levels. Of all the estimated DALYs for transport noise, 94 % result high sleep disturbance (HSD). Because so many people are estimated to suffer from HSD, the DALYs are very sensitive to changes in the disability weight (with confidence intervals ranging from 0.04 to 0.09) and less sensitive to changes in exposure levels (e.g. due to other constant used for conversion of Lden to Lnight) or the exposure-response-functions. Further uncertainties result e.g. from missing exposure-response-functions for certain transport sources. Other potential sources of uncertainty relate to individual and societal factors affecting noise levels indoors (such as regular location of sleeping rooms, window opening habits, window insulation etc.), and exogenous factors affecting those habits (such as climatic prerequisites and house ownership).[1]


The relative impact of ozone on public health is medium (40-200 undiscounted, age-weighted DALYs), see Figure Relative public health impact of the selected environmental stressors and Figure Relative contribution of the nine targeted stressors. Even though not all health effects could be included, the selected morbidity health endpoints are estimated to account for 90% of the total effects. Uncertainties in the calculations relate, amongst other issues, to the estimated number of years of life lost for mortality. Ozone impacts are highest in the Mediterranean countries, represented here by Italy, as can be expected. Levels in the Netherlands are the lowest, probably because meteorological factors and relatively high levels of nitrogen oxide pollution consuming atmospheric ozone in congested areas. Ozone levels have been slowly increasing during the last decade and due to the secondary nature of ozone air pollution, reduction of the exposures is challenging. From the point of view of health impact assessment the duration of loss of life at death is a key factor that hopefully will be estimated more accurately in future.[1]

Particulate matter (PM)

In the six participating countries PM is estimated to cause a loss of 1.8 million DALYs annually, including 1.3 million years of life lost due to mortality (73% of the total DALYs). Overall 67 % of the estimated environmental burden of disease in the EBoDE study was explained by exposure to PM2.5 making it the most significant environmental factor affecting public health (see Figure Relative public health impact of the selected environmental stressors and Figure Relative contribution of the nine targeted stressors). Uncertainties in the PM related burden of disease relate to the exposure-response functions for e.g. chronic bronchitis; and the potential of double counting of morbidity effects by combining the restricted activity days and lower respiratory symptom days. Overall, the PM epidemiology has been most thoroughly reviewed of the stressors included in this study. In the context of the CAFE study the estimates calculated for total and cause specific mortality were debated; the scientific evidence and mechanistic understanding of the causal processes are stronger for cardiopulmonary causes, but the effect estimates are higher for total non-violent mortality. We followed the CAFE approach to report the slightly lower cause specific results, which also can be used to meaningfully look at the ratio of mortality versus morbidity impacts. Particulate matter impacts are the lowest in Finland and France and the highest in Belgium and Italy, which are known to be hotspots of particulate matter air pollution. Annual average concentrations of PM10, monitored in EU urban background locations, show no significant decrease over the period 2000–2007 (Airbase, 2009). Available data are still too limited for reliable assessment of PM2.5 trends, but particulate matter exposures are expected to have a very slight downward trend due to the improvements in vehicle engine technology and emission controls in industry and energy production. However, the emissions of resuspended particles created by road traffic are expected to continue to grow due to the increasing traffic volumes. A significant reduction in current PM levels could be achieved only if all feasible emission reduction measures were implemented (the maximum feasible reduction scenario) (WHO, 2010c). This is a challenge for the current policy as it involves the implementation of new technologies (e.g. low-emission diesel cars) and the creation of conditions that support individual behavioural change. The use of health impact assessments as a standard tool in air quality and health policy, together with follow-up programmes (accountability) focusing on health consequences, is urgently needed across the Region (WHO, 2010c).[1]


Radon is estimated to contribute 600-900 DALYs (undiscounted) per million people in the participating countries (see Figure Relative public health impact of the selected environmental stressors and Figure Relative contribution of the nine targeted stressors). The radon related burden of disease is the highest in France and Belgium; and lowest in the Netherlands. These differences are mainly caused by the differences in geological substrates beneath houses and the use of different building materials. For example, in Belgium the average radon concentration in houses is 35 Bq/m3 for Flanders and 70 Bq/m3 for Wallonia (southern part of the country). This difference is mainly due to the larger uranium concentration in rocks present in the southern part. On top, there are often relatively more cracks in these rocks simplifying the release of radon. Concentrations of radon in houses of more than 400 Bq/m3 are sometimes measured in the southern part of Belgium. Measures as placing an impermeable screen and adapting the ventilation system may substantially reduce the radon indoor concentration. For radon, we have used a relative risk to calculate cases of lung cancer (see Health impact of radon in Europe). However, for this specific stressor, it is also possible to calculate attributable cases using a unit risk. The UR model is presented as part of the uncertainty analyses in Uncertainties and limitations.[1]

Results by country

Using the WHO global burden of disease database (see Burden of disease, health and population data), we have estimated for each country what the fraction of the total burden of disease is that can be attributed to the nine environmental stressors considered in EBoDE. The results are shown in Figure Environmental fraction attributable. This environmental fraction ranges from 3% in Finland to 6.5% in Italy. Relatively, Finland has the largest BoD and the smallest EBD, while for Italy this is vice versa. [1]

Environmental fraction attributable to the studied nine stressors (in blue) of the total burden of disease in the participating countries (discounted age-weighted DALYs per million people).

The results for individual countries are presented in appendix A. These can be used to look at the environmental burden of disease from a country perspective, and to see which factors contribute substantially in specific countries, relative to their contribution in other countries. When comparing the relative contribution of all stressors per country, radon has the highest relative contribution in Finland. Relatively, the disease burden in France has the lowest contribution from PM. SHS comes only on the fifth place in France, whereas in other countries SHS is usually in the top of the contributors. On the other hand, traffic noise relatively has the largest contribution in France, due to the fact that only data from agglomerations are included in our estimates for France. Relatively, the disease burden in Italy has the highest contribution of ozone. Italy is also the only participating country where the impact of lead exceeds that of ozone. Formaldehyde did not exceed the threshold in Italy, resulting in zero estimates. Also in the Netherlands, formaldehyde exposures did not exceed the threshold, resulting in zero estimates. In Belgium, the impact of air pollution by particulate matter is relatively largest. Flanders is the Western European hot spot for PM pollution (IIASA, 2004). This situation is caused by a high population density, an intense industrial activity and a large volume of transit traffic linked to important harbors. The daily PM10 standard, enforced by the European Commission (1999/30/EC) is still being exceeded more times than allowed in Flanders.

Trends and policy implications

The main results of EBoDE are calculated for the year 2004 (2005 for PM and ozone). This was one of the latest years for which exposure and health data were still relatively completely available. The original objective of the project was set as estimation of the environmental burden of disease also for the present situation, year 2010. However, due to difficulties in collecting national data, reliable trend analysis for all the stressors in all the participating countries proved to be too challenging for the given time and resources. For many of the stressors sufficient data for reliable trend analysis was not available (e.g. formaldehyde, lead) or the variability was too large for identifying trends with statistical significance (e.g. ozone, dioxins). However, in order to gain insight into the use of the EBD assessment methodology and the potential use of burden of disease estimates in developing and evaluating environmental policies, we also estimated exposure trends for the year 2010 (Table 4-4). Country specific factors affecting the trends were not evaluated due to the lack of data and statistical difficulties in the trend estimation. Only for second-hand smoke exposures, national trends were derived separately (see Table 4-5 and Figure 3-2).

Crude estimated exposure trends from 2004 to 2010.

Stressor Estimated trend from 2004-2010 Remarks
Benzene -2% per year Model based on trends in outdoor data. Decreasing indoor smoking may also lead to lowering exposures from SHS
Dioxin no trend Large variability and limited data availability hinder identification of a reliable trend
Second-hand smoke -4% per year (see country data below) Power model based on numerous data points from national and international surveys conducted between 1990 and 2008.
Formaldehyde no trend Large variability and limited data availability hinder identification of a reliable trend
Lead no trend Trend could also be slightly lowering.
Noise no trend Trend could also be slightly increasing.
Ozone no trend Increasing trends may occur in rural areas. Large year-to-year variation
PM2.5 -2% per year AirBase analysis and recommendation by ETC/de Leeuw (personal communication)
Radon no trend Changes in building stock and construction structures extremely slow

Estimated exposures for 2010 for second-hand smoke in children and non-smoking adults.

Year 2010 Children Adults Women Man
Lower* [%] Upper* [%] Lower [%] Upper [%] Lower [%] Upper [%] Lower [%] Upper [%]
Belgium NA NA 25 30 24 29 25 31
Finland 3 NA 14 14 14 14 14 14
France 17 27 13 22 15 25 11 19
Germany 20 NA 20 28 19 27 21 29
Italy 29 NA 22 26 19 23 24 28
Netherlands 14 28 18 27 16 23 21 31

NA: Adequate data not available

  • Lower and upper estimates correspond to different computations of survey data.


  1. 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 1.14 Otto Hänninen, Anne Knol: European Perspectives on Environmental Burden of Disease: Esimates for Nine Stressors in Six European Countries, Authors and National Institute for Health and Welfare (THL), Report 1/2011 [1] [2] Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content Cite error: Invalid <ref> tag; name "EBoDe" defined multiple times with different content