Uncertainty analysis: agriculture

The text on this page is taken from an equivalent page of the IEHIAS-project.

As part of the EU-funded INTARESE project, which contribued to the development of this Toolbox, a case study was carried out to assess the health impacts of agricultural land use change in England and Greece.

Recognition of the uncertainties in an integrated assessment are an important part of the process, even if they can't be quantified. Here we describe the qualitative uncertainty approach and our findings for our case studies.

Approach

Uncertainty in the agriculture case study is organized in two steps:

1. Identification of all uncertainty sources, grouped according to the following classification:

   Scenario uncertainty refers to the description of the context (scenario setting) as a prerequisite for either modelling or measuring experimental data. It includes descriptive errors, aggregation errors, errors in selection of the assessment tier and errors due to incomplete analysis. It often includes the purpose of the environmental health impact assessment and consistency between the scenario definition and the scope and purpose of the assessment.

   Model uncertainty reflects the limited ability of mathematical models to represent the real world accurately and may also reflect lack of sufficient knowledge. It is principally associated to model boundaries, extrapolation limits, modelling errors and correlation (dependency) errors. It also includes errors due to the implementation of tools and software.

   Parameter uncertainty refers to data values that are not known with precision due to measurement error or limited observations (sampling error). Sometimes it consists of variability as an inherent property of the heterogeneity or diversity in the parameter, such as parameters expressed as a function of the entire population. Usually, variability cannot be reducible through further investigation. It is also possible for the uncertainty and variability of parameters to be combined.

2. Qualitative/semi-quantitative characterization of uncertainty sources in three dimensions:

   a) direction of the influence of the uncertainty source on the results

   O/U: Over/Under (denoting the direction of the influence of the specific uncertainty source on the output of the estimation)

   b) level of uncertainty

   L/M/H: Low/Medium/High (denoting the level of the influence of the specific uncertainty source to the output of the estimation)

   c) appraisal of knowledge base of the uncertainty source

   L/M/H: Low/Medium/High (denoting the scientific consistency of the knowledge base underlying the assessment)

Results

The uncertainties identified in the Greece and England case studies are described in turn, followed by a brief synopsis of the similarities between case studies (step 1). The uncertainty matrix (step 2) for the two case studies is attached at the bottom of this page.

1.1 Identification of uncertainty sources in case study for Greece

Source - Exposure

• The spatial resolution of the analysis (4x4km grid) is finer than that of available key data (i.e. pesticides sales, ATEAM scenario maps, population), which introduces parameter uncertainties :
  • Baseline crop data from LAU-2 level (ESYE estimates) to 4x4km grid is accomplished via area weighting. This method introduces parameter uncertainty, due to the limitations of the algorithm and the lack of any surrogate (auxiliary data).
  • Main crop projections for the scenario years introduces model uncertainty, due to the lack of information on crop variations from the ATEAM model (focuses only on land use).
  • Energy crop projections for the scenario years, introduce parametric uncertainty. The ATEAM (16x16km) makes projections for the energy crops; any uncertainty introduced is from the spatial distribution of energy crops (ATEAM) and the change in resolution.
  • Population data, are available from ESYE are at LAU-2 level and disaggregated to a high resolution grid.
• Pesticide type and usage rates for various crops were estimated mainly by collecting sales data and soliciting expert advice. Thus, there are model uncertainties regarding total pesticide quantity, the computed pesticide applications (mainly the rates) as compared to the actual ones. It has to be noted, however, that the input data thus derived were deemed as being much closer to real application rate.
• The pesticides AS employed in the years 2020 and 2050 scenarios are associated with significant model uncertainty, despite efforts made to prepare a realistic list by excluding the already, or soon to be, withdrawn AS, and to include appropriate AS –approved or pending approval- for each major crop. Means to reduce this uncertainty include direct contact with pesticide manufacturers and importers and consultation with the relevant competent authorities at the national and European level (Ministry of Agriculture, European Commission – DG Environment, DG Entreprise).
• The dispersion calculations introduce parameter uncertainty due to:
  • Using the box-volume model, the wind speed and mixing height were calculated using the CALMET model. Due to the limited number of meteorological stations and data in the area and the method used to interpolate.
  • Use of the focal sum model, introduces parameter uncertainty due to assumption made that all meteorological conditions are similar.
• The Emission factors (EF) were derived from pesticides usage data from the Netherlands. As a result, significant parameter uncertainty is introduced in the study. Moreover, EFs for AS that were not available from Netherlands data were derived by interpolating on the basis of vapour pressures of other similar AS.
• It should be stressed that PM Emission factors refer to whole Greece and they are not representative of local conditions, this introduces parameter uncertainty.
• Emission factors for pollen. Since no data for pollen emission factors were available in literature, a methodology was proposed for the estimation. This methodology is based on information on release rates of pollen which vary depending on local conditions, introducing model uncertainty in calculations.

Exposure – health effects

• Modelled exposure is another source of parameter uncertainty:
  • For instance, an average AI ambient air concentration is estimated from the annual AI usage using the box volume model, without considering the physical properties of the different AI and local meteorological conditions (wind speed/direction and mixing height). Thus, computed human intake, by assuming a fixed period of exposure to this concentration, for all AS, involves significant uncertainty; for example, AI physical properties (e.g. volatility, half life) differ significantly, and it is uncertain to what extent these assumptions represents reality.
  • The assumed uniform exposure (for each grid cell and averaged year) of the entire population, in the area considered (4x4km grid) introduces scenario uncertainty, since each with this method we are neglecting individual exposure patterns.
  • Assessment of health impact is affected by the aforementioned scenario uncertainties related to human intake by inhalation (in various population groups), and the deficiencies caused by neglecting significant exposure pathways as well as the effect of population behavioural patterns.
• PM Exposure response functions: PM ERFs estimations have been based on average measures of PM concentrations. Average concentrations are measured in large cities and are an important source of error. Moreover, extrapolating ERFs that have been derived from a particular population to other populations for impact assessment may reduce the validity of results as many factors, including climatic conditions, age, different lifestyles, housing etc., produce bias.
• The fact that endotoxin exposure response functions have been derived from a small population size (i.e. only few farmers) creates difficulties in extrapolating these functions in whole rural population. Different climatic conditions and animal husbandry practices introduce a kind of uncertainty. Furthermore, as the amount and duration of exposure play an important role in protection against asthma these ERFs may not be appropriate for every cases. Another point that should be mentioned is that the majority of studies assess exposure either to first years of life or to a particular occupation.
• A major source of uncertainty in this assessment method is related to the application of toxicological data:
  • The limitations and uncertainties are well-known; i.e. extrapolation of dose response functions from animals to humans and from large to small doses, experimental conditions in toxicological studies that do not resemble actual conditions of human exposure to pollutants, etc.
• For carcinogenic health outcomes, the assumed life-time exposure to an estimated constant concentration is another source of model uncertainty. In this case, the estimation is biased towards the more conservative side, in order to make sure that latent effects can be appropriately captured by the analysis.

1.2 Identification of uncertainty sources in case study for England

Source - Exposure

• There is parameter uncertainty in some of the raw data sets used for this assessment and, in most instances simple methods were used to address these data deficiencies.
  • The LAU2 agricultural census data (i.e. June agricultural returns (JAR)) contains data gaps due to data suppression to protect anonymity of small holdings. For example, if few farms are within a particular LAU2 area, information for those farms will only be displayed at the district level. If there are too few at the district, they will be recorded in the county total. Thus gaps in crop area and livestock counts were filled using an iterative area-weighting process from the next highest known level aggregation. While the total area of crop (or number livestock) within counties was maintained, the allocation of crop/livestock to specific LAU2 areas known to be suppressed is subject to error which cannot be quantified.
  • Given the large number of pesticide active substances in GB (ca. 350), the pesticide usage (PU) data were broadly aggregated into herbicides, insecticides and fungicides. As this grouping is based only on pesticide function, and the pesticides within each of these broad groups do differ in their toxicity, this aggregation will have introduced uncertainty into the overall toxic effects of these groups.
  • There is also parameter in the PU survey data, in that no distinction between missing and no data is available for the county-level estimates. The methodology for the survey requires visiting a statistically valid number of farms at a regional level. As a result not every county may have a representative farm visited. This does not mean, however, that a particular crop is not grown in that county or that certain pesticides normally applied to that crop were not used.
  • Corine land cover 2000 was used as the primary dataset by which to delineate agricultural areas to facilitate disaggregation of the pesticide usage and agricultural survey data to a finer resolution for modelling (250x250m). While the categories in Corine broadly distinguish between types of agriculture, it does not provide accurate field boundaries nor information on particular crops grown in each area. Uncertainties in this context may have been reduced by using higher resolution Land Cover Map for GB, however, cost prohibited its use in this assessment.
• A minimum spatial resolution of 250x250m was selected to ensure that modelling was not attempted at a resolution below that of the input data sets (i.e. 1:100,000 vector Corine has a notional accuracy of ca. 100m). All other input data, however, are at larger spatial scales including: county-level pesticide usage, LAU2 agricultural census (JAR), 5x5km REGIS scenarios. Correlation analysis was thus used to assess associations before disaggregating these data to finer spatial scales.
  • Mask area weighing was used to disaggregate the PU and JAR. This method introduces parameter uncertainty, due to the limitations of the algorithm and inaccuracy of the auxillary data used as the mask. While the overall pesticide usage at the county level is assured, error in pesticide usage assigned to each LAU2 units is expected. This error is difficult to quantify without ground truthing or validation with independent data. Crop correlations were 0.36–0.91 and 0.02-0.96 while livestock correlations ranged from 0.38-0.61 and 0.41-0.93 for East Anglia and the northwest, respectively.
• The definitions of crop categories within the various data sets (i.e. JAR, PU, and REGIS) were not exactly the same leading to scenario uncertainty. To overcome this issue, a concordance table between the JAR and PU was first generated giving 11 common categories. These were then matched with combinations of the categories in REGIS, for the purpose of computing the scenario data sets, on the basis of correlation analysis. Correlations ranged from 0.19-0.58 and 0.30-0.08 for crops in East Anglia and the Northwest, respectively. Correlations for livestock for both study areas were 0.29-0.76.
• It is assumed that the pesticides represented in the PU survey (ca. Year 2000) are representative of those used in the future. The available pesticides, however, are rapidly changing causing significant model uncertainty. Even in the ten years since this survey, ca. 45% of the active substances have been withdrawn. Those known to be withdrawn have been excluded from this assessment.
• Focalsum - A single, centrally located meteorological station was selected for each study area. This was done to simplify modelling, however, introduces potential parameter uncertainty in assuming weather conditions are consistent across each study area. Sub-study area model runs could have been done using additional meteorological stations however, sites with appropriate measurements for the full time series were difficult to obtain.
• As with the Greece case study, parameter uncertainty is introduced in the various emissions factors (EFs):
  • Pesticide EFs were derived from pesticide usage in the Netherlands. As a result, significant parameter uncertainty is introduced in the study. Moreover, EFs for AS that were not available from the Dutch study were derived by interpolating on the basis of vapour pressures of other similar AS.
  • PM EFs refer to whole country, and are not representative of local conditions. Furthermore, if no emission factors were available, EFs highly matching country-specific conditions were applied.
  • General EFs for endotoxin were derived from the literature. These were not country specific, thus do not take account of different modes of feeding and ventilation systems in animal housing.

Exposure – health effects

• As with the Greece case study, parameter uncertainty also occurs in modelling exposures
  • Although agricultural activity tends to be seasonal, concentrations were calculated as annual averages using the focalsum approach. Whilst focalsum modelling approach can allow for sub-annual modelling, simply by changing the timeframe of the input meteorological data, annual modelling was preferred to correspond to the emission factors calculated on a yearly basis.
  • Furthermore, concentration is used as a proxy for exposure in computing attributable burden.
  • Assigning exposure via postcode location assumes individuals remain within the 250x250m grid cell in which their residence is located. This is an unrealistic assumption, however, time-activity was outside the scope of this assessment.
• Exposure misclassification is another major source of parameter uncertainty in this assessment.
  • This assessment was undertaken at the LAU2 (or higher) level, in which it was assumed that all persons in a particular LAU2 area have the same level of exposure. This is an assumption as we expect significant within-LAU2 area variation in the various agricultural exposures, as indicated by the varying land use within LAU2 areas apparent in Corine land cover. To take account of this issue, postcodes were used to compute population weighted exposures at the LAU2 level on the basis of concentrations modelled at a much finer resolution (i.e. 250x250m).
  • Uniform exposure is assumed for each 250x250m grid cell in the GB study areas. Population data attached to postcode point locations were then used to compute weighted exposures for different geographies (e.g. population weighted LAU2 exposure). Postcodes, however, typically represent 15 households (but can reach up to 100 in some cases) and range in size between urban and rural areas. In some rural areas, therefore, many postcodes in rural areas are likely greater than 250x250m leading to potential exposure misclassification in computing population weighted exposures at higher levels of aggregation.
• As with the Greece case study, there is uncertainty in the ERFs that were selected:
  • Most epidemiological studies related to pesticide exposure are based on specific occupational groups, where the exposure is expected to be higher than for bystanders. Existing meta-analyses for relevant pesticide groups and health outcomes were difficult to find. A literature review was thus undertaken, though our own meta-analysis was not possible, given the wide range of pesticide AIs (often focused on AI no longer in use), particular occupational groups, and specific disease outcomes. Several existing systematic reviews were used as a basis to derive hypothetical relative risks for exposure categories (e.g. non-exposed, exposed and/or non-exposed, low, medium, high) to represent indicative ERFs for the broad pesticide groups. Exposure misclassification was also noted in this stage of the assessment, as parameter uncertainty, where sensitivity analysis illustrated that the definition of exposure categories had a large effect on the computation of attributable burden.
  • In the absence of specific ERFs for agricultural, existing ERFs for traffic-related PM10 and PM2.5 for all-cause mortality (available within the toolkit) were used. This is a likely source of parameter uncertainty.
• Exposures were computed for a populations projected to the year 2031 which approximately corresponds to the timeframe for the RegIS 2020 scenario (i.e. representative of the time period 2011 – 2040). Official trend-based population projections (including assumptions about births, deaths and migration) were taken from the UK Office for National Statistics. A simple linear model of the county change rates was applied to compute LAU2 (and greater) level estimates for age/sex strata. These projected population data potentially include scenario, model and parameter uncertainty.
• For cancers, national background rates of disease were used rather than regional or county level estimates.

1.3 Similarities in uncertainty for both Case Studies

On the basis of the results from the two case studies for pesticides, the following brief comments may be made regarding various aspects of the methodology:

Selected Scenario. There are certainly many driving forces expected to shape the future in agricultural land use; thus, developing credible scenarios is very complicated. Nevertheless, it appears that for the type of environmental health impact assessment considered in this study, an appropriate scenario should possess the following two main attributes: a) It should involve, or allow the introduction of, policy issues, relevant to agriculture, preferably in a transparent way. This kind of transparency and flexibility would facilitate evaluation of policy alternatives. b) It should be characterised by a sufficient level of detail, thus, requiring a minimum of labour (for enhancement) to adapt it to the needs of the health impact assessment.

Pesticide data. Various types of problems have been identified in relation to pesticide data. In addition to lack of reliable data for most of the European countries, there are problems of data spatial resolution and specificity (in relation to crops) even in countries where records are kept (e.g. GB). Other problems due to the large number of AI in the market and the incomplete characterisation of their various properties (physical, chemical, toxicological, etc) are outlined above. It is important to stress here the difficulties created in the present study by the ever changing (especially after year 2000) list of AS approved by the EC, as a result of the PPP Directive implementation. This variability, very difficult to predict in the long run, tends to introduce a great uncertainty, and constitutes an additional factor to be considered in this assessment.

Dose response functions. As outlined above, this is a critical issue for the study at hand. Relevant data from epidemiological studies are very limited, and toxicological dose response functions, although available for many (but not all) AS, are characterised by significant limitations and uncertainties.

Scale of analysis. This study is carried out at different resolutions in England and Greece respectively. It appears that selection of different resolutions does not affect the health impact estimates.

2. Qualitative/semi-quantitative characterization

The combined uncertainty matrix can be viewed and downloaded: File:Agriculture uncertainty matrix.pdf