Help:Practical guide on stakeholder involvement

PLEASE NOTE: This page is under construction.

PLEASE NOTE: This page is the revised Practical Guide on Stakeholder Involvement. For the old version click here.

Author(s) of this page:
Sjuurd

... This Practical Guide on Stakeholder Involvement consists of two parts. The first part deals with general issues in preparing for stakeholder involvement. It addresses stakeholder identification, stakeholers´ possible contributions to the integrated risk assessment, barriers to involvement and other organisational issues in stakeholder involvment preparation, and concludes with organisational forms of stakeholder involvement in integrated riks assessment. The second part focusses on stakeholder involvement issues in a face-to-face participationary setting. ...

PREPARING FOR STAKEHOLDER INVOLVEMENT

Why should you involve stakeholders? (1)

"History shows us that the common man is a better judge of his own needs in the long run than any cult of experts." (L. Gulick, 1937)

The question that precedes the practical issues of stakeholder involvement is why should you involve stakeholders? The answer reflects in all stakeholder involvement issues; it determines the openness of the integrated risk assessment, the organisation of stakeholder involvement, and the type of stakeholders you invite. It is therefore important that you make explicit your reasons to involve stakeholders.

Reasons for stakeholder involvement

Fiorino (1990) grouped the arguments for stakeholder involvement in to three categories; he distinguishes substantive, normative and instrumental arguments for stakeholder involvement.

Substantive arguments are practical arguments, like: Non-experts see problems, issues, and solutions that experts miss. (Isacson, 1986) More inclusive procedures enrich the generation of options and perspectives, and are therefore more responsive to the complexity, uncertainty and ambiguity of the risk phenomena (International Risk Governance Council 2006) And a more intensive stakeholder processes tends to result in higher-quality decisions. (Beierle, 2002)

Stakeholder involvement because stakeholder involvement is the norm and thus expected is a normative argument. The instrumental argument states that stakeholder involvement may increase the acceptance and usability of the integrated risk assessment process and outcome. (INTARESE, 2007b) To wit, stakeholder involvement can increase public trust in research and government institutions and possibly decrease conflict.

In addition to Fiorino’s substantive, normative and instrumental arguments, a fourth group of stakeholder involvement reasons can be distinguished, namely ethical reasons. Stakeholder participation is a mean to manage the legitimacy problem, which is caused by uncertainty about the evidence of harm. And owing to the Liberal foundation of the regulatory system, evidence of harm is key to justifying regulatory interventions. (INTARESE, 2006) Another ethical reason to involve stakeholders is that government should obtain the consent of the governed. (Stern & Fineberg, 1996)

Who are the stakeholders? (2)

A stakeholder is a person or organisation that has (may have) some interest related to the issues of the integrated risk assessment.

Stakeholder involvement starts with stakeholder identification and selection. A stakeholder is a person or organisation that has (may have) some interest (i.e. stake) related to the issues of the integrated risk assessment. In practice, you can think of stakeholder groups in different ways:

Thinking of stakeholder in different ways:
i. You can identify stakeholders based on their relation to the risk (Briggs & Stern, in press).	ii. You can identify stakeholders based on their roles in the integrated risk assessment.	iii. You can identify stakeholders based on the different perspectives they represent (e.g. Hage & Leroy 2007; Kloprogge & Van der Sluijs 2006).
Think of: People and organisations who (possibly) create the risks (risk generators). People or organisations who transmit the risks to other persons and/or organisations (risk purveyor). People or organisations who are effected by the risks (victims). People or organisations who benefit from the risks (risk beneficiaries). For example, companies who financially gain from the risks, because the risk (negative externality) is not included in the price of their products. Agents that hold the formal responsibilities to anticipate, reduce or manage risks, for example emergency services, planners, and regulators. Agents that hold the formal responsibility to deal with the consequences of risks, for example health services, insurance companies, and employers. People, organisations and institutions that inform about the risks (informants), for example, the media, scientists and monitoring agencies.	Think of: People, organisations and institutions that initiate the integrated risk assessment (initiators), for example policy makers, ministry, parliament, experts, research institutes, universities, industry, etc. People and organisations who perform the dose-response assessment, the exposure assessment, and the risk characterisation (assessment executors). People and organisations who can provide data, specific information, inside information, critical views, etc. to the integrated risk assessment (input providers and issue-framers), for example pressure groups, local residents, industry, research institutes, monitoring agencies, etc. People and organisations who can provide feed-back on the integrated risk assessment issues, process and outcomes (critics and users), for example, exposed local residents, experts, policy makers, journalist, industry, etc. People and organisations who can facilitate the integrated risk assessment by providing resources, such as money, researchers, laboratories, and meeting rooms and coffee (facilitators), for example research institutes, agencies, and governments. People and organisations who can obstruct the success of the integrated risk assessment (obstructers).

Tools for identifying stakeholders:
Brain Storming	Mapping	Snowball Method	Argumentation Analysis
In a brainstorm session you try to answer the lead-question: who are the stakeholders in this integrated risk assessment? All suggestions are written down. When no one can come up with unidentified stakeholders anymore, the list can be discussed.	With the mapping technique, stakeholders are allocated to a group, for example stakeholders are grouped per role or risk relation.	The snowball method is very useful to identify unrecognised stakeholders. You ask each stakeholder who they consider to be a stakeholder in integrated risk assessment. You continue doing this, until no new names pop up.	You can use the argumentation analysis, when you identify stakeholders based on the different perspectives. You list all possible perspectives on the issues of the integrated risk assessment and subsequently search for persons or organisational representatives who have such a view. It is helpful to scan the media or to use a top-down approach for perspectives on the issues of integrated risk assessment. With a top-down approach, you derive from story lines or general cultural theories the possible perspectives. (Kloprogge & Van der Sluijs, 2006)

It can be difficult to identify the interested and affected parties, when there is no clear picture of integrated risk assessment issues. The best solution to this problem is to identify representatives of the more general public and/or environmental or community groups. Do ask yourself however: How representative are the stakeholder representatives?

How can stakeholders be of service in the phases/components of the integrated risk assessment? (3)

Most likely, you want stakeholder input (e.g. their views, information, data, etc.) or their commitments. However, it could also be that you want stakeholder protection or money or you just want them to be satisfied and silent.

Stakeholders can make different contributions in the phases/components of the integrated risk assessment. Identifying and articulating your preliminary expectations about their possible contributions (in the phases/components of the integrated risk assessment) has several benefits. Firstly, your preliminary expectations help in planning stakeholder involvement in the phases/components of the integrated risk assessment. Secondly, they guide you in selecting the organisational forms of stakeholder involvement (see 8). Thirdly, your expectations can lead you in selecting participants for in the integrated risk assessment process (see 11). Consider at least these stakeholder contributions:

• framing input: perspectives, critics, and suggestions
• assessment input: knowledge, information, and data
• comments: critics and suggestions (during the process)
• feed-back: critics and suggestions (after the process)
• facilitation: mediation skills, negotiation skills, meeting rooms, website maintenance, money, etc.

• obstruction (negative contribution): going to press - publishing critics without internal deliberation, refusal to co-operate, etc.

Possible stakeholder contributions per phase/ component of the integrated risk assessment

issue framing: defining the purpose and scope of the integrated risk assessment

Stakeholder involvement in the issue framing serves to increase support and acceptance of the integrated risk assessment process and outcomes.

If stakeholder are involved in setting the purpose and scope of the integrated risk assessment, the integrated risk assessment's relevance, usability and acceptability increase. To wit, if the problem definition of the integrated risk assessment (purpose) fits in the stakeholders' problem perceptions, the stakeholders perceive the integrated risk assessment as a relevant and useful contribution to the solution of the problem and recognise (accept) the policy measures. If you involve a plurality of perspectives (different problem perceptions), the public support for and acceptance of the integrated risk assessment process and outcome increases. In order to achieve this, a plurality of stakeholders should be able to present their views and make comments.

Illustrative examples of addressing the wrong problem (and thus not receiving public support for and acceptance of the integrated risk assessment) are the Yucca Mountain case and the Antwerp waste incinerator case. Stern & Fineberg (1996) describe the Yucca Mountain case; the assessment of nuclear waste storage risks did not address all issues, such as fairness of dumping nuclear waste in a region that does not have nuclear power plants and already hosts the national nuclear testing facility. (They scoped the problem to narrow.) Many people perceived the risk assessment irrelevant and misleading, because it addressed only part of the problem. Craye, Funtowicz & Van der Sluijs (2005) describe, in the Antwerp waste incinerator case, that residents did not accept the assessments of the waste incineration risks, partly because these assessments did not address their problems (wrong purpose).

indicator selection

Stakeholder involvement in indicator selection serves to secure that the integrated risk assessment addresses all (to the stakeholders) relevant issues.

The selected set of indicators determine the applicability and relevance of the integrated risk assessment. To optimise the applicability and relevance, all to the stakeholders relevant issues should be addressed. In order to secure this, stakeholders should be able to provide input to and comment on the indicator selection.

risk assessment

insight knowledge, specific information, data, expertise

impact valuation

value judgments

uncertainty marking

insight knowledge, specific information, expertise

appraisal

Per phase/component of the integrated risk assessment	stakeholders' possible contributions : (not exhaustive)
DEFINING the integrated risk assessments' PURPOSE and SCOPE	input: perspectives critics suggestions facilitation: mediation skills meeting rooms or website maintenance ...
INDICATOR SELECTION	perspectives input: knowledge information data comments: critics suggestions facilitation: mediation skills negotiation skills meeting rooms or website maintenance ...
RISK ASSESSMENT	perspectives input: knowledge information data comments: critics suggestions facilitation: mediation skills negotiation skills meeting rooms or website maintenance ...
IMPACT VALUATION	input: experience expertise information data comments: critics suggestions ...
UNCERTAINTY MARKING	expertise comments: critics suggestions ...
APPRAISAL	perspectives expertise information comments: critics suggestions facilitation: mediation skills negotiation skills meeting rooms or website maintenance ...

Whom should we involve for sure? (4)

Particular stakeholders can play an important role in the integrated risk assessment process, because they have the ability to obstruct or accelerate the process, they hold valuable knowledge, information or data, or they can provide resources for facilitation. Based on your preliminary expectations about the stakeholders' contributions (see 3), you can list the stakeholders you should involve to benefit from their contributions.

Try to collect a plurality of stakeholder views on the different integrated risk assessment issues. This increases the usability of the integrated risk assessment process and output, and the possibility of their acceptance.

Stakeholders that should be involved for sure
It depends on the type of integrated risk assessment, which stakeholders you should involve for sure. No conclusive general list can be given. The involvement of a particular stakeholder can be redundant in the first integrated risk assessment, but the same stakeholder can make a major contribution to the second integrated risk assessment. You yourself should consider which stakeholders to involve for sure. You can take the following stakeholder groups in consideration. (Note that this list is not exhaustive and important stakeholders could be missing.)
risk assessors policy makers (from the ministry, municipality, etc.) experts in the issues addressed in the integrated risk assessment non-governmental organisations (NGOs), such as environmental protection organisations local residents (representatives) or community (representatives) pressure groups industry (representatives) agriculture and fishery (representatives) commerce (representatives) educational institutions medical community employer and employee unions ...

Who should not be involved? (5)

Perhaps this question strikes you as awkward, but it addresses the default assumptions of stakeholder involvement in integrated risk assessment (assumptions that influence the openness of the integrated risk assessment). The default assumption of this guide is that integrated risk assessment is in principle open; anyone who wants to be involved should be involved. This means that you should have good arguments to exclude stakeholders from involvement.

Two opposing default assumptions on stakeholder involvement in integrated risk assessment

The assumption of Closed integrated risk assessment

The classical default assumption on stakeholder involvement is that there is in principle no need to open up (i.e. involve stakeholders in) the integrated risk assessment. If stakeholders are involved in the integrated risk assessment, their involvement is of a passive nature (i.e. stakeholders do not participate in the integrated risk assessment) and the inclusion of their perspectives is based on a top-down approach (i.e. stakeholders do not articulate their own perspectives, but risk assessors derive theoretical stakeholder perspectives from the general classification of ideas). (e.g. Kloprogge & Van der Sluijs, 2006)

The assumption of Open integrated risk assessment

This default assumption on stakeholder involvement states that integrated risk assessment is open to anyone who wants to be involved. It made its debut in the nineties of the twentieth century together with the introduction of post-normal science. Stakeholder involvement springing from this default assumption is more active in nature and the inclusion of stakeholder perspectives in the integrated risk assessment is based on a bottom-up approach; stakeholders participate in the integrated risk assessment and articulate their own perspectives on the issues. (e.g. Kloprogge & Van der Sluijs, 2006; Craye, Funtowicz & Van der Sluijs, 2005; Funtowicz & Ravetz, 1993; Ravetz, 1999; Van der Sluijs, 2002) |}

What could each stakeholder expect from us? (6)

At the start of a risk assessment process, stakeholders do often not know that they have an interest in the risk assessment, let alone that they know precisely what they want from it. Stakeholders are not interested in involvement in the risk assessment, unless they see that their contribution has an influence on decision making. (e.g. Fraser, Dougill, et al., 2006; Newig, 2007) Hence, stakeholders expect that they can influence decisions, if they take the effort of contributing to the integrated risk assessment. Related to this expectation are other potential stakeholder expectations: Stakeholders could expect to maintain the status quo; they expect the risk assessment consequence not to touch their interest. Stakeholders could also expect to be actively involved in a particular matter. Stakeholders could want to be informed, so that they can react to any inconvenient developments. Stakeholders could expect to be heard, so that they can express their views. It is important to live up to stakeholders’ expectations for successful stakeholder involvement. Therefore, by answering the question what does each stakeholder expect from us? you identify, estimate and group stakeholders' expectations, which enables you to tailor the involvement organisation and facilitation (see 8).

What are the barriers for stakeholder involvement? (7)

"Research since the 1970's suggest that the tendency of non-state actors to participate - and thus to invest time and other resources - is primarily a function of the degree to which an actor perceives a problem to touch his own interests, combined with the degree of perceived chances to influence the output of the decision process." (J. Newig, 2007)

Asking stakeholders what refrains them from being involved and/or considering the (possible) barriers for stakeholder involvement enables you to act upon it. You can endeavour to eliminate or lower the barriers that stakeholders encounter in their involvement. Discuss (organisational) solutions for the possible barriers in an internal meeting.

Barriers to stakeholder involvement
Barriers related to problem perception and power to influence The most difficult barriers to stakeholder involvement relate to the stakeholder's problem perception (context) and power to influence. (Newig, 2007) These barriers are the most difficult to decrease, because perception is not easily changed.	Barriers related to involvement facilitation Other barriers to stakeholder involvement relate to the disorganisation, a lack of proper involvement facilitation, or a lack of resources. (U.S. EPA, 2001) Good involvement organisation and facilitation take these barriers away by taking them into account.
For example, stakeholders do not wish to be involved, because: They perceive the problem (from which the integrated risk assessment springs) as your problem and not theirs. (And they are not willing to solve your problem.) Stakeholders see not the benefits of being involved. Stakeholders mistrust (semi)governmental organisations. (And they think you are from a (semi)governmental organisation.) Stakeholders believe that authorities do not take their concerns and opinions seriously. (And they see you as authority representative.)	For example, stakeholders involvement is distorted, because: Stakeholders are unaware of the risk assessment. Stakeholders do not have enough time. Information is inadequate: chaotic, too detailed, too broad, not to point, etc. The discussions are too technical for stakeholders to be understandable. Experts use too much jargon in the discussions (and there is nobody who translates this for non-jargon speakers).

How do we involve identified stakeholders, so that they can make their contributions? (8)

There is no use of involving stakeholders, if stakeholders cannot make their contributions to the integrated risk assessment process and defend their interests. Stakeholders involvement should be organised in such a way that stakeholders can defend their interests and make their contribution (i.e. articulate their views, share their knowledge information and data, criticise, make suggestions, facilitate, etc.). Therefore also stakeholder involvement barriers should be taken into account in designing, organising and facilitating stakeholder involvement.

Mostert (2003) distinguishes six levels of stakeholder involvement with organisational forms. Consider what would be the most suitable level and form of involvement for each identified stakeholder, taking into account the stakeholders' possible contributions (see 3), expectations, and barriers to involvement.

Mostert's different levels of stakeholder involvement:	Different organisational forms of stakeholder involvement:
level 1: Stakeholders are informed	by means of: leaflets and brochures website and mailings press releases and press conferences briefings information centers ...
level 2: Stakeholders are consulted: Stakeholders are asked for their views.	by means of: questionnaire surveys telephone surveys interviews public hearings focus groups with: story telling commenting Delphi-method ...
level 3: Stakeholders participate in discussions.	Discussions can be organised as: internet forum or wiki (suitable for mass collaboration) with: argumentation method (see: Argumentation Analysis and Contributing to a discussion) workshops with: grid method brainstorm sessions ...
level 4: Stakeholders participate in designing.	Designing (models, causal chains, variables, solutions, etc.) can be organised as: a meeting with: scenario-building method group-model-building method role playing and bricolage ...
level 5 & 6: Stakeholders have a say in decision making.	Decision-making can be organised as: referendum voting of representatives negotiations for agreement

How do we provide unidentified stakeholders a possibility and opportunity to become involved? (9)

Also unidentified stakeholders should have the possibility and opportunity to become involved in the integrated risk assessment. Unidentified stakeholders are often forgotten stakeholders. If they represent yet unrepresented perspectives, they might have good comments and raise issues that have been overlooked by the other stakeholders. Therefore you should provide them the possibility and opportunity to make these comments and raise these issues.

Organisational forms to involve unidentified stakeholders:
Levels of stakeholder involvement:	Organisational forms of stakeholder involvement:
level 1: Stakeholders are informed.	leaflets and brochures website press releases and press conferences information centers ... And if obted for a higher level of stakeholder involvement, you can draw their attention to the
level 2: Stakeholders are consulted.	questionnaires (available on the website, or at the information centers) public hearings (state time and location) ...
level 3: Stakeholders participate in discussions.	internet forum or wiki (suitable for mass collaboration) with: argumentation method (see: Argumentation Analysis and Contributing to a discussion) or inform them about the time and location of the workshops with: grid method brainstorm sessions ...
level 4: Stakeholders participate in designing.	meetings with: scenario-building method group-model-building method role playing and bricolage ...
level 5 & 6: Stakeholders have a say in decision making. (After notification of their existence)	not applicable

How to inform stakeholders (10)

"Communicating such information that it is understandable to your stakeholders is an important first (and ongoing) step in stakeholder engagement. All other activities, from consultation and informed participation to negotiation and resolution of grievance, will be more constructive if stakeholders (…) have accurate and timely information about the project, its (possible) impacts, and other aspects that may have an effect on them." (International Finance Corporation 2007 p.27)

There is a whole range of possible media to inform stakeholders, e.g. leaflets, brochures, website, mailings, press releases and press conferences, briefings, information centers, etc. (See table 8 under level 1.)

When you inform stakholders there are some important things to think of:

• Explain why you think that their (stakeholders’) involvement is important.
• Explain what a risk assessment is, so that those stakeholders who have never heard of risk assessment understand it.
• Explain the purpose of the risk assessment. Why do policy makers need information about these (un)certain risks?

IMPORTANT ISSUES IN STAKEHOLDER PARTICIPATION

The more interactive form of stakeholder involvement in integrated risk assessment is stakeholder participation (levels 2 - 4, see table 8). This second part addresses issues that you would not directly think of when organising stakeholder participation in the integrated risk assessment. ...

Whom do you invite for face-to-face participation? (11)

In case you want stakeholder to participate in the integrated risk assessment on a face-to-face basis, you face a selection of participants. There are important things to think of, when selecting stakeholders for participation:

• The groups of participants should be a representative selection of the stakeholder perspectives.
• Consider the possible contributions of the stakeholders. ... Which contributions can best be made in a face-to-face meeting?
• Consider the communicative and social skills of the participants.

How to facilitate trust building? (12)

Trust is a crucial component of any stakeholder involvement initiative. Trust ensures an effective working relationship. (U.S. EPA, 2001)

If people trust each other, they are more willing to collaborate (Huxham 1996; Lubell 2000; Pretty 1995). Trust and participants’ trustworthiness are created, when stakeholders learn to know each other: what they do, how reliable and consistent they are, how they react, whom they represent, how they present themselves, etc. If people learn to know each other, they can assess the others' trustworthiness by estimating their integrity, benevolence and ability to contribute. (Mayer, Davis & Schoorman, 1995) In addition, when stakeholders know each other, they are more willing to listen to each other (which is a requisite for agreement).

In case of face-to-face communication, a pre-meeting introduction activity (dinner, lunch or a social drink) is a perfect occasion for participants to learn to know each other and thus to build their trust. It is also the opportunity to enthuse stakeholders for involvement in the risk assessment and to verify your estimations about their expectations. "There is a reason that lobbyists and politicians do much work at receptions and lunches." (Innes & Booher 1999 p.19)

In case of non-face-to-face communication (questionnaire surveys, telephone surveys, brochures, internet fora, wikis, etc.) trust creation is more difficult, because of the remote relationship between participants. Building trust demands more time and is almost solely dependent on the participants honouring their promises and living up to the others' expectations, for example securing privacy or delivering qualitative work on time. (e.g. Aubert & Kelsey, 2003)

In any case, you can facilitate trust building among participants by admonishing them to be integer and to account for other opinions when working on the integrated risk assessment.

Do all stakeholders acknowledge each other's roles in the integrated risk assessment? (13)

It is important that all stakeholders accept each other’s role in (the execution of ) the integrated risk assessment (that includes also the role of the risk assessors). Without mutual recognition of each other’s roles, conflict arises which has a negative influence on the risk assessment quality (De Dreu & Weingart 2003). If participants in the risk assessment do not recognise each other’s roles, authorities, responsibilities and coordinative lines remain vague. Conflict might arise continuously, delaying the project’s progress, or worse, making collaboration impossible. Mediating between stakeholders to establish mutual recognition of roles is an important stakeholder management issue.

The benefit of mass collaboration

Mass collaboration reduces the problems springing from the default of mutual role recognition. If the integrated risk assessment is organised by the ideas of mass collaboration, discussions and parts of the assessment output production take place on internet fora or in wikis and participating stakeholders work together on a shared set of tasks. (e.g. INTARESE 2007a) The stakeholder collective shares responsibilities, which implies the absence of coordinative lines. This semistructured anarchical organisation of the integrated risk assessment avoids the dilemma of role recognition.

How to deal with conflict?

Likely bringing together different views generates conflict. Some conflicts must be settled in order to procede in the integrated risk assessment process and yield effective risk assessment output. According to Rahim (1983, 1992) and Thomas & Kilmann (1974) disputes can be settled or resolved in five different ways. You can settle a dispute by avoindance (you ignore the dispute), accommodation (you subordinate your view/opinion to those of others), or by competition (you superordinate your view/opinion to those of others). Or you can resolve a dispute by means of compromise (you combine your view/opinion with those of others) or by means of collaboration (you and the others integrate your views/opinions in a common standpoint by means of argumentation)

Achieving a compromise among the participants who all hold different opinions is a tough job and requires political skills. Unfortunately no one-fits-all method for establishing agreement can be given. Questions that can be helplful in guiding you to establish a compromise are:

• Which standpoints can be grouped and/or combined?
• Which standpoints lead to the same conclusion/decision?
• How can participants' objections be taken away?

Argumentation

Disputes among participants can be settled is many ways, but only logic argumentation can truly resolve their disagreement. Participants should be encouraged to resolve their disputes by means of argumentation. In the argument, the disputants try to convince the opponents of their staindpoint by expounding their reasoning and attacking the reasoning of their opponents. The explanation of reasoning shows the (underlying) cause of the dispute and (hopefully) converge the disputing parties' views to a common new view.

Argumentation is only useful for solving disputes, when the disputants are willing to give up their original standpoint and are not carried away by their emotions (i.e. they must be willing to listen to the opponents). Furthermore, simple discussion rules can guide an argumentation to resolve disputes.

References

Aubert, Benoit A. & Barbara L. Kelsey (2003) - Further understanding of trust and performance in virtual teams. - in Small Group Research 34(5) pp.575-618

Beierle, Thomas C. (2002) - The quality of stakeholder-based decisions. - in Risk Analysis 22(4) pp.739-749

Briggs, David & Richard Stern (in press) - Risk response to the environmental hazards to health: Towards an ecological approach. -

Craye, Matthieu & Silvio O. Funtowicz (2005) - A reflexive approach to dealing with uncertainties in environmental health risk science and policy. - in International Journal of Risk Assessment and Management 5(2/3/4) pp.216-236

Dreu, Carsten de & Laurie R. Weingart (2003) - Task versus relationship conflict, team performance, and team member satisfaction: A meta-analysis. - in Journal of Applied Psychology 88(4) pp.741-749

Fiorino, Daniel J. (1990) - Citizen participation and environmental risk: A survey of institutional mechanisms. - in Science, Technology and Human Values 15(.) pp.226-243

Fraser, Evan D.G., Andrew J. Dougill, Warren E. Mabee, Mark Reed & Patrick McAlpine (2006) - Bottom up and top down: Analysis of participatory processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental management. - in Journal of Environmental Management 78(2) pp.114-127

Funtowicz, Silvio O. & Jerome R. Ravetz (1993) - Science for the post-normal age. - in Futures 25(September) pp.739-755

Gulick, L. (1937) - Notes on the theory of organization. - in Papers on the Science of Administration, L.Gulick & L. Urwick (Eds.) New York: Institute of Public Administration

Hage, Maria & Peter Leroy (2007) - Leidraad stakeholderparticipatie voor het Milieu- en Natuurplanbureau: Praktijkwijzer. http://www.mnp.nl/nl/publicaties

Huxham, Chris (Ed.) (1996) - Creating Collaborative Advantage. London: Sage

INTARESE (2006) - Deliverable 7: Uncertainty report.

INTARESE (2007a) - Deliverable 16: Risk characterisation protocol.

INTARESE (2007b) - Deliverable 17: Risk characterisation methodology report.

Innes, Judith E. & David E. Booher (1999) - Consensus building as a role playing and bricolage. - in Journal of the American Planning Association 65(1) pp.9-26

International Finance Corporation(2007) - Stakeholder Engagement: A good practice handbook for companies doing business in emerging markets.

International Risk Governance Council (2006) - White paper No.1: Risk governance; Toward an integrative approach. Geneva: IRGC

Isacson, Peter (1986) - Pollutant regulation and public sensibility. - in Environmental Impact Assessment Review 6(3) pp.229-232

Kloprogge, Penny & Jeroen P. van der Sluijs (2006) - The inclusion of stakeholder knowledge and perspectives in integrated assessment of climate change. - in Climatic Change 75(.) pp.359-389

Lubell, Mark (2000) - Cognitive conflict and consensus building in the National Estuary Program.- in American Behavioral Scientist .(4) pp.629-648

Mayer, R.C., J.H. Davis & F.D. Schoorman (1995) - An integrative model of organisational trust. - in Academy of Management Review 20(.) pp.709-734

Mostert, Erik (2003) - The challenge of public participation. - in Water Policy 5(2) pp.179-197

Newig, Jens (2007) - Does public participation in environmental decisions lead to improved environmental quality? - in Communication, Cooperation and Participation 1(April) pp.51-71

Pretty, Jules N. (2007) - Participatory learning for suitable agriculture. - in World Development 23(8) pp.1247-1263

Ravetz, Jerome R. (1999) - What is post-normal science. - in Futures 31(.) pp.647-653

Sluijs, Jeroen P. van der (2002) - A way out of the credibility crisis of models used in integrated environmental assessment. - in Futures 32(.) pp.133-146

Stern, Paul C. & Harvey V. Fineberg (Eds.) (1996) - Understanding Risk: Informing Decisions in a Democratic Society. Washington D.C.: National Academy Press

United States Environmental Protection Agency (2001) - Stakeholder Involvement & Public Participation at the U.S. EPA: Lessons learned, barriers, and innovative approaches.