Participating in assessments

From Opasnet
Jump to navigation Jump to search


Assessments and participation

Integrated environmental health impact assessment (IEHIA) is an endeavour of analysing relations between environmental phenomena and human health for the purpose of informing decision making upon actions to reduce adverse health effects and enhance beneficial effects. IEHIA is by its nature a multi-disciplinary endeavor that addresses issues of potential interest to a great number of people with different kinds of perspectives; scientific experts appointed to the task and policy makers with an obligation to deal with the issue, but also e.g. representatives of industry and commerce, NGO's as well as the general public. IEHIA can be perceived as a science-based activity taking place on the interface between science and society aiming for not only good governance, but also enlightenment and awareness among plural groups and members of the society at large.

For some time already, scientific literature on fields relevant to IEHIA, e.g. health impact assessment (HIA), environmental impact assessment (EIA), risk assessment (RA) and integrated assessment (IA), has been emphasizing the importance of interaction with stakeholders and the public. Partially this is a consequence of the developments in participatory governance, e.g. The Aarhus Convention aiming at access to information, public participation in decision-making, and access to justice in environmental matters [1]. Partially the incentive for participatory assessment processes has also arisen from the needs of assessors to improve the effectiveness of their assessments, e.g. in terms of relevance, acceptability and information quality. However, far too often the assessment practices still remain in the form of closed processes, where contacts with users are reduced to minimum, collaboration within the community of experts is rare, and participation is allowed only when obligated by regulation. Even in processes where participation has become common practice, it is often seen more as an additional burden than a substantive part of the assessment or decision making process [2] [3]. The old tradition of separating science-based knowledge creation from its use still lives strong in the assessment communities.

At the same time the nearly ubiquitous web-access at many parts of the world as well as development of collaborative software, web 2.0, and methods and practices for collective knowledge creation have made it possible to open up knowledge-intensive processes that were traditionally perceived as necessarily closed. Linux, Human Genome Project, and Wikipedia are good and often referred examples of such new kinds of open collaboration processes, but many more, perhaps less extreme examples, can be found e.g. from user-lead technological product development and innovation [4]. It is hard to believe that such a transition would not be possible also in science-based policy support, including IEHIA and other forms of assessment.

Of course it must be said that many participation practices have already been developed for and widely applied in IEHIA related assessment and decision making processes. There are some problems, however. Firstly, most participatory approaches consider participation as a part of decision making, not so much as a part of producing the information basis for decisions. Secondly, the prevailing assessment processes are designed as closed processes by default and the participatory aspects are mostly considered as add-on modules that bring only local or provisional moments of openness, or semi-openness, to these processes. If the ideals of participation are to be fulfilled in IEHIA, the assessment procedures must be designed so that participation is included as an intrinsic property of the whole assessment process itself. Consequently, the participants should be brought as an essential part of the assessment process.

This perspective assimilates participation with social learning by collective knowledge creation, where the participants, together with assessors and decision makers, learn about the issues at hand by collaboratively engaging in finding answers to the questions in consideration. Participants’ learning is seen as an important outcome of the assessment in addition to the primary objectives of producing an assessment report and influencing the decision where the information is used. This is in accordance with the direct participation theory of democracy [5]. Collective knowledge creation among decision makers, assessors and other participants, is also a major vehicle in improving the outcomes in terms of the primary objectives, the assessment product and its use.

Reasoning for participation

"History shows us that the common man is a better judge of his own needs in the long run than any cult of experts." [6]

The reasons for citizen participation have been identified by Fiorino [7] as substantive, normative or instrumental. Another classification of reasons for participation is given by the Economic Commission for Latin America and the Caribbean [8] as ethical, political, pragmatic or epistemological. Making a fusion of these classifications, the reasoning behind stakeholder involvement is discussed below under four headings as follows:

  1. Normative and ethical reasons
  2. Instrumental, pragmatic reasons
  3. Epistemological, substantive reasons
  4. Political reasons

Fulfilling the normative requirements and addressing ethical concerns sets the minimum level for participation in IEHIA. In many cases there are regulations concerning the rights of stakeholders or citizens to participate in the societal decision making processes and thereby to also provide their input to the assessment processes. Public hearings at different stages of environmental impact assessment processes e.g. on major energy production site plans is an example of this. From an ethical perspective the scope of participation should extend at least to those whose well-being is or might be affected. Of course determining who is or who is not affected by something can be very difficult, so this type of reasoning can only be considered as a guideline, rather than a rule. According to this reasoning, the perspective to participation is: "who and how do we have to include in the assessment?".

The instrumental, pragmatic reasons relate to improving the outcome of the assessment by means of participation. This could be accomplished e.g. by increasing the sense of ownership, trust and acceptance of the assessment among those who are invited to participate. Also improved relevance of the assessment through inclusion of multiple views and perceptions can be sought for by participation. A related example would be inviting local people nearby a planned waste incineration site to express their concerns early on in the assessment process, and address those concerns in the assessment. Although all the locals may not be happy with the outcome, they are more likely to accept it if their concerns were given a fair treatment and well-founded answers to their questions were provided within the assessment. Following this reasoning, the perspective to participation is: "whose concerns should we include in the assessment, if we want things to go smoothly?".

Probably the greatest potential of benefiting from participation lies in making use of the diverse knowledge and plurality of views among stakeholders and public. Stakeholders may possess some local or other special knowledge about the phenomena being assessed that is not held by experts or that is not available in any official databases or information sources. Perhaps even more importantly, stakeholders and citizens are the best representatives of their values and are thus a crucial source to be included in IEHIA and other assessments that often deal with controversial issues. Non-experts see problems, issues, and solutions that experts miss [9]. Participation can be seen as a means to improve the actual content of the assessment by bringing in knowledge and values of the stakeholders or public. The perspective to participation in this reasoning is: "who could know something that we would not otherwise obtain, and how to incorporate that into the assessment?".

Political reasons relate to the basic ideas of democracy. The society must have control over issues that have impacts on the well-being of of the society and its members, i.e. the government should obtain the consent of the governed [10]. Whether approaching the issue of participation from the point of view of pluralism (polyarchy, interest group liberalism) or direct participation, it is essential for the democratic system itself to incorporate possibilities for participation. Actually, this kind of reasoning forms the background for other above mentioned reasons as it points out that participation is necessary if a society were to be democratic rather than technocratic. Following this reasoning, the perspective to participation becomes: "is it fair not to let interested people have influence on the outcomes that affect them?".

All in all, the reasoning behind participation is basically quite practical: to come up with better assessments and consequently better decisions, actions and consequences. Collection and synthesis of the knowledge and views of a diverse group of people tends to lead to better outputs than just relying on the knowledge and views of only few individuals, even if they were experts. Also, inclusion of diverse groups to contribute to the work tends to increase the acceptability of the outputs and can help to improve the usability of the outputs. Even the efficiency of the assessment process can be enhanced by participation, although badly designed and managed participatory practices can also turn out counterproductive in this sense. More inclusive procedures enrich the generation of options and perspectives, and are therefore more responsive to the complexity, uncertainty and ambiguity of the risk phenomena [11] and more intensive stakeholder processes tends to result in higher-quality decisions [12].

Participation and openness

Basically, participation in IEHIA means allowing non-experts to join the information production process by sharing their knowledge, providing questions, and expressing their values. If all the reasons for participation are considered, the purpose of participation is to improve both the substance and functionality of the assessment output, not just to fulfill an obligation. Also, it should not be only a one-way process of informing participants, neither merely drawing information from participants. The overall outcome of the assessment and/or decision making process is the main goal, but also social learning, particularly in the direct participation view to democracy, is important [5]. In a dynamic two-way participatory process the participants both contribute to the assessment and learn as a part of the process. In fact many of the desired outcomes underlying the reasons for participation can be seen rather as results of societal learning than intrinsic properties of the assessment product.

In a dynamic two-way process, where participants have an active role, it is not meaningfully possible to keep the assessment process closed to experts, and perhaps decision makers, only. Thereby, the concept of openness becomes a central issue in participatory IEHIA. Openness considers the ways how interaction between assessors, decision makers, and other participants can be organized and managed. Some important aspects of openness can be identified as follows:

  • Scope of participation
  • Access to information
  • Scope of contribution
  • Impact of contribution
  • Timing of openness

Scope of participation refers to who, and on what basis, are allowed (or inversely not allowed) to participate in the assessment. Access to information refers to which parts of assessment information are set available to participants. Scope of contribution refers to which aspects of the assessment are different participants' contributions invited and allowed to. Impact of contribution refers to what extent may a particular contribution have influence on the assessment and its outcomes, i.e. how much weight is given to participants' contributions. Timing of openness refers to when, e.g. in which phases of assessment, are participants' contributions invited or allowed. The overall openness of an IEHIA process can be considered as a function of all different aspects of openness.

The degree of openness can be managed in terms of the above-mentioned aspects in relation to the purpose and goals of the assessment, taking into account the situational, contextual, and practical issues, e.g. legal requirements, public perceptions, available resources, time constraints, complexity of the case etc. The degree of openness can also be adjusted separately for different groups of participants as needed and the degree of openness may vary from assessment to another. The default in participatory IEHIA should, however, be complete openness, unless otherwise can be well argued. In open participation anyone is allowed to raise any points related to an assessment at any point during the making of the assessment. Any limitation of openness, in terms of any aspect of openness, must always be well defended. This is quite contrary to the current perception, where the default tends to be a closed process, and deviations towards openness require reasoning.

Challenges of participation

Unfortunately the benefits of participation are no achievable without some costs. Opening the assessment processes for participation brings about new challenges for IEHIA. Correspondingly, the methods and tools for IEHIA need to be designed in a way that these challenges can be overcome. Still, however, perhaps the biggest challenge lies in changing the mindset and attitudes of assessors and decision makers to perceive participation as an enhancement to assessment practices, not as a burden. Some of the most obvious challenges to IEHIA brought about by participation, along with some brief explanation, are discussed below. Some more consideration on tackling these challenges is provided also in the chapter on facilitating participation.

Quality of non-expert contributions

An often heard argument for closed assessment processes is that non-experts do not have enough knowledge to make sensible or relevant contributions. As a consequence it is considered that their involvement is either futile or even harmful. This thinking, however, highly overvalues experts' capabilities. For example, in the context of environmental health, most problems are so broad and multi-faceted, that even the greatest environmental health expert of all ends up being a non-experts in relation to many aspects of the problem. Also, IEHIA considers issues where natural and societal phenomena are multiply intertwined, and often involve aspects about which the lay-members of the society themselves are the best experts. After all, some of the main benefits that are sought for by participation do come from expanding the information input to the assessment to cover issues that could not be obtained from experts, databases, scientific journals and other common information sources of scientific studies. The assessment processes need to be designed and managed in a way that the participants have a meaningful role in the process and that their information input is dealt with in a useful way. Also the methods and tools for collecting, synthesizing and communicating assessment information must be designed so that the plurality of sources and types of information included in the assessment can be dealt with.

Inclusion of values and perceptions in a scientific assessment process, not only decision making process

Most participation approaches consider participation as a part of the decision making process, not the assessment process. Similarly, it is often considered that the values and perceptions of stakeholders or public only have a role in the decision making process, not the "scientific and objective" assessment process. However, if the input to the decision making process is desired, it should be included already in the assessment that the decision making is based on. How else could the input be systematically incorporated into the overall process of assessment and decision making? This input includes participant value expressions and perceptions as well. Although the old idealistic view of a value-free and objective scientific assessment strictly separated from the value-laden, political decision making process still lives strong in many people's heads, that view is disingenuous. The values of assessors inherently guide the assessment towards certain directions, even if they were left implicit, unanalyzed and hidden under a facade of scientific objectivity. Clearly, broadening the value base and making its analysis and impacts to the assessment explicit is a more defensible option. The participant input, including both knowledge and values, should, and can, be incorporated already into the assessment. Value statements are in many ways similar to other kinds of information, e.g. observation data, and there are no fundamental reasons why value statements could not be analyzed systematically in assessment.

Dealing with disputes

In participatory assessments on controversial and important issues it is likely that there are no obvious, definite and generally accepted answers to many questions. For example, values might be in dispute, evidence may not be decisive, or there may be lack of agreement on understanding how a problem is to be addressed. Anyhow, the probable emergence of disputes in participation should not be considered as a problem, but a possibility. Disputes tend to point out weaknesses in the assessment and addressing them in a proper manner is likely to make the assessment better. Also, disputes are useful from the social learning point of view, as they trigger discourses that can lead e.g. to broadening of perspective, correction of false assumptions, and enhanced understanding upon the assessed issues among both external participants and assessors as well as decision makers. Again, the assessment processes, methods and tools need to be designed in a way that they can accommodate and facilitate constructive deliberation upon the issues in dispute.

Protecting from intentional jamming and delaying of assessment by junk contributions

Sometimes possibilities of participation are considered as possibilities of hampering on-going processes of assessment or decision making. This could mean e.g. dissemination of disinformation, formulating repeated appeals on minor issues in a formal process, or just flooding the process with an overwhelming flow of junk contributions. This is a reasonable threat, but it is actually more relevant in cases of closed processes with no explicit or fair treatment of outsider contributions. If participation is organized so that it is open, meaningful, effective and fair, many of the reasons to take such extreme actions to hamper the process are lost. It is still necessary to guarantee that the methods of participation can deal effectively, but fairly, with irrelevant, ungrounded, or even hostile contributions.

Low willingness to participate

Often the discussions on participation consider how to allow or enable participation, but common is also the case that it is difficult to find people willing to participate. The fact that often people do not want to participate does not, however, overrule the fact that possibilities for participation should be increased. In some cases the willingness may well be a result of distrust in the system and doubts upon the effectiveness of participation. Low willingness to participate might be a symptom of dysfunctional participatory practices, rather than disinterest in the issues assessed or decided upon. For example, based on three case studies from different continents, Fraser and co-workers concluded that the process of engaging people provides an opportunity for community empowerment; that stakeholders and decision-makers consider participation irrelevant unless it formally feeds into decision-making [13]. In addition to being meaningful, participation should be made as smooth and effective as possible. In some cases it might also be necessary to come up with means to motivate certain important stakeholder or citizen groups to participate in order to guarantee a broad enough representation of perspectives to be included in an assessment.

Cost and time expenditure of participation

Organizing broad participation that has substantial impact on the assessment requires expertise, takes time and effort, and is costly. This is true in many traditional ways of organizing public participation and stakeholder involvement. However, these methods do not make much use of the possibilities provided by modern information and communication technology. Face-to-face meetings, group negotiations, public surveys, public hearings etc. all have their strengths in organizing participation, but a big part of the potential outcomes that can be expected to be achieved by these methods, could also be achieved in virtual settings. Moving into web-workspace-based participation also enables much broader participation without the need to limited in terms of time and space. Also the costs of virtual participation are in most cases much smaller and it can allow more speed and transparency to the process. The traditional participatory efforts can be focused in cases where physical contact is considered necessary and complemented and supported with the means of virtual participation possibilities.

Assessors' and decision makers' attitudes towards participation

The last, but not least, challenge is the attitudes of the people who are in charge of assessments and decision making processes; assessors themselves and decision makers as users of the assessments. The prevailing assessment tradition is that assessments are closed processes, and participation by non-experts is seen as a threat to the quality and objectivity of the assessment. Most of the current methods, tools and practices have also been developed according to this mindset. Despite a whole lot of studies that indicate a need to move towards more open participation as well as encouraging examples from other fields, participation is still often considered as an annoyance or even a burden. Could IEHIA, as a new integrated approach to environment and health assessment, be a pioneer in this sense, and take also the task of integrating people and perspectives as a part of its mission?

Facilitation of participation

Facilitation of participation is about methods and tools that provide the means to enable participation. Methods are the means to achieving the goals of the assessment and tools are the means to applying the methods as intended. The methods and tools to facilitate participation should not be considered as separate methods and tools for participation in addition to the methods and tools for assessment, but rather as the participatory functionalities of the assessment methods and tools. As stated above, participation should be integrated into the assessment process itself, not treated as a separate add-on process alongside or, as sometimes happens, after the assessment. A more general account of facilitating collaboration in assessment is provided on page Facilitating mass collaboration in assessments.

What basically is required in enabling participation in assessments is to make the participants become a part of the assessment process by:

  • Providing participants possibilities to get acquainted with the assessment
  • Providing participants possibilities to contribute to the assessment
  • Synthesizing participant contributions into the assessment
  • Explicating the impacts of the participant contributions to the assessment
  • Providing possibilities to do it all again in repetitive iterations

Although the point of view above is that of an individual, it should be realized that when considering participation as an activity of a collective, it is a compilation of multiple, overlapping individual processes taking place simultaneously.

The first bullet actually comprises of two different factors, the availability of information and usability (or comprehensibility) of information. The participants need to have access to the information in a suitable media when it is needed, but they also need to be able to understand, or internalize, the information. This must, and can, be facilitated by assessment methods and tools. It must be noted that both factors, but in particular usability, are also determined by the capabilities and other properties of participants themselves, and education and other modes of participant support may also be needed.

Equipped, ideally, with adequate acquaintance with the issues of assessment, the participants are most often likely to identify points where their perceptions, values, and understanding of the issues do not match with what is currently represented in the assessment. Things may be missing or they may be in conflict. The methods and tools must provide support for identification of these points as well as formulation and externalization of the statements regarding these points.

Perhaps one of the most challenging things in enabling participation, but also in knowledge creation at large, is the synthesis of contributions into existing body of knowledge. The whole activity of information collection from participants is futile unless their contributions become incorporated into the assessment.

If the participant contributions are duly synthesized in the assessment, they also have corresponding impacts on the assessment outcome. The impact still needs to be clearly explicated for two reasons. Firstly, in a collaborative process, the contributions and their impacts must be made explicitly available as a part of the shared information to other contributors, so that they can, in turn, internalize them as parts of their participatory actions. Secondly, as mentioned above, the participants will consider participation as irrelevant, unless they see that their contributions have a real impact on the outcome. Again, this is not only a question about the interaction between the participants and the assessment, but also about the interaction between the assessment and its use, and what role do the decision makers, as the users, give for the assessment in their decision making process.

Methods for facilitating participation

The methods for participation can be considered in terms of two categories according to their perspective to the issue of participation. The first category is methods for collective knowledge creation, considering the issue from the point of view of developing the shared information object in collaboration. The second category is methods for arranging events of participation, focusing on the participatory procedures. These two categories are briefly described below.

For a major part the basis for enabling participation in assessment is determined by the ways that question generation, information collection, hypothesis creation and development, information representation, and interaction among contributors is organized and supported. By contributors we mean the whole collective of assessors, decision makers (problem owners), and external participants that jointly contribute to a particular assessment. To date the methods of collaborative knowledge creation are more in the form of theoretical principles of applied epistemology rather than practical guidelines. Examples of theories relevant to collective knowledge creation in assessment are interrogative model of inquiry (I-model)[14], trialogical approach[15], hypothetico-deductive method[16], abductive reasoning[17], Bayesian inference[18], and pragma-dialectical argumentation[19]. These methods are considered in more detail on pages Mass collaboration, Scientific method and Discussion.

How, when, where etc. to organize participation is naturally more or less affected by the choices regarding the degree of openness in its different aspects in particular assessments. The procedural methods for participation are various and designed for different needs and different contexts. Despite some considerations on suitability of different approaches for different types of situations and problems [20] [5], the method descriptions tend to be focused on managing the participatory process, and remain vague on incorporating participant contributions to the assessment. The issues of openness in participatory IEHIA and different approaches to participation are considered in more detail in Organizing stakeholder involvement.

Tools for facilitating participation

The role of tools in facilitating participation is in aiding the participatory use of chosen assessment methods while striving for the goals of the assessment. Here we consider facilitation in the case of unlimited participation, as it includes and exceeds all forms of more limited participation. We also focus on facilitation in virtual settings where participation is supported by or takes place over information networks, to a greater or lesser extent. This perspective does not rule out, but rather complements or reorganizes the more traditional approaches to participation.

The requirements for participatory tools can also be considered as comprising of two categories, those of facilitating the activities of an individual, and those of integrating the individual activities as collaboration. All in all, the compilation of such tools constitutes a collaborative assessment workspace.

Facilitation of internalization and externalization of knowledge among plural assessment participants [21].

A collaborative workspace is a virtual working platform that allows open groups to participate in assessments. It is an interface for the users to access the assessment contents, make their contributions to the assessment and communicate between each other. It also provides tools to organize and manage the user contributions. In fact the collaborative workspace is a virtual location of storing, manipulating and representing information. It enables the participants to communicate with each other in the form of making their contributions to the contents of the assessment. Enabling this communication through contribution is the primary function that the collaborative workspace provides. Enabling this also includes managing openness, both according to individual users and user groups or according to locations within the information structure of the content. Furthermore, the collaborative workspace provides users with access to specific tools that they can use for making their contributions. Basically the idea behind the workspace is that everything can be searched for, found, saved, exchanged and discussed about in one place. In other words: The workspace functions as the "glue" between all parts of the assessment, helping the assessors to keep it together and in order, as well as enhancing both efficiency and effectiveness of assessments.

In order to be functional, the collaborative workspace should provide its users with:

  • access to browse, search, read and use the information contents of the assessment
  • access to use specific tools to analyze and explore assessment information
  • possibility to contribute to the assessment

In the case of an individual user, satisfying these requirements is technically quite simple and possible to implement in many different kinds of systems. An example could be a stand-alone content management system in a single computer complemented with access to simulation tools. The actual challenges come from representing information in ways that support participants’ cognitive processes of interpreting and internalizing assessment information as well as externalizing ones contributions as parts of the assessment. Division of assessment information content into manageable and comprehensible pieces by ontological information structure, representation of assessment as causal diagrams, and possibility to play with assessment models are examples of possible facilitating functionalities that a collaborative workspace could host to address these challenges.

In the normal case of multiple participants, the situation becomes more complicated. The individual contributions need to be turned into collaboration. The challenge for the workspace becomes one of synthesizing contributions and managing changes.

In order to make the above mentioned functionalities manageable in situations of multiple participants, the collaborative workspace must also enable:

  • version and history control of all individual pieces of information
  • management of openness, both according to users and according to content
  • management of simultaneous use of tools and simultaneous editing of content

See also

Keywords

References

  1. ↑ UNECE (1998). Aarhus Convention. The UNECE Convention on Access to Information, Public Participation in Decision-making and Access to Justice in Environmental Matters. http://www.unece.org/env/pp/documents/cep43e.pdf Aarhus Convention
  2. ↑ Inkinen, A. (2007). Whose knowledge? The role of participation in environmental decision-making, in: NIBR Working Paper 2007:115, Papers from the 8. Nordic Environmental Social Science Research Conference June 18-20 2007. Workshop 5 Environmental Governance and Policy Implementation, 117–132.
  3. ↑ Similä, J., Inkinen, A. and Tritter, J. (2008). Public Participation by Appeal - Insights from Empirical Evaluation in Finland, Journal of Environmental Law Advance Access published on January 1, 2008, DOI 10.1093/jel/eqn021. J Environmental Law 20: 391–416.
  4. ↑ von Hippel 2005, http://web.mit.edu/evhippel/www/democ1.htm
  5. ↑ 5.0 5.1 5.2 Laird, F.N. 1993. Participatory analysis, democracy, and technological decision making. Science, Technology, & Human values, Vol. 18 No. 3, Summer 1993 341-361.
  6. ↑ Gulick, L. 1937. Notes on the theory of organization, in Papers on the Science of Administration, L.Gulick & L. Urwick (Eds.) New York: Institute of Public Administration
  7. ↑ Fiorino, Daniel J. 1990. Citizen participation and environmental risk: A survey of institutional mechanisms. Science, Technology and Human Values 15(.) pp.226-243
  8. ↑ ECLAC (Economic Commission for Latin America and Caribbean 2002. A Latin American and Caribbean Perspective (Report on the Latin American and Caribbean Regional workshop on Science and Technology for Sustainable Development, Santiago, Chile 5-8 March 2002; Series Seminars and Conferences No. 24; Santiago, Chile: ECLAC) in Interfaces Between Science and Society, Guimarães Pereira et. al (Eds.) 2006 Greenleaf Publishing Ltd. pp.36
  9. ↑ isaacson, P. 1986. Pollutant regulation and public sensibility. Environmental Impact Assessment Review 6 (September):229-32
  10. ↑ Stern, Paul C. & Harvey V. Fineberg (Eds.) 1996. Understanding Risk: Informing Decisions in a Democratic Society. Washington D.C.: National Academy Press
  11. ↑ International Risk Governance Council (2006) - White paper No.1: Risk governance; Toward an integrative approach. Geneva: IRGC
  12. ↑ Beierle, Thomas C. 2002. The quality of stakeholder-based decisions, in Risk Analysis 22(4) pp.739-749
  13. ↑ Fraser, Evan D.G., Andrew J. Dougill, Warren E. Mabee, Mark Reed & Patrick McAlpine (2006). Bottom up and top down: Analysis of participatory processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental management, in Journal of Environmental Management 78(2) pp.114-127
  14. ↑ Hintikka, J. 1985. True and False Logics of Scientific Discovery, in J. Hintikka and F.J. Vandamme (eds.): Logic of Discovery and Logic of Discourse, Plenum, New York.
  15. ↑ Hakkarainen, K. & Paavola, S. 2009. Toward a trialogical approach to learning. In Schwarz et al. (Eds.) Transformation of knowledge through classroom interaction. Sense Publisher.
  16. ↑ Popper, Karl R. (1935). Logik der Forschung. Julius Springer Verlag. Reprinted in English, Routledge, London, 2004.
  17. ↑ Paavola, S., Hakkarainen, K. and Sintonen, M. 2006. Abduction with Dialogical and Trialogical Means. Logic Jnl IGPL. 2006; 14: 137-150.
  18. ↑ en:Bayesian inference
  19. ↑ Eemeren, F.H. van, & Grootendorst, R. (2004). A systematic theory of argumentation: The pragma-dialectical approach. Cambridge: Cambridge University Press.
  20. ↑ Fiorino, Daniel J. 1990. Citizen participation and environmental risk: A survey of institutional mechanisms. Science, Technology and Human Values 15(.) pp.226-243
  21. ↑ Harrer, A., Moskaliuk, J., Kimmerle, J., and Cress, U. 2008. Visualizing wiki-Supported knowledge building: Co-evolution of individual and collective knowledge. WikiSym 2008 Proceedings [Online]. Available at: http://wikisym.org/ws2008/proceedings/research%20papers/18500052.pdf [Accessed 31.10.2009].

Related files

<mfanonymousfilelist></mfanonymousfilelist>

Retrieved from "https://en.opasnet.org/index.php?title=Participating_in_assessments&oldid=20194"