Pragmatic knowledge services

This page is a nugget. The page identifier is Op_en5059
Moderator:Mikko Pohjola (see all)
Upload data {{#opasnet_base_link:Op_en5059}}	Unlike most other pages in Opasnet, the nuggets have predetermined authors, and you cannot freely edit the contents. Note! If you want to protect the nugget you've created from unauthorized editing click here

This page (including the files available for download at the bottom of this page) contains a draft version of a manuscript, whose final version is published and is available in the Journal of Universal Computer Science (J.UCS) Volume 17, Issue 3. If referring to this text in scientific or other official papers, please refer to the published final version as: Pohjola, M.V., Pohjola, P., Paavola, S., Bauters, M., Tuomisto, J.T., 2011. Pragmatic Knowledge Services. Journal of Universal Computer Science 17:472-497. http://dx.doi.org/10.3217/jucs-017-03-0472.

The revision history, including the paper that was presented at the I-KNOW 2010 conference, can be found from the corresponding Heande page (password protected).

Title

Pragmatic Knowledge Services

Authors and contact information

Mikko V. Pohjola

(National Institute for Health and Welfare, Kuopio Finland, firstname.lastname@thl.fi)

Pasi Pohjola

(National Institute for Health and Welfare, Helsinki, Finland, firstname.lastname@thl.fi)

Sami Paavola

(University of Helsinki, Finland, firstname.lastname@helsinki.fi)

Merja Bauters

(Helsinki Metropolia University of Applied Sciences, Finland, firstname.lastname@metropolia.fi)

Jouni T. Tuomisto

(National Institute for Health and Welfare, Kuopio, Finland, firstname.lastname@thl.fi)

Abstract

Knowledge, innovations and their implementation in effective practices are essential for development in all fields of societal action, e.g. policy, business, health, education, and everyday life. However, managing the interrelations between knowledge, innovation and practice is complicated. Facilitation by suitable knowledge services is needed. This paper explores the theory of converging knowledge, innovation, and practice, discusses some advances in information systems development, and identifies general requirements for pragmatic knowledge services. A trialogical approach to knowledge creation and learning is adopted as a viable theoretical basis. Also three examples of novel knowledge services Opasnet, Innovillage, and Knowledge Practices Environment (KPE), are presented. Eventually, it is concluded that pragmatic knowledge services, as hybrid systems of information technology and its users, are not only means for creation of practical knowledge, but vehicles of a cultural change from individualistic perceptions of knowledge work towards mediated collaboration.

Keywords

Collective knowledge, Trialogical approach, Innovation, Knowledge practices, Pragmatism, Collaborative knowledge services, Open assessment, Opasnet, Innovillage, KP-Lab, KPE

Categories

L.1.2, L.2.3, L.3.6, L.6.0, L.6.2.

Introduction

Knowledge and innovations are essential for the guidance and development of virtually all fields of practice, e.g. policy, business, health, education, and everyday life. The development of current knowledge societies has changed the ways of working with knowledge towards producing and cultivating knowledge in collaboration with different stakeholders and transfer of practices in relation to the advances in knowledge and innovations [Knorr-Cetina, 07]. This societal change also generates challenges to which the ways of conducting knowledge work and the related knowledge services need to answer. A pervasive example of challenges in converging knowledge, innovation and practice is climate change, where advances in climate science and efforts invested in international climate policy fail to result in effective mitigation and adaptation actions [Brunner, 06], [IARU, 09], [IPCC, 07], [Mickwitz, 09]. An example discourse from a scientific Climate Congress (Copenhagen, March 2009) illustrates the reality of contemporary climate science-policy interaction. While the IPCC chair re-stated in his keynote speech that "we in the Intergovernmental Panel on Climate Change do not prescribe any specific action, but action is a must", the keynote speeches by the policy representatives urged the scientific community to "use their expertise to guide policy" [Hedegaard, 09], "provide the necessary knowledge needed to make the necessary decisions" [Sander, 09], and "express the knowledge in a way that has an intended effect" [Ashton, 09]. The conventional models of linking knowledge and action with chain-like mechanistic relationships between two distinct communities do not suffice in addressing the complexity of sustainable development [van Kerkhoff, 06].

As alternative models for knowledge-practice interaction, for example van Kerkhoff and Lebel [van Kerkhoff, 06] suggest regarding knowledge-action relationships as arenas where research-based knowledge and practice interact, but not necessarily in a simple or straightforward manner, and Brunner [Brunner, 06] calls for a pragmatic paradigm for policy practice that i) considers knowledge as intertwined with action, ii) develops context-sensitive practical knowledge, and iii) evaluates knowledge and actions according to their purposes. The relationships between science and policy, research and practice, or assessment and decisions are subjects of intense discussion in current research [Pohjola MV, manuscripta]. These discourses highlight the main points of improvement, but provide only marginal guidance on practical implementation (see also [van Kerkhoff, 06]). For example, while the academic discourse on assessments in environment and health emphasizes the importance of dialogue with stakeholders and public, the assessment practices often remain a monologue of experts, where contact with users, other experts and public is taken only when obligatory [Pohjola MV, manuscriptb]. Research to date has fallen short on what it means to manage the boundary between science and policy [McNie, 07].

It appears that the importance of converging knowledge, innovation, and practice as well as its main challenges are identified, but the conceptual and practical means for its implementation are lacking. Suitable knowledge services are needed. It is here argued that such knowledge services must: i) build on solid conceptual understanding about the interrelations between knowledge, innovation, and practice, and ii) possess corresponding functionalities to activate effective interaction between knowledge, innovation and practice. It is fundamental that these two issues are considered as intertwined aspects of the same whole. In the following, we briefly present a trialogical framework as a potential foundation for explicating what is required for such knowledge services. By presenting and discussing three novel knowledge services that implement the trialogical approach, we aim to explicate the practical implications of these requirements for what we call pragmatic knowledge services.

Knowledge, innovation, and practice

Shaping of human activity and practices requires innovations and cultivation and creation of knowledge in collaboration. Integration of plural interests and perspectives is an essential part of these processes. Development of shared practices also requires innovative development and use of technologies as well as creation of new kinds of modes of action. These issues are considered below, particularly in terms of their implications for the development of supporting knowledge services.

Collective knowledge creation

Creation of new knowledge is rarely a cognitive process of a single individual. Typically, cognitive tasks are physically, socially and temporally distributed and the new ideas and hypotheses are often materialized as external artefacts [Paavola, 06]. Approaches to distributed cognition have for long emphasized that inquirers in a search of knowledge are not usually processing things only in their heads, but use various resources from their environment to guide the search for new ideas (see [Hutchins, 95], [Salomon, 93]). Also argumentative processes of producing new hypotheses and ideas, i.e. abductive search for hypotheses, can be considered collaborative rather than happening only in individuals' heads [Paavola, 06]. Abductive inference produces tentative solutions to be worked collaboratively. They can be either applied in practice until better solutions are formulated or as intermediate steps that guide and promote the search for better solutions. In pragmatic knowledge creation process the search for novel ideas can be supported by, and often necessitates, abductive argumentation [Paavola, 09a]. By means of argumentation it is possible to reason why one tentative solution should be considered as superior to others or argue about the types of a preferred solution even before such solutions are found. Argumentation also functions as a mediator between collaborators.

The central idea deriving from pragmatism, aiming to integrate the issues of knowledge, innovation and practice, is that people through participation "continuously construct and re-construct the social meanings that shape our thoughts and actions" [Simpson, 09, 1333]. This means that knowledge creation is fundamentally a social process and this process is essentially linked with the ways we act and the kinds of practices we create and maintain. Knowledge becomes intertwined with action and especially with social action [Simpson, 09, 1334-6]. Also the tools and artefacts we use as the means of our action are a part of this continuous process of construction and re-construction. Although the idea of the social aspect of knowledge is not new, in current knowledge societies it extends from socially maintained and explicated knowledge also to the means of knowledge construction.

Innovation

In innovations knowledge becomes integrated into action as systematically developed means for practice. The process of innovation relies on application and generation of knowledge aiming to develop something that can be grounded in practice. The outcomes of innovation can be realized in many ways, not only in terms of economical benefits, and the common definition of innovations as commercialized inventions is too narrow and technology-centred. In recent literature, innovation has been described as the multifocal development of social practice [Tuomi, 03]. It has also been argued that the criterion for successful technical innovations is that they become social institutions, i.e. their use becomes rooted in the common everyday practice [Pohjola P, 09]. The ideas concerning systemic innovations [Andersen, 08], open innovations [Chesbrough, 03], and democratized innovations [von Hippel, 05] point out additional challenges for knowledge services to support innovation by highlighting both the importance and complexity of networked innovation activity.

Contemporary investigations on innovation processes imply that they cannot be understood as linear sequences of independent sub-processes, but rather as multidirectional [Pinch, 84] and multifocal [Tuomi, 03]. They are not merely processes carried out by product developers and R&D departments, but require the participation of various groups of stakeholders, from users to different kinds of professionals. For example, innovation in health care, such as new drug treatment for some disease, requires expertise and involvement of various parties: from patients, nurses and doctors to drug developers, directors of health care organizations, legislation etc. It is only in these systems or networks of multiple actors where the innovations become existent. Involvement of these actors in the early stages of multifaceted innovation processes is essential and knowledge services need to provide support for collective knowledge creation and innovation throughout the whole process, from idea generation to normalization of practices [May, 09].

Trialogical approach to knowledge creation and learning

The “trialogical” approach has been suggested and applied especially in the context of computer-supported collaborative learning (CSCL) as a novel framework for considering the issues of knowledge creation and innovation (see [Hakkarainen, 09], [Lakkala, 09], [Paavola, 09b]. It emphasizes the role of collaborative development and reconstruction of concrete, shared artefacts in mediating knowledge creation, as well as reflecting and transforming knowledge practices, the ways of collaboratively working with knowledge, with supporting processes, and executing knowledge tasks.

A basis for the trialogical approach is an epistemological distinction between three basic metaphors of learning and human cognition associated with monologues, dialogues, and trialogues (figure 1). The monological processes of information sharing and knowledge acquisition, and dialogical processes of learning through communication and participation, are supplemented with knowledge creation as a trialogical process of collaborative development of epistemic artefacts and practices.

Figure 1. The three metaphors of learning.

Innovative activity from the trialogical perspective means that all relevant parties should become involved in the processes of learning and production of knowledge artefacts. There are various unsuccessful examples of attempts to routinise new technology without appropriate inclusion of relevant users into the development and learning processes (for an example in health technology assessment, see [Edmondson, 01]). The trialogical processes extend to the organization of work around concrete artefacts and practices in addition to mere information sharing or communication.

The trialogical approach has been developed in the context of collaborative learning. We maintain, however, that the explicit linking of knowledge creation with practices (e.g. [Hakkarainen, 09]) and with cognitive processes and artefacts as their products and mediators, the trialogical approach is a viable foundation for the convergence of knowledge, innovation and practice also more generally.

Facilitation of converging knowledge, innovation, and practice

The development of information systems has been dominated by two general approaches: technology-centred development and research, and user-oriented development and research. The former has focused on the systems, applications, and technology for representing, organizing, and manipulating information (and knowledge), while the latter has put its emphasis on the ways of interaction between the users and the system [Pohjola P, 10]. These conventional views are currently challenged e.g. by mass collaboration and the Pragmatic Web.

In the early phases of the web (Web 1.0) the emphasis was on presenting existing information in a syntactically structured way (syntactic web) provided by the HTML markup for representing information in the web for users to browse and search. In the Web 2.0 era, the Semantic Web initiative promoted meaningful description of data by means of ontologies that were intended to provide taxonomies of concepts where meanings and relations between concepts could be defined in a unifying way. The increasing interest towards the Pragmatic Web relates to the limitations of the Syntactic Web and the Semantic Web, for example the limited possibility of describing meanings of signs and symbols. Like in any other use of signs and symbols, their meanings become defined and changed in use and interaction between people (and knowledge systems). Meaning is a social phenomenon.

Investigations regarding the Pragmatic Web have been directed for example to examining how communicative actions within a pragmatic context are performed via Web media. These investigations have analyzed how mutual understanding and commitments to actions can evolve in conversation in knowledge systems that support pragmatic aspects of knowledge [de Moor, 10]. Some Pragmatic Web investigators have also proposed extensions to the Semantic Web, such as enhancing human collaboration with techniques for ontology negotiations and pragmatic ontology building in communities of practice [Schoop, 06]. It has also been argued that the Pragmatic Web is not merely a knowledge exchange medium; it should be an active knowledge system that supports human interaction and accomplishment of knowledge tasks [Delugach, 06]. Accordingly the Pragmatic Web, as well as any other information system, should be conceived as a hybrid network constituted by both the users and the technology [Pohjola P, 10].

The emergence of wikis and other web-based collaborative software have enabled the development of new kinds of practices for co-producing knowledge in virtual workspaces where masses of people can engage in collaborative work (e.g. [Tapscott, 06], [Noveck, 09]). These means of collaboration and the artefacts they produce have also become interesting objects of research and development [Cress, 08], [von Krogh, 09]. The rise of social media is said to have brought conversation back into the heart of the Internet, but now, in contrast to the early stages of Internet, intricately interlinked with content [de Moor, 10]. Building on what has been discussed above, certain general requirements for knowledge services to facilitate convergence of knowledge, innovation, and practice can be identified. A pragmatic knowledge service should:

1. Enable collaborative knowledge creation
2. Support development and application of collaborative knowledge practices
3. Support practical implementation of knowledge
4. Adapt to changing contexts, situations, and purposes

The first requirement is centred on the technical properties of the system that constitute the workspace by which the users engage in collaboration. This may mean e.g. tools and functions for managing shared artefacts and collaboration among plural participants with heterogeneous capabilities, as well as organization of contributions.

The second and third requirements extend more to address also the content of the system in guiding the dynamics of the user collective. The technical properties can also provide support e.g. by re-use of shared artefacts, and tools for discussing, developing, and sharing practices. Relying solely on tools and information provided by the workspace may not, however, be sufficient and also social practices outside or besides the workspace may be relevant.

In a pragmatic setting the issues in creation and use of collective knowledge are contextual and situational and vary from a case to another. Consequently, the technical properties and contents of the workspace, the practices of its use, and the practices of knowledge implementation need to be allowed to adapt through the interactions between the workspace, the user collective and the societal context.

Three examples of novel knowledge services

The examples, Opasnet, Innovillage, and Knowledge Practice Environment (KPE), represent different perspectives to implementing the trialogical approach in a knowledge service. Opasnet and Innovillage are developed by the National Institute for Health and Welfare (THL) in Finland in collaboration with multiple partners in Finland and Europe. KPE is a development of a multi-partner project where the research work has been coordinated by the University of Helsinki, and the technical development of KPE by Metropolia University of Applied Sciences, Helsinki.

All the examples are described in terms of their a) purpose, b) method, c) system, d) use, and e) contextual fit. Of the three examples, Opasnet and KPE are already existing and functioning knowledge services and described in terms of their current manifestations, while Innovillage is still in its early development and mainly considered according to planned designs.

Opasnet

Opasnet is a web-workspace for producing and providing science-based support for policy making in the field of environment and health. It provides a virtual arena for open collaboration on generating practical solutions to problems of societal relevance. Opasnet aims to improve increased awareness and understanding among both those who make decisions and those who are affected by those decisions. It welcomes decision makers in public policy, industry and commerce, experts of different kinds, as well as civil society organization representatives, consumers, and citizens as active participants in open assessments. Opasnet is developed in the context of environmental health, but its scope of application is intended as extensible in principle to all systematic practice-driven endeavours of collective knowledge creation.

The main principles in organizing open assessments are:

1. Assessments create collective knowledge by searching solutions to practical problems by means of science and account of plural values.
2. Assessments are endeavours of describing reality as causal networks of interrelated phenomena. Conclusions to guide decisions and actions are drawn based on analyses made over the network. No intentional distortion of information (e.g. going for the worst-case scenario) is accepted.
3. Participation in assessments is unlimited. Limiting of openness is allowed only based on well argued cogent reasons. [Pohjola MV, manuscripta]
4. Information objects produced in assessments should be freely available for anyone to use and develop further.
5. Assessment performance comprises of i) quality of information in relation to the problem addressed, ii) applicability of produced information in its intended use, and iii) efficiency of its production process [Pohjola MV, 10].
6. Also methods and tools of open assessment are subject to open critique.

The most distinctive aspect in open assessments is openness. As the issues of environmental health are relevant to virtually anyone, anyone can be a relevant contributor to an environmental health assessment, and the knowledge created in environmental health assessments can be of relevance to anyone. Openness is also seen as an essential aspect of scientific inquiry and to enhance the relevance and applicability of the knowledge created in assessments. On the other hand, it brings about practical challenges for managing assessments.

Opasnet is a collaborative workspace for conducting open assessments. It provides the assessment participants a) the virtual location of and access to the workspace, b) the information available in the workspace, c) a structure for organizing information, and d) tools to aid and guide collaborative production of information within the workspace. Opasnet is located in the open internet (http://en.opasnet.org).

Technically Opasnet consists of a wiki, a database, and a modelling and simulation environment. The main interface between Opasnet and its users is the wiki. It is built on the Mediawiki platform, and many of its basic functions resemble those e.g. in Wikipedia (http://en.wikipedia.org). The database stores numerical information for and from the modelling and simulation environment used for mathematical operations and analysis. They are also essential, as quantitative data and models often form the core of the information produced in environmental health assessments. Collaboration, however, mainly takes place through textual and graphical communication that a wiki supports well. Thereby it is most often adequate that the models, their application, and their results are described and discussed as parts of the assessment, although those actually participating in modelling and analysis would only constitute a small fraction of the collaborators in the assessment as a whole.

The main information content of Opasnet consists of past and on-going assessments and their parts, i.e. variables. In addition there are descriptions of the methods and tools needed in open assessments and other supporting information e.g. about research projects, studies, terminology, lectures etc. The supporting information aims to guide and aid in using the system effectively according to its purpose. The information is structured as wiki pages (figure 2), and related data (in the database) and models (e.g. files for external applications) can also be read or launched from or linked to the wiki pages. Every wiki page also has a related discussion page.

Figure 2. An assessment page in Opasnet.

Certain information objects, e.g. assessment and variable have a predefined formal structure; name, scope, definition, and result. The sub-attributes under each attribute vary depending on the object (table 1.). A simple unified information structure is cognitively ergonomic and aids in targeting contributions to relevant locations within the system, and enhances the re-use of previously produced objects.

Most contributions to Opasnet take place in textual or graphical form in the wiki. The ways of contributing are: a) reading existing content, b) commenting of existing content with a page-specific comment box, c) participating to or starting a free discussion on a discussion page, d) participating to or starting a formal argumentation on a discussion page, e) structuring comments and freely formatted discussion fragments into formal argumentation structure on a discussion page, and f) editing contents of a page or creating a new page.

Table 1. The attributes and sub-attributes of assessment and variable objects

Attribute	Explanation	Assessment sub-attributes	Variable sub-attributes
Name	A descriptive identifier
Scope	A question describing the issue(s) of interest	Purpose Boundaries Scenarios Intended use Participants
Definition	Explains how the question is answered and provides rationale for the answer	Decision variables Indicators Value variables Other variables Analyses Indices	Dependencies Data Unit Formula
Result	The current answer to the question	Results Conclusions

The two first kinds of contribution are possible without logging into the system, but the latter require creating a user account and logging in. Anyone can create an account. Contributions by multiple discussants can be organized on discussion pages according to pragma-dialectical argumentation [van Eemeren, 02]. Argumentation consists of a statement regarding the actual content describing the issue of interest, a hierarchy of arguments either attacking or defending the statement or other arguments, and a resolution. Corresponding templates for discussion structure, attacking argument, defending argument, and (neutral) comment have been implemented as buttons in the edit window of Opasnet. Once a stable resolution has been found, it should be implemented on the content page accordingly. If no single solution can be found, the resolution consists of all views that are still considered valid after the argumentation. Also other comments and discussions should be taken account of when editing the pages.

Opasnet users can also contribute by rating Opasnet pages according to their i) scientific quality and ii) usefulness (see figure 2). The user evaluation provides feedback the contributors to the page and guides other users in perceiving and interpreting the content. Opasnet also has some project managerial functions, e.g. task lists for nominating/suggesting tasks to different users. The users can also volunteer as moderators of wiki pages and the user community is assumed to self-organize to adopt different housekeeping roles as happens e.g. in Wikipedia.

The development of Opasnet started in 2006, and it has been piloted in several assessments in research projects, e.g. a benefit-risk assessment on consumption of farmed salmon (http://en.opasnet.org/w/Benefit-risk_assessment_on_farmed_salmon). It begins to be ready for full-scale use, but experiences on broad participation are still limited. In addition to the main Opasnet site in English, there is a Finnish language version of Opasnet focusing on issues of domestic interest (http://fi.opasnet.org), and a limited access site Heande for use e.g. in research projects or other situations where complete openness of all content is not possible due to different reasons. In the main Opasnet in English there have been more than 300000 visits since Opasnet was opened, and there were more than 2000 individual visitors during 2009. The number of active editors is currently between 20 and 50, and there are nearly 1500 content pages. At the Assessment and Modelling Unit of National Institute of Health and Welfare, the primary developer of the system, Opasnet has gradually become one of the main tools for carrying out everyday work tasks.

Mostly Opasnet has been used by a relatively limited group of scientific experts in environmental health in roles of developing content in the system. Certain assessments, e.g. on a plan to build a municipal solid waste incineration plant in a town in south-west Finland, (http://fi.opasnet.org/fi/Poltto, in Finnish), have also attracted considerable attention by public, mainly as readers, but also as discussants. Serious attempts to involve professional decision makers and other intended users of assessment results into using Opasnet have been rare and results thus far are scarce. Thereby the effectiveness of the approach has not yet been sufficiently demonstrated. However, there are several new research projects that have adopted Opasnet as their collaborative workspace. For future development it will be necessary to attract more practitioners and public to participate in creation, and particularly use, of knowledge.

The most common challenges in using Opasnet appear to relate to: a) finding information, or whether it exists, within the system, b) knowing what one is "allowed" to do within the system, c) overcoming the fear of making mistakes in an open system, d) using the edit-window to make contributions to Opasnet-wiki, e) applying the argumentation format to organize discussions, f) deciding upon the object type and name when creating a new object, g) applying the attribute structures, i) managing relations between related and/or similar objects, j) managing the relations between wiki pages and external models addressing same phenomena, k) uploading data to Opasnet Base, l) creating a real linkage between assessments and their intended use, and m) invoking active collaboration among more than 2-3 individual users on a shared object or set of objects. It appears that the threshold to adopt Opasnet should be lowered and the benefits of using the system need to be better demonstrated. Some of the technical barriers can also be expected to be gradually overcome through the technical development of the Mediawiki platform as well the increasing familiarity among the potential users with collaborative software.

Participation has become a central issue in public policy making upon environmental issues during the last few decades. Participation is addressed e.g. in the Rio Declaration on Environment and Development [UNEP, 92] and the Aarhus Convention on Access to Information, Public Participation in Decision-Making and Access to Justice in Environmental Matters [UNECE, 98], as well as in environmental legislation on different levels. Also the public has become more accustomed to use open information sources to obtain knowledge upon issues of their interests and also to use that knowledge to influence decisions regarding those issues. However, the currently common approaches to environmental health assessment are still relatively conservative and in practice assessments mostly tend to be exclusive rather than open [Pohjola MV, manuscriptb]. Among researchers openness is often met with concerns regarding e.g. reduced quality, loss of credibility, vandalism, and intentional bias.

In this setting Opasnet has been received with apparent interest, but also quite persistent reluctance to adopt it into everyday use. This is probably partly due to underdevelopment of the method and incomplete or complicated properties of the workspace as well as the effort required to become acquainted with the system. Partly it also derives from the conflicts between the principle of open collaboration and the prevailing practices in research as well as policy making. Researchers often fear that operating in an open system hinders, or completely prevents, scientific publications, and thus accumulation of scientific merit. It may also be perceived to lead to the erosion of the traditional expert status. Also professional decision makers may be unwilling in practice to share their power, and the meaningfulness of participation in decision making processes is sometimes questioned by decision makers, stakeholders, and problem owners alike (see e.g. [Inkinen, 07]).

All in all, it still appears that the society at large is gradually moving towards broader acceptance of systems and practices that build on openness and collaboration. One example is the first prize recently awarded to Opasnet in the World Summit Awards Finland competition in the e-government & institutions category (http://www.mindtrek.org/2010/wsa), opening up a chance for Opasnet to participate in the global World Summit Awards in 2011 (http://www.wsis-award.org/).

Innovillage

Innovillage is an environment for the development, assimilation and evaluation of technology and services in the social and health sector. Its central idea is to support open participation in the development processes of social and health care services and to provide a method for evaluating the effectiveness and outcomes of these development processes. It promotes a multifaceted perspective to innovations where all the relevant participants should have a possibility to engage in the innovation process. This means that the participants in a service innovation process can include patients, health or social care professionals, developers, researchers and management. Innovillage provides an approach and tools for innovations that can overcome the shortcomings of existing approaches, for example, in addressing the local and contextual dependencies of social and health services.

The theoretical background of Innovillage is in multi-perspective view of systemic innovations, founded on the Actor-Network Theory (ANT). Initially, ANT has been developed in science and technology studies (Latour, Callon) and is here extended with the trialogical approach. ANT sees innovations and technology as embedded in a network of both human and non-human actors [Latour, 05], in which the technology and innovations become defined. This nature of innovations (both social and technical) requires that the knowledge service needs to provide support for extracting information about the relevant aspects of development and implementation of a practice. From the trialogial perspective, it is crucial that all relevant members of the network can contribute to a shared object of work and provide their perspectives for the innovation process. Ignoring issues of usability, practical know-how or ethical aspects, not to mention legal or economic issues, can lead to the failure of the innovation process.

For conceptualizing the services and assessing the effectiveness of the services, the developers of Innovillage have developed a Relational Evaluation Approach (REA), a framework for the development and assessment of technology and services in social and health care. The REA is at the heart of Innovillage and it is applied in Innovillage for i) describing the essential components of the service innovation, and ii) assessment of its outcomes in relation to the purposes that the service has been developed for. REA is applied in three stages:

1. As a conceptual tool for developing ideas in the early stages of the innovation process, and developing an Implementation Model (IM) for the service.
2. Assessment of the implementation process of the service in a certain context (according to the IM).
3. Assessment of the longer term effects of the service in relation to the purpose and expected outcomes of the service described in the IM.

When constructing the IM of the service, the REA is applied in describing its central and essential elements as a hypothesis of what is required in order to produce the intended outcomes and to fulfil its defined purpose. The aspects of the REA framework are i) definition of purpose and expected outcomes, ii) a description matrix for more specific characterisation (table 2), and iii) a process-like summary.

In its general form, the description matrix consists of six topics which are described from three perspectives (if necessary): clients', professionals', and organisational perspective. The required level of detail in descriptions depends on the purpose and outcomes of the service. For example, the purpose of a new kind of service for the home care of elderly people could be specified as providing more quality life years at home, and the outcomes e.g. as maintaining a good quality of life, reducing mental problems from institutionalization, and cost-effectiveness in terms of reduced amount of labour. The characteristic features of the service are detailed in the description matrix e.g. in terms of what kind of professionals are required, what kinds of clients there are, what education is needed, what use of tools is required from the professionals and the clients, and what of organisational prerequisites does the service have. Eventually, the summary describes the processes (actions) that the service is made of, such as daily food delivery, medical examinations etc.

The Implementation Model is used as a conceptual tool or an artefact for communicating the features of the service. It is also the model for implementing the service in different contexts. Each implementation in a certain context naturally has its specific independent features, as the resources and skills of professionals vary between organizations, the clients are not a homogeneous mass, and organizational structures are different. The implementation of the service thus requires tailoring of the general model according to the specifics of the context. Innovillage also provides tools and concepts for specifying the implementation processes and for evaluating both the implementation and the outcomes of the service.

The development work and innovation processes are carried out in a collaborative web-workspace. The REA is built into the system and the collaboration in the web-workspace is structured by its concepts. It provides an environment for the participants to collaborate on specifying the general features as an IM of services in relation to their specified purposes and expected outcomes and assessing existing services according to an IM. These processes are conducted in the system by enabling the users to create projects, create networks with other projects and other tasks to support and enhance collaboration. The IM's and the assessments are shared knowledge artefacts that coordinate the collaborative work. The work carried out within the system is supported with a library of methods for the development and evaluation of health and social services. The workspace also consists of a database of (descriptions of) existing services which can be exploited as templates in the development of a new service. The Innovillage workspace actually has much in common with Opasnet: a collaborative working environment, emphasis on open participation and involvement, aim to combine the expertise of various stakeholders into the development process, and shared objects of development.

Table 2. REA description matrix with some example questions

Topic of description / perspective	Clients' perspective	Professionals' perspective	Organisational perspective
Tasks and division of labour	What kind of tasks is required from the clients?	What kind of tasks the professionals are required to take? What kind of division of labour is required among the professionals?	What kind of organisational tasks does the service require?
Actors / Agents	What kind of actors do the clients consist of?	What kind of professionals is required in the service?	What kind of organisational actors does the service require?
Tools, skills, knowledge	What kind of knowledge, skills and use of tools does the service presuppose from the clients?	What kind of knowledge, skills and use of tools are required from the professionals?	What kind of knowledge, skills and use of tools are required from the organisational actors?
Rules and principles	What kind of ethical and other rules and norms relate to the clients' activity?	What kind of ethical and other rules regulate the professionals' work?	What kind of ethical and other rules regulate the organisation?
Laws and regulations	How do laws and regulations relate to clients' activities?	What kind of influence do laws and regulations have for professional work?	How do laws and regulations affect the organisational activities?
Economics	What kind of economical resources does the service require from clients?	What kind of effects do the activities of the professionals have on economical resources?	What kind of economical resources are required from the organisation?

The main functionalities for facilitating collaboration are 1) management of projects in which the innovation or assessment processes are conducted, 2) collaborative workspace where the service and its assessment are conceptualized, 3) a network manager for enhanced interprofessional collaboration between different people from different professions, and 4) a search interface, which provides both a professional-type systematic search with specific keywords and a more natural-type of interface for searches (see [Hearst, 09]). The main user group is professionals who conduct development projects and work in the social and health sector. Other relevant user groups are clients participating in the development of services, managers responsible for the projects, as well as project funders.

In addition to a web-workspace for collaborative development and assessment, the knowledge service also entails support for collaboration and transfer of knowledge and skills within and between networks, such as face-to-face meetings, consultation thematic workshops, and tutoring. Their function is to promote interprofessional working practices and democratic participation by different stakeholders. It is not presumed that mere technology (as a knowledge service) with its functionalities and affordances could facilitate open and interprofessional innovation processes in an effective way. Instead, the knowledge service should be a combination of technology and other facilitating services and practices.

The Innovillage knowledge service is a combination of open participation and more structured and managed types of work. Rather than relying on the assumption of self-organizing communities as the users of the system [Pohjola P, 10], the system aims at supporting the development of effective working practices that enhance the multifaceted approach to developing novel services. This can be done in the service e.g. by inviting different professional networks into co-development and co-creation of services, the ways of creating an interprofessional network by inviting various stakeholders into a development process or by interacting with other projects and developers.

An example of interprofessional collaboration is the ongoing pilot project within Innovillage where more than ten municipalities in Finland implement a developed and piloted set of five services in social and health care. The services are first described as an Implementation Model by local service development professionals together with a group of forthcoming users and other social and health care professionals. At the second stage, the municipalities create an implementation plan according to the IM's and specific needs of their organisation. During the course of implementation they conduct an assessment of the service and its implementation within their individual contexts. The individual projects involve various stakeholders ranging from managerial level to social workers, nurses as well as health and social service clients. The central participants from different municipalities that work with similar services also come to form a wider interprofessional network within Innovillage.

One central aim in developing Innovillage is to provide support for development and management of working practices. Experiences from the use of knowledge services show that even services with the best functionalities and affordances do not generate collaborative development and innovation activity without motivated and committed practitioners. It has also been noted that the practitioners need to be engaged into knowledge creating processes and be motivated to work for the outcomes of the process (see [Engeström, 08]). Successful collaboration also requires the creation of the ways of working with knowledge, i.e., creating knowledge practices [Hakkarainen, 09]. By providing the support for managing work roles and work tasks, the system aims at supporting the development of effective working practices that can become normalized in the practicing community [May, 09]. This means that the service should enable the practitioners to construct the ways and practices of collaboration both within and outside the system. Facilitation of the socio-technical innovation process of Innovillage requires consideration of the supporting knowledge service as a hybrid system of both technology and its users.

Innovillage has been developed according to the acknowledged need for enhancing the creation and assessment of services in social and health care. Although there is much pep talk for creating more customer-oriented ways of service development, much of existing practices and social structures (funding, organizations, etc.) support project-oriented and professional-driven development processes. An increasing amount of work where clients and client perspectives are more involved is, however, being made. As Innovillage is in a developmental phase, the experiences of contextual fit are still limited. The workspace and the REA are currently being piloted and tutoring and workshops based on REA are only just starting. However, Innovillage builds on an existing system for describing services in social care, so there already are professionals who are familiar with the approach and have previous experiences that support the kind of work that Innovillage promotes. Generally the professionals are also eager to get involved with the new extended approach to describing and assessing services.

The challenges of the Innovillage knowledge service in a national scale relate to a required cultural change in social and health care service innovation. Much development work in the social and health sector is done in individual projects and there is a lot of overlapping work being done. In many cases also the outcome effectiveness of these projects is not properly evaluated. What is required is a change towards increasing and more open distribution of the services developed in the projects and more efforts on the implementation and assessment of these services. Innovillage aims to facilitate this kind of cultural change, but also participation of the multiple stakeholders is required to make it happen.

KPE

Knowledge Practices Environment (KPE) is a virtual environment with a set of integrated tools and functionalities for working with knowledge artefacts, and for planning, organizing and reflecting on related tasks, artefacts and user networks [Markkanen 08], [Lakkala 09]. It is developed in a large, five-year (2006-2011) EU-funded Knowledge-Practices laboratory (KP-Lab) project (see http://www.kp-lab.org/ and http://www.knowledgepractices.info). An explicit goal of the KP-Lab project has been to develop and investigate tools, practices, and models that support collaborative knowledge creation processes and trialogical learning. A basic starting point for the project has been to develop tools to support flexible work and learning with knowledge artefacts and related practices and processes both in educational and working contexts. The focus has been especially in higher education courses where students and teachers collaborate with outside organizations and learn “authentic” project work and knowledge practices. These more regularly take place in universities of applied sciences, but similar practices have been also investigated in universities.

In the KP-Lab project following design principles have been formulated to characterize the general features of trialogical learning (cf. [Paavola 09b]):

1. Organizing activities around shared objects: A central idea of trialogical learning is that work and learning are organized around developing shared, concrete objects, that is, conceptual artefacts (e.g., ideas, plans, models) through concrete, material products (e.g., prototypes, design artefacts) and/or practices (e.g., ways of working in higher education).
2. Supporting integration of personal and collective agency and work: People integrate their own personal work and group’s practices and resources for developing shared objects, combining participants’ expertise and contribution into the shared achievement.
3. Emphasizing development and creativity on shared objects through transformations and reflection: Interaction and transformations between tacit knowledge, knowledge practices, and conceptualizations are seen as a driving force in processes of knowledge creation.
4. Fostering long-term processes of knowledge advancement with shared objects (artefacts and practices): Trialogical learning requires sustained, long-standing work for the advancement of the objects of inquiry.
5. Promoting cross fertilization of various knowledge practices and artefacts across communities and institutions: Knowledge work in KP-Lab engages people in solving complex, authentic problems and producing objects also for purposes outside the educational institution. An essential aspect of the KP-Lab project is hybridization between schooling/studying and research cultures as promoted in various investigative learning practices.
6. Providing flexible tools for developing artefacts and practices: Trialogical learning cannot easily be pursued without appropriate technologies that help the participants to create, share and elaborate, reflect and transform knowledge artefacts and practices. Collaborative technologies should provide affordances for trialogical learning processes.

These design principles are quite general, meaning that they could not be implemented straightforwardly to guide the practices and technology development. They have, however, provided broad outlines for characteristics of learning and for the needs required from the mediating tools developed in the KP-Lab project.

KPE is a web-based application providing tools, functionalities and features for sustained collaborative working with shared artefacts, processes, and practices. KPE provides virtual working spaces, called shared spaces, for the collaborative work, enables viewing the knowledge objects and their relations from different perspectives and supports object-bound development of all items in a shared space. Basic tools and functionalities include, in addition to the common upload, versioning etc. functions, the following: integrated wiki, note editor, commenting, context-based chat, semantic tagging, linking of items allowing also spatial organisation, real-time and history based awareness features, and various analytic tools, among others. The tools and functionalities are highly integrated into the basic views to enable versatile and flexible connection, organization and reflection of all information related to the knowledge objects, processes and people concerned. The role of technology for enhancing trialogical practices is framed by four types of mediation which specify the above mentioned design principles to the general aims of the technology development. These types of mediation are reformulations of the ones introduced by Rabardel and Bourmaud [Rabardel 03], see [Hakkarainen 08], i.e., epistemic, pragmatic, collaborative, and reflective mediation (table 3).

Table 3. Short descriptions of the types of mediation supported by KPE

Type of mediation	Description
Epistemic mediation	Creating and working with knowledge artefacts. The aim is to give support for users to create, transform and organize shared knowledge artefacts, to support commenting on shared artefacts (object-bound commenting and chat), and development of shared artefacts (drafting and versioning iteratively), as well as sustained use of knowledge artefacts and conceptualizations.
Pragmatic mediation	Organizing and coordinating knowledge-creation processes. The aim is to provide flexible possibilities and support for planning work processes, support for updating and revising the plans, as well as coordinating the collaborative knowledge processes with other practices.
Collaborative mediation	Building and managing networked communities and social relations required for carrying out knowledge-advancement efforts. The aim is to support networking, community building and interaction around shared processes and artefacts as well as interaction across different groups and communities allowing users to lean, learn, share and combine on each others' competencies, expertise and experience.
Reflective mediation	Making visible, reflecting on, and transforming knowledge practices. The aim is to enable users to reflect on their practices (processes), and jointly analyze and developed the practices and processes.

The system is designed to support multimediation by providing a shared knowledge space that facilitates all four modes of mediation and their flexible use. A shared space in KPE can be either a personal space or a collective space. A collective space is created for the knowledge community involved in a trialogical process. Each shared space includes three main views: a Process View, a Content View and a Community View. The Process View supports time based chronological way of organizing tasks. It is mainly used for projects with explicit tasks and deadlines. The Content View includes all the items that the users have produced, e.g. content-based chat, notes, uploaded files, web-links, wiki-links or Google documents, as well as the tasks (figure 3). The items can be commented, discussed, tagged and linked. The Community View provides a visual presentation of all members of the shared space and textual description of the members contact information, items, action and assigned responsibilities they have. The members can create groups and assign roles to themselves as well as create specific e-mail lists for groups or shared space. The views can also be tailored by users according to their specific needs (Tailored View).

All of the views provide synchronous awareness features, e.g. a lock if someone works on an item, a hand if someone is moving an item, colour-based coding of on-line information, etc. In addition, asynchronous awareness information is provided by means of recent changes, notifications and histories (version lists) of items.

Figure 3. Content View: Items and tasks of a student pair in an example course.

KPE has been developed and investigated in large research cases in higher education courses and projects especially in Finland, Netherlands, Austria, Germany, and Sweden. It has also been used in various smaller courses all over Europe. In the production version of the environment there are 105 shared spaces and in total the amount of registered users is around 1200. At the moment of writing this, the project is still ongoing, meaning that the system, following the agile development method ideas, is a beta version and has some usability issues to be settled. KPE has provided a test bed, which has amounted to such requirements, that it has not turned out as useful as was expected in the design process. Therefore, some of the functionalities and features need to be simplified and the core functions need to be brought to the front of the user experience. Strong points of KPE have been the possibility to structure and organize the group work in a flexible manner, for example visually and/or non-linearly, to use one platform for various tasks and processes, and to use different tools for various purposes. Some restrictions have been experienced with some tools, e.g. for producing text documents, conferencing, etc., which are not integrated, but users would like to be integrated with KPE. Multimediation, e.g. combining epistemic and pragmatic activities together, is, however, seen as a clear advantage of KPE.

The KP-Lab project challenges existing practices in the higher education. Existing knowledge practices in regular university courses are still more oriented towards individualistic learning. More widely used virtual learning environments, such as Moodle or Blackboard, provide only limited support for collaborative knowledge creation practices, and support more information sharing or work with ready-made tasks provided by teachers. Furthermore, none of the current learning environments combine the spatial, semantic and filtered categorization and organization of knowledge artefacts and practices/tasks in similar holistic manner as KPE does. Courses in universities of applied sciences are more often oriented to learn project work done collaboratively but also there the challenge is to learn more in-depth practices of collaboration. There is a clear need for developing courses and projects aiming to develop tools and practices needed in the modern knowledge work.

Summary and comparison of examples

The main characteristics of the three example knowledge services are summarized and compared in table 4 according to the general requirements for pragmatic knowledge services identified above. Much of the differences in the approaches can be considered to derive from the different use purposes in their contexts. Despite some differences, all of the examples hold properties related to each of the requirements. Conceptually they also address the essence of the trialogical approach, collaborative development of shared artefacts, in a similar fashion, although the technical implementations vary from one to another.

KPE is a collaborative learning environment making it primarily a service for creating knowledge for its own sake, although not indifferent to the uses of knowledge. The approach in KPE more or less assumes a specified group of users, such as a class or participants of a course, although in principle it does not pose any restrictions to the size of the group. The KPE workspace provides a multitude of tools for flexible use by its users for a broad range of specific purposes.

Opasnet is a knowledge service for creating knowledge to support societal decision making. As such it explicitly includes the intended use of knowledge in its method as well as the workspace implementation. Also the descriptions of methods and tools are explicit objects of collaborative development. Opasnet adopts an extreme approach to openness by allowing unlimited participation, and provides a general structure to guide collaboration, which leaves space for improvised use in collaborative knowledge creation for virtually any purpose.

Table 4. Summary and comparison of the main characteristics of the three example knowledge services.

	Opasnet	Innovillage	KPE
Collaboration	A unified information structure on a Mediawiki platform Method and tool guidance and other information in addition to assessments and variables Assessments described as causal networks of variables A range of contribution options: reading, commenting, discussion, argumentation, and content editing Complete openness as a default	Three structured forms of knowledge objects: Implementation Model (IM), assessment of implementation, assessment of effectiveness Relationships between objects either hierarchical (e.g. belonging to an IM) or other links (e.g. functional similarity indicated by a user) Various ways of creating knowledge objects (e.g. individually, by a group, in open collaboration) All objects can be commented by all users	Content View (knowledge objects), Process view (structuring the process), Community view (group formation), Tailored view (re-arrangement of items) Wiki pages, short text notes, comments, links, object-bound chat histories, or uploadable files Relationships between items as 1) visual or conceptual links, 2) spatial arrangement, or 3) semantic tags Awareness features and recent changes help organising the work flow Brainstorming by light textual note tool and the sketch pad for drawing ideas.
Knowledge practices	Also methods and tools described and developed within Opasnet Previously developed objects re-usable as such, as templates, or as examples Wiki-practices resemble those in other Mediawiki applications	Tutoring and workshops to support collaborative ways of working Library of methods for organizing innovation and development projects	Analytic tools for analyzing and visualizing knowledge practices Shared spaces can be used as examples of tool sets, of spatial or semantic organization, or of different tasks structures Previous works of users as examples for newcomers and for re-use
Knowledge implementation	Assessment questions formulated according to practical knowledge needs Intended knowledge users (explicitly intended to be) involved in assessments Recommended decisions or actions described in assessment conclusions	Structured assessment of implementation of new service practices Implementation Model as one of the central elements Professional tutoring for implementation	Participants from different organizations preferred and easily added A space can be all open or restricted depending on the need of the project
Adaptivity	All content subject to change as new knowledge emerges An environment for sharing and integrating information produced either within or outside the system Generic design allows application in collective knowledge creation for all purposes	Knowledge objects are iteratively evaluated; assessments should affect the implementation models Change of experiences and information between different networks working on similar issues User-experiences provides feedback to assessment and models	All knowledge artefacts are changeable Virtual spaces can be re-used or copied The sets of tools can be selected by the users to fit their needs Users can share and adopt-adapt best practices in an integrated collective forum

Innovillage is an innovation environment, which inherently brings the practical implementation of knowledge into the core of the knowledge service. This also results in explicit inclusion of activities of supporting knowledge practices and knowledge implementation outside or besides the workspace in the knowledge service concept. The approach can be characterized as semi-open, as it opens some aspects of the process for unlimited participation, but leaves certain aspects for professional collaboration. Innovillage provides the most rigorous structure for the service, which also makes it most bound to the specific purpose it is primarily developed for. This does not, however, prevent its flexible use within its intended context of use.

All of the examples challenge the prevailing paradigms in their contexts. The basis of the challenge is the same for all; the trialogical approach. The individualistic perceptions of learning and knowledge creation appear as deeply rooted in common practices of all knowledge work, not only education. This appears as resistance to adopt collaborative trialogical knowledge processes, even if their purposes, methods, and practical implementations were in principle welcomed and accepted. It does not, however, undermine the needs to develop the current methods and implementations, but does indicate a need for a simultaneous cultural change.

Conclusions

The trialogical approach to knowledge creation and learning provides a good framework for considering convergence of knowledge, innovation, and practice, and developing pragmatic knowledge services. Such knowledge services are required to enable collaborative knowledge creation, support development of knowledge practices and practical implementation of knowledge, and adapt to changing needs. They can not be considered as mere information systems, but as socio-technical hybrids, including also the human actors engaging in creation and use of knowledge.

The theories of knowledge creation and learning, the advances in information systems development, and the three examples of novel knowledge services indicate that the conception of pragmatic knowledge services is feasible. Although experiences from the three examples highlight needs for conceptual and technical improvement in developing effective pragmatic knowledge services, also broader cultural changes regarding knowledge work is needed. The change from individualistic learning and knowledge creation towards trialogical collaboration can be considered as representing the shift from an information society towards a knowledge society. The current and future pragmatic knowledge services are not only means for creation of practical knowledge in their specific contexts of use, but also vehicles of this broader cultural change.

Acknowledgements

The authors would like to thank the funders of the research considered in this article; Academy of Finland, the National Technology Agency of Finland (Tekes), and EU (BENERIS; http://en.opasnet.org/w/Beneris, KP-Lab; http://www.kp-lab.org, and INTARESE; http://www.intarese.org/ projects sponsored under the 6th EU Framework Programme for Research and Development). Also special thanks to the organizers, reviewers and participants of the I-KNOW 2010 and I-Semantics 2010 conferences for their contributions to developing this article.

References

[Andersen, 08] Andersen, P., Dejer, I.: Systemic innovation in a distributed network: the case of Danish wind turbines 1972-2007. Strategic Organization 6(1):13-46.

[Ashton, 09] Ashton, J.: Politics and Gaia: Surviving the collision. Keynote speech at the Climate Change: Global risks, Challenges & Decisions, Copenhagen, 10 March 2010. http://climatecongress.ku.dk/speakers/johnashton-plenarysession-10march2009.pdf/

[Brunner, 06] Brunner, R.D.: A paradigm for practice. Policy Sciences 39:135-167.

[Chesbrough, 03] Chesbrough, H.W.: Open Innovation: The new imperative for creating and profiting from technology. Boston: Harvard Business School Press.

[Cress, 08] Cress, U., Kimmerle, J.: A systemic and cognitive view on collaborative knowledge building with wikis. Computer Supported Collaborative Learning 3:105-122.

[Delugach, 06] Delugach, H.: Active Knowledge Systems for the Pragmatic Web. In PragWeb 2006: Proceedings of the First International Conference on the Pragmatic Web, 21--23 September 2006, Suttgart, Germany, 67-80.

[Edmondson, 01] Edmondson, A., Bohmer, R., Pisano, G.: Disrupted routines: Team learning and new technology implementation in hospitals. Administrative Science Quarterly 46:685-716.

[van Eemeren, 02] van Eemeren, F.H., Grootendorst, R., Snoeck Henkemans, A.F.: Argumentation. Analysis, Evaluation, Presentation. London: Lawrence Erlbaum Associates, Publishers.

[Engeström, 08] Engeström, Y.: Enriching activity theory without shortcuts. Interacting with Computers 20:256-259.

[Hakkarainen, 08] Hakkarainen, K.: Features of trialogical learning: An introduction of research and development of Knowledge-Practices Laboratory (KP-Lab). Internal working paper for the KP-Lab project.

[Hakkarainen, 09] Hakkarainen, K.: A knowledge practice perspective on technology-mediated learning. Computer-Supported Collaborative Learning 4:213-231.

[Hearst, 09] Hearst, M.A.: Search User Interfaces. Cambridge: Cambridge University Press.

[Hedegaard, 09] Hedegaard, C.: IARU Climate change congress opening remarks. Keynote speech at the Climate Change: Global Risks, Challenges & Decisions, Copenhagen 10 March 2010. http://climatecongress.ku.dk/speakers/conniehedegaard-plenarysession-10march2009.pdf/

[von Hippel, 05] von Hippel, E.: Democratizing Innovations, Cambridge, MA: The MIT Press.

[Hutchins, 95] Hutchins, E.: Cognition in the Wild. Cambridge, MA: The MIT Press.

[IARU, 09] IARU (International Alliance of Research Universities): Synthesis report from Climate Change: Global Risks, Challenges & Decisions, Copenhagen, Denmark 10-12 March 2009. Copenhagen: University of Copenhagen. http://climatecongress.ku.dk/pdf/synthesisreport

[Inkinen, 07] Inkinen, A.: Whose Knowledge? The role of participation in environmental decision-making. NIBR Working Paper 2007:115 Papers from the 8. Nordic Environmental Social Science Research Conference June 18-20 2007 Workshop 5 (pp. 117-132).

[IPCC, 07] Intergovernmental Panel on Climate Change (IPCC): Climate change 2007: Synthesis report. Geneva: IPCC. http://www.ipcc.ch/pdf/assessment-report/ar4/syr/ar4_syr.pdf

[van Kerkhoff, 06] van Kerkhoff, L., Lebel, L.: Linking knowledge and action for sustainable development. Annual Review of Environment and Resources 31:445-477.

[Knorr-Cetina, 07] Knorr-Cetina, K.: Culture in global knowledge societies: knowledge cultures and epistemic cultures. Interdisciplinary Science Reviews 32(4):361-375.

[von Krogh, 09] von Krogh, G.: Individualist and collectivist perspectives on knowledge in organizations: Implications for information systems research. Journal of Strategic Information Systems 18:119-129.

[Lakkala, 09] Lakkala, M., Paavola, S., Kosonen, K., Muukkonen, H., Bauters, M., Markkanen, H.: Main functionalities of the Knowledge Practices Environment (KPE) affording knowledge creation practices in education. In C. O'Malley, D. Suthers, P. Reimann, A.Dimitracopoulou (Eds.), Computer Supported Collaborative Learning Practices: CSCL2009 Conference Proceedings (pp. 297-306). Rhodes, Creek: International Society of the Learning Sciences (ISLS).

[Latour, 05] Latour, B.: Reassembling the social: An introduction to Actor-Network Theory. Oxford: Oxford University Press.

[Markkanen, 08] Markkanen, H., Holi, M., Benmergui, L., Bauters, M., Richter, C.: The Knowledge Practices Environment: a Virtual Environment for Collaborative Knowledge Creation and Work around Shared Artefacts. In Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2008 (pp. 5035-5040). Chesapeake, VA: AACE.

[May, 09] May, C., Finch, T.: Implementing, Embedding, and Integrating Practices: An Outline of Normalization Process Theory. Sociology 43:535-554.

[McNie, 07] McNie, E.C.: Reconciling the supply of scientific information with user demands: an analysis of the problem and review of the literature. Environmental Science and Policy 10:17-38.

[Mickwitz, 09] Mickwitz, P., Aix, F., Beck, S., Carss, D., Ferrand, N., Görg, C., Jensen, A., Kivimaa, P., Kuhlicke, C., Kuindersma, W., Máñez, M., Melanen, M., Monni, S., Branth Pedersen, A., Reinert, H., van Bommel, S.: Climate Policy Integration, Coherence and Governance. PEER Report No 2. Helsinki: Partnership for European Environmental Research.

[Noveck, 09] Noveck, B. Wiki Government. Brookings Institution Press.

[Paavola, 06] Paavola, S., Hakkarainen, K., Sintonen, M.: Abduction with Dialogical and Trialogical Means. Logic Journal of the IGPL 14(2):137-150.

[Paavola, 09a] Paavola, S.: Abduktiivinen argumentaatio - hypoteesien hakemisen ja keksimisen välittyneet strategiat (Abductive argumentation - Mediated strategies for searching and discovering hypotheses). A paper (in Finnish) in a conference 'Suomalainen argumentaation tutkimus', 26-27 November 2009, University of Turku.

[Paavola, 09b] Paavola, S., Hakkarainen, K.: From meaning making to joint construction of knowledge practices and artefacts – A trialogical approach to CSCL. In C. O'Malley, D. Suthers, P. Reimann, A. Dimitracopoulou (Eds.), Computer Supported Collaborative Learning Practices: CSCL2009 Conference Proceedings. (pp. 83-92). Rhodes, Creek: International Society of the Learning Sciences (ISLS).

[Pinch, 84] Pinch, T., Bijker, W.: The social construction of facts and artefacts: Or how the sociology of science and the sociology of technology might benefit each other. Social Studies of Science 14:399-441.

[Pohjola MV, manuscripta] Pohjola, M.V., Tuomisto J.T.: Openness in participation, assessment and policy making upon issues of environment and environmental health. Manuscript.

[Pohjola MV, manuscriptb] Pohjola, M.V., Leino, O., Kollanus, V., Tuomisto, J.T., Gunnlaugsdottir, H., Holm, F., Kalogeras N., Luteijn, J.M., Magnússon, S.H., Odekerken, G., Tijhuis, M.J., Ueland, Ø., White, B.C., Verhagen, H.: State of the art in benefit-risk analysis: Environmental health. Manuscript.

[Pohjola MV, 10] Pohjola, M.V., Tuomisto, J.T., Tainio, M.: Properties of good assessment: The factors that contribute to assessment performance. A paper in SRA-Europe 2010 Annual Meeting, 21-23 June 2010, King's College, London. http://www.sraeurope.org/filedown.aspx?file=3778.

[Pohjola P, 09] Pohjola, P.: Technical Things. An essay on the social constitution of technical artefacts. Cologne: Lambert Academic Publishers.

[Pohjola P, 10] Pohjola, P.: The Pragmatic Web: Some Key Issues. In: A. Paschke, N. Henze, T., Pellegrini and H. Weigand (Eds.), Proceedings of the 6th International Conference on Semantic Systems and the 5th International Conference on Pragmatic Web, ACM, DOI= http://doi.acm.org/10.1145/1839707.1839745.

[Rabardel, 03] Rabardel, P., Bourmaud, G.: From computer to instrument system: a developmental perspective. Interacting with Computers, 15(5), 665-691.

[Sander, 09] Sander, H.: The spoken word counts. Keynote speech at the Climate Change: Global Risks, Challenges & Decisions, Copenhagen, 10 March 2010. http://climatecongress.ku.dk/speakers/helgesander-plenaryspeaker-10march2009.pdf/

[Salomon, 93] Salomon, G. (Ed.): Distributed Cognitions: Psychological and Educational Considerations. Cambridge: Cambridge University Press.

[Schoop, 09] Schoop, M., de Moor, A., Dietz, J.: The Pragmatic Web: A Manifesto. Communications of ACM 47. 75-76

[Simpson, 09] Simpson, B.: Pragmatism, Mead and the practice turn. Organization Studies 30:1329-1347.

[Tapscott, 06] Tapscott, D., Williams A.D.: Wikinomics: How Mass Collaboration Changes Everything. Toronto: Penguin Group.

[Tuomi, 03] Tuomi, I.: Networks of Innovation. Change and meaning in the age of the internet. Oxford: Oxford University Press.

[UNECE, 98] United Nations Economic Commission for Europe (UNECE): Aarhus Convention. The UNECE Convention on Access to Information, Public Participation in Decision-making and Access to Justice in Environmental Matters. http://www.unece.org/env/pp/documents/cep43e.pdf

[UNEP, 92] United Nations Environmental Programme: Rio Declaration for Environment and Development. http://www.unep.org/Documents.Multilingual/Default.asp?documentid=78&articleid=1163

Related files

<mfanonymousfilelist></mfanonymousfilelist>