Obstacles and drawbacks hampering the use of Open Risk Assessment

This page is a collection of reasons (methodological, technical, practical, psychological, and so on) that from experience are known/deemed to hamper the use of Open Risk Assessment (ORA).

GOAL: By expressing the problems and recognizing the most urgent ones, this collection should be useful for aiding the further development of ORA and its wider acceptance and usage.

Suggestions for solutions will also be presented on this page.

Experiences from the Helsinki CCZ case

Practical inconvenience and additional work due to breaking apart an assessment into separate wikipages

At the starting point, the Helsinki workplan was a single contiguous (bulleted) text, making it easy to scroll and edit, to print/export the text for reports, to get a quick overview of the extent and stage of work, and to see the entire hierarchy of titles and subtitles (in the automatic contents).

Breaking the text apart into separate variables (wikipages) will complicate/prevent the above tasks, especially:

creating a need to jump back and forth between many wikipages
requiring additional work for printing/exporting the separate pages for reports
making it more laborious to estimate the stage of completion

Alleviation to #1: The "Use breadcrumbs" option in the wiki-user preferences makes it easy to return to previous pages.

Concerns about publication rights and authorship

Insofar as scientific publications are planned to be written about (or based on) this impact assessment (IA), open participation creates some worries/concerns about publication rights:

who will be the authors of the publication(s), if a large number of persons will contribute to the IA, but with greatly differing levels of contribution?
can one be confident that parts of the unpublished work (e.g. methods, structure, ideas, etc.) will not become subject to plagiarism, because of the open participation? (even when the participation is restricted, the information might spread further)

Reluctancy for revealing erroneous/preliminary work to other researchers

Due to the extent of the Helsinki IA, the novelty of most of the contents for the main author, and the limitations of time, some of the (early) contents is bound to be erroneous or poorly thought-of, making it somewhat unpleasant to reveal the contents to fellow researchers.

Partial solution: All participants need to be prepared to take a fruitful attitude for making IAs. New IAs are never complete from the start. When inviting new participants to join an IA, one must inform them about the gradually evolving nature of the work, as well as of the resource constraints. In early stages of the assessment, the presence (and even predominance) of gaps, inaccuracy, and errors should not be viewed as demerits, but rather as the natural starting point for joint elaboration. After all, this is a key argument for open participation. Furthermore, due to the interlinked requirements for data and sub-models, and the many practical uncertainties (such as the availability of data of various quality levels and from various sources), IAs often also exhibit an iterative character, which may require several rounds of revision and re-definition for some parts of the assessment. Thus, preliminary and temporary choices will likely be necessary, especially in the early stages of the assessment.

How would the formal argumentation method work in practice?

Most risk assessments of practical value are subject to a huge number of potential matters of dispute. This raises the worry about many practical problems:

simply due to the lack of time, many disputes may even remain totally unaddressed (apart from the participant that first raised the dispute)
the number of disputes may become so high that no single researcher may address all of the disputes
- ----': . Practical suggestion: is there a tool reporting all open disputes? --Erkki Kuusisto 16:46, 18 January 2008 (EET) (type: truth; paradigms: science: comment)
even when a dispute has been addressed by several participants, it may fail to be properly settled because of:
- a persistent disagreement between two or more opinions (possibly manifest as an "edit war") -> how should one deal with situations like this?
- lacking resources, resulting in disputes being dropped/forgotten/left open
some disputes may seem to be settled (due to "authority opinion"), but actually remain unsettled. This situation may result if the dispute involves known authorities/specialists in the field, possibly deterring some participants from expressing their discrepant opinion, however well thought-of.

A major weakness of the formal argumentation method may be that it relies on rational thinking, and written explication of the problem.

While rational thinking and explication are useful methods for many purposes, they are also tremendously slow as compared to the "intuition-supported decision-making". The latter is the method being used in the overwhelming majority of practical decision situations - even most of those taken by an expert. "Intuitive thinking" can rapidly incorporate vast amounts of multidimensional information, without an explicit analysis of the internal workings of this information.
In contrast, the formal argumentation necessitates "writing open" the justification of the opinion, which may amount to writing large amounts of text - taking a lot of time. -> The worry is that people may therefore prefer not to be involved in disputes, even if they would have the best opinion (and would know how to justify it).

How do computer sub-models fit into the ORA variable structure?

Before starting the conversion to ORA structure, my impression is that the variable structure, especially the "Definition" attribute, is mostly oriented towards fitting relatively simple mathematical functions.

It is not immediately clear how useful this structure is for large, dedicated (often commercial) computer models, such as a traffic model and dispersion models. However, such models constitute the major part of the Helsinki-case model chain.

Conclusion: Need to test this.