Joint method development period in Kuopio in 2007

From Opasnet
Revision as of 11:56, 21 March 2007 by Mikko Pohjola (talk | contribs)
Jump to navigation Jump to search

Joint method development period was an idea from the second annual meeting of Intarese. The idea in short is to get the key method developers from different institutes in one place for a long enough period to be able to produce and finish something during that time. The joint method developement period is organized by KTL and targeted mainly to SP1 & SP4 people, although partipcipation is open to anyone interested.

This page contains information on the joint method development period and a brief description of the means and tools that are currently being used in applying the pyrkilo method in environmental health risk assessments in KTL. In the end of the page, there is also a copy of the WP1.4 plan for months 13-30. Please feel free to make major edits when necessary.

Check also these links.


Comments and discussion from the SP1 leaders from London meeting 20.3.

Introduction (Erik)

Comments from David

  • Where do policy options come from?
  • Some WPs are looking at positive effects as well. These might not fall on source-effect causal chain.
  • Two ways to see what we'll do.
    • Everything is forced into full chain approach
    • If the case does not match the approach, do something else, but there is a need for guidance what to do then.
      -->Matti: Everything is layered, and these layers may be express conflicting conclusions for a certain policy.
      David: I don't see this layering as a major problem; each WP will start from some layer that is natural to them.
      Jouni: All diagrams should be based on causal connections, mind maps etc. should not be used. This is the basic guidance.
      Anne: In the workshop we started from the basic diagram but then squeezed it a bit to fit the case. We need a tool for doing the diagrams.
  • Marco: It's funny to hear that some think full chain approach (FCA) is a narrow thing. One thing is to use variables with measurable quantities. I don't see a problem in including positive AND negative effects.


  • David: Some WPs are doing rigid things, some more flexible. Does this group have guidance on how rigid they should be?
    Matti: FCA should be used whenever possible.


  • Jouni: We should look at SP3 diagrams in practice and not talk about this in theory.
  • David: Diagrams have been there for 6 months
  • Gerard: SP3 is waiting for comments, and nobody gives it. There is a lack of collaboration.
  • Erik: People already had their mindmaps when the projects started, and they are unwilling to give that up.
  • Irene: It's confusing. Some say you must start from certain ends, I think you can start from either end of FCA and go up or down.
  • Erik: We selected 7 cases bases on stressors, not health effects.
  • David: Yes, and many in SP3 don't like that, because they want to focus on health outcome instead of a stressor.
  • David: SP3 wants to play the game, but they have not been able to get advice and ask for further help although they know they need it.


  • Erik:Do we need FAQs and also less frequently asked questions?
  • David: Most of the problems are in the overall concepts
  • Erik: This is to blame me, because the framework is missing.
  • Martin: Are the issues captured in the guidance documents?
  • Marco: Workshop emailings: many had in their heads ideas about scoping. What questions can we really tackle with this approach? Housing/energy effiiciency: does it work? We should realise that the approach can be good even if it does not work with a specific case.
  • Anne: USTUTT is working on glossary about the major issues, including "integrated".
  • David: Let's make a statement: These issues will be closed by the training workshop. We need guidelines.
  • Erik: I'm afraid that we will give too general guidance, which does not help.
  • Matti: We're providing guidance and models in WP1.2.
  • David: SP3 is already half way doing their job. They'll hate it to be guided afterwards.
  • Gerard: We need documents like Uncertainty document.
  • David: I agree. There are some general questions that need initial solution now.
  • Erik: We need to get contact to SP3 leaders before the training workshop.
  • Gerard: I disagree, we need good documents and not meetings.
  • Erik: HIA included benefits, RA only risks. Intarese contains both but does not force HIA.


  • Erik: There is an issue on who will define the policy scenarios in a case.
  • David: There are examples of cases when this has been done well or badly. Key readings: Chris Murray (this is in internet [or intranet?]).


  • David: We are going to SRA-E with speakers from SP1 and SP3. Send basic information.

Wiki environment and workshop outcomes

Jouni presented the results of the workshop and suggested the use of Mediawiki. This was strongly opposed and considered as an irrelevant issue in this meeting. The focus should be on the contents instead.


  • Matti: Before the process, two steps:
    • Political framing about what is the basic question (experts participate).
    • Scientific framing about the key issues within the boundaries. This changes only when someone comes up with strong arguments to expand.
  • David: I agree but there must be open participation also.
  • Matti: I agree, I just didn't mention it.
  • Martin: May not happen in this order: experts write a draft, stakeholders participate, then fixed by politicians.
  • Erik: Science can help in framing.


  • Erik: Framework has been circulated between David, Erik, Irene, Anne. (How-to-do type paper). This will circulate more widely soon.
  • David: Causal graphs from SP3 will be updated soon (next week).

Key issues in WP1.X

3 Key things in WP1.4

  • Connections to the framework; finishing the workshop report
  • Finishing methodology reports to Wiki
    • Monte Carlo guidance
  • Method development
    • Stakeholder involvement
    • QALY/DALY, monetarisation
    • Templates vs. specific
  • --> David: This should be elaborated with SP3
    • Stakeholders: we could have a breakdown session at the training workshop.
    • Erik: Should we prioritise these to get something in the training workshop.


  • David: there is a heated debate about how you can combine tox data (in general).


Marco: High-resolution data and its use?

  • Matti: There are several uses.
  • Erik: Why couldn't we use high-resolution data with general dose-responses. It should be possible.


  • Anne: It is a general issue of multiple data/no data/not exactly applicable data (or models)
  • --> this goes to 1.3 to think about


Valuation and weighing

  • Martin: We need to standardise the process of deriving values, not the standards for weights.
  • Erik: Yes, but there is some previous knowledge about weights and we can use that.
  • David: There have been lots of studies in valuation studies. SP3 may want to just take a value from a shelf.


Marco: 3 key issues in WP1.5

  • Uncertainty
    • Tools for practical implementation (is this MNP?)
    • Training session for autumn (probably October)
  • Mixture exposure, vulnerable groups, ethical value systems, equity
    • We have had problems with the level of generality: these have not been practical enough.
    • Not far enough to provide informative output in the training workshop
  • Economic valuation...
  • -->David: We must say something on these issues in ithe workshop
  • Anne: This comes back to issue what to do when the data/model does not fit.
  • Gerard: Mixture issue is included in WP1.3 protocol - good place for WP-communication.
  • Matti: We could have a walkthrough of these issues in May with one of the SP3 cases.
  • David: MNP tool was not welcomed in Copenhagen meeting - lot of filling in without much of useful output.

Training workshop

  • Marco: Previously they have been general presentations. Do we want to have some hands-on issues
  • Erik: Yes.
  • Anne: Maybe some plenaries and enough time on a few case studies.


Stakeholders:

  • On the first pass, we should test at least on some cases this.
  • Martin: van den Sluijs has a manuscript on stakeholder involvement. We should get him involved.


  • Erik: Practical interaction, not lecturing
  • For SP3 mainly: leaders and deputies, but also others
    • SP3 should be able to give presentations
  • 22.-23.5.2007 CEFIC, Brussels starts early (arrive previous day)
  • David: How to make SP1 to learn from SP3?
  • Anne: Send beforehand list of possible trainings, and people can vote
  • Martin: Make 3 slides per case but ask: what are the uncertainties, what are the key issues,...
  • Gerard: We need an active workshop, but we should work in one group instead of split up.
  • Erik: SP3 must be able to prepare in advance. Not go through every step.
  • David: I can extract comments form SP3 within the next month.
  • Erik: Stakeholder involvement report came out a few months ago from UK
  • David: CDC has a 200-page document + software
  • Erik: Airport case study could be used + SP3 cases
  • Anne: Some WPs could test some methods beforehand
  • David: It's not their job to do it now but during the next 2 a.
  • Gerard: We can go through meta-analysis calculations in 30 min
  • Anne: I can do DALYs
  • Erik: We must 1) be reacquanted to the WPs (only briefly) 2 go through cases 3 how this reflects my case
  • By 27.3. Send email to David about a suggestion of items for the workshop

18-month deliverables

  • DL in end or April, 15.4. final draft circulating within SP1, 1 week for comments, 1 week for final edits.

Method development period

We welcome everyone interested in these methods to Kuopio for a two-week workshop in March 12 to 23. The participating institutes (and their representatives) are RIVM (Anne, SP1), USTUTT (Alex, SP4), NILU (Sjur WP4.1), UU (Hanna WP1.3), IC (Clive) and KTL (Risk analysis group WP1.4 + air hygiene lab WP1.2). In addition Mari Vanhatalo from University of Helsinki / EVAHER project will participate in the workshop for a couple of days.

The aims of this workshop

  • to familiarise everyone to the tools that are being used and developed for risk assessment in the participating institutes
  • to identify possible overlaps, gaps, and interface mismatchs, and try to find a reasonable solutions to these
  • to work together on a specific case study in practice with the existing tools
  • to gain practical experience on the tools and methods and identify development needs
  • to write a report about what we learnt for internal use in Intarese (or even for external use?)


The picture attempts to represent an outline of the available methods and tools to be used in the workshop case study. Comments, additions, corrections etc. are very welcome.

The structure of the workshop

We will be working on a practical case attempting to make an integrated risk assessment using the methods and tools we have or are developing within Intarese. The case study topic will be airports (air pollution+noise; air traffic+surface traffic).

The workshop participants could be roughly grouped e.g. as follows:

  • core team: Jouni, Mikko, Anne (12.3.-19.3.), Alex (16.3.-23.3.), Hanna (12.3.-19.3.), Sjur (12.3.-15.3.), Clive (14.3. evening - 19.3. morning), Mari (15.3.-16.3.)...
  • KTL team: Marko, Olli, Anna, Virpi, Eva, Miranda, Markku, Juha V (system support), Aleksi, Einari, ...
  • local leaders: Matti, Juha P
  • remote leaders: David, Erik, Marco, Gerard, Rainer, Aasmund, ...

The core team will be working full-time on the case study during the whole workshop period, or at least the time of their presence in Kuopio. KTL team will also strongly take part in making the assessment as well as meetings and discussions, but only a proportion of their daily work time will be required to be spent on working on their responsibility areas of the case study. The local leaders will be welcome to attend every part of the workshop, especially the early phases, but no individual particular tasks will be allocated to them.

The remote leaders will be incorporated in the process by preliminary e-mail discussions on the case topic, a teleconference meeting on the second day of the workshop, access to follow-up on the Intarese-wiki, an intermediate report delivered to the SP1 meeting (+ feedback on it) and final reporting on the workshop. The lessons learned will be also presented in the SP4 meeting coming up in late March or April. Participation to daily morning meetings via phone is also possible, please inform about your interest in advance on this to the list of meetings below.

organization of the participants

Name Seat Computer Apartment Phone number
Anne room 259 (with Juha V) desktop available Katiska A 5/1 +358-17-201487
Hanna room 260 (with Virpi) desktop available Katiska A 5/2 +358-17-201479
Sjur room 257 (with Mikko) brings own laptop Katiska A 5/3 +358-17-201347
Alex room 257 (with Mikko) I will bring my own laptop Katiska A 5/3 +358-17-201347
Clive room 273 desktop available Katiska A 3/1
Mari room 256 (with Marko) brings her own laptop own accommodation +358-17-201485

Katiska = student apartment building, Katiskaniementie 6, 70700 Kuopio (Rauhalahti)


Working-hours per person

ADD YOUR OWN HOURS INTO THE TABLE BASED ON WHAT YOU CAN CONTRIBUTE TO THE WORKSHOP

Name Mon Tue Wed Thu Fri Mon Tue Wed Thu Fri
Jouni 7 h 6 h 5 h 6 h 4 h 7 h - 3 h 3 h 5 h
Mikko 7 h 7 h 7 h 4 h 4 h 6 h 7 h 7 h 7 h 7 h
Anne element element element element element element - - - -
Hanna element element element element element element - - - -
Sjur 2-3h 7h 7h 3h - - - - - -
Alex 1-2 h 2-3 h ca. 2 h - (travelling) ca. 7 h ca. 7 h ca. 7 h ca. 7 h ca. 7 h ca. 7 h
Clive - - element element element element - - - -
Mari - - - element element - - - - -
Marko element element element element element element element element element element
Olli element element element element element element element element element element
Anna 4 h 4 h 4 h 8 h 4 h 6 h 6 h - 4 h 4 h
Juha V element element element element element element element element element element
Virpi element element element element element element element element element element
Marjo element element element element element element element element element element
Einari element element element element element element element element element element
Aleksi element element element element element element element element element element
Eva element element element element element element element element element element
Miranda - - - - - element element element element element
Markku element element element element element element element element element element
Total working hours element element element element element element element element element element


The schedule & meeting minutes

Tuesday 13.3.

Morning meeting 9:30 Minutes

Remote participants: Alex (+49 711 685-87838)

The planned teleconference at 13:00 was eventually cancelled due to technical difficulties. We got some comments on phone from Erik and they can be found at the case study page along with some discussion.

Wednesday 14.3. Morning meeting 9:30 Remote participants: Alex (+49 711 685-87838)

Discussed issues:

List of methods, tools and programs that we have and what we would like to have:

  • Scoping tool
    • needed
    • Describe functionalities
    • Demonstration
  • Stakeholder + communication:
    • Selection of stakeholders?
    • Meetings
    • Mediawiki
  • Uncertainty
    • MNP UNC.Guide (reminder for each variable)
    • WP1.3:checklist
    • Monte-Carlo simulation, Bayesian statistics
    • Sensitivity analyses tools (Frey et al. 200? article)
  • Perception
    • Online tool
  • Presentation and communication
    • Guidance exists (EPA?)
    • Structured reports (also other parts than the final report)
  • Expert judgement
    • Selection of experts
    • Expert elicitation (excalibur)
    • WP1.3 (and WP1.5)
  • Value of information
    • EVPI, EVPXI and so on
    • Could answer to question:How far you should go?
  • Indicators
    • Guidance in developing
    • Need for improved DPSEEA
    • Use of WHO indicators: needs work
    • What is the difference between indicator and variable? (Mikko:indicators are variables with special interest)
    • SP3?
  • Combination of data
    • Meta-analysis methods for epi and tox data


Thursday 15.3.

Morning meeting 9:30

Agenda/highlights of discussion:

  • Introduction of what has been going on to the newcomers (Patrycja, Clive, Mari)
    • Patrycja to present her BBN model in friday morning meeting
  • MNP uncertainty guide demo (Anne, Sjur)
    • the tool can be adapted to meet Intarese needs
  • Intarese demonstrator demo (Sjur)
    • an updated demonstrator to be done later this year
    • this workshop should provide a rough suggestion of the needed toolbox functionalities
  • tools needed in Intarese toolbox
    • participants to have a look and edit the page - will be discussed later in more detail
  • contents of variable, input and method pages
    • serious effort to making the case required thursday and friday
    • limitations of night-time noise chosen as the only policy option to be considered


Friday 16.3.

Morning meeting 9:30 IN B-WING MEETING ROOM

Agenda/minutes:

  • What to present in SP1 meeting next tuesday?
  • Getting on with the case assessment
    • emphasis on identification of linkages between variables and methods/tools
    • noise policy case as the essential part of the case study assessment
  • Patrycja's BBN model presentation was given in the afternoon 15:00


Weekend 17.3.-18.3.

Monday 19.3.

  • Morning meeting 10:30, essentials:
    • framework, methods and tools -tables appear to be the most important output of the workshop so far
    • assessment workspace concept seems to capture effectively the primary needs for the Intarese toolbox
    • the case study assessment has been useful in coordinating our discussions about the framework methods and tools, but does not appear to be producing hardly any direct results
    • the main points to be written out intermediate report page for SP1 meeting

Tuesday 20.3.

  • Morning meeting 9:30, essentials:
    • continue working on the tables
    • contribute to development/description of the assessment workspace
    • case study work can be frozen for the time being at least until we get the SP1 meeting feedback

Wednesday 21.3.

  • Morning meeting 9:30
Agenda:
*feedback from SP1 meeting
*decision of focusing efforts on last 3 days of workshop
*what kind of report (and other output for general presentation) to produce?
  • assessment work goes on...

Thursday 22.3.

  • Morning meeting 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE
  • Alex's brief presentation about health impact valuation + discussion
  • discussion/meeting about WP1.4 deliverables due end of April (Jouni, Alex, Mikko, ...)
  • assessment work goes on...

Friday 23.3.

  • Morning meeting 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE (in 2nd floor lecture hall)
  • Preparing a report on the workshop

Issues to be resolved

What are the methods that will be recommended for case studies?

What are the tools and software that will be recommended for case studies?

Which methods and tools will be included as parts of the Intarese general method?

Will the Intarese general method and its products be totally open access (General Public Licence GPL)?

Pyrkilo interface

In this section, we describe the different tools that we have been developing in KTL and within ERAC (Environmental Risk Assessment Center, Kuopio. ERAC is a joint effort of KTL, University of Kuopio, and National Geological Survey of Finland). We have NOT added other programs and tools to this platform, such as the demonstrator of the current toolbox, or the uncertainty program by MNP. This is not because we wouldn't think they aren't important. They are. But we thought it is better to first show what we have and what we don't have in our institute, and only then add other things to the platform.

Overview

The purpose of the platform is to offer all functionalities that are needed to perform an and publish its results. This includes issue framing, drafting the model and variables, collecting data, estimating the values and distributions for the variables, evaluating stakeholder preferences, computing the models, storing the results, and displaying the results to the endusers. Several people and groups of people would use this platform. Depending on his/her role, a person can participate in several different phases of the risk assessment process and contribute to several different ways by offering understanding, opinions, and information. There are many user interfaces to deal with the many tasks that occur during a risk assessment process. The technical details are destribed below in more detail. The methodological issues are described in a manuscript about pyrkilo method, and a draft manuscript about efficiency issues related to the method.

File:Pyrkilo platform.PNG

Programs used in the platform

The platform consists of several programs.

MediaWiki

MediaWiki is the central program in the platform. It is the same program that is used to run Wikipedia, the open encyclopedia. It is basically a content management system with an internet-based, user-friendly interface that allows a number of people to work on a set of documents simultaneously. Its major properties are

  • a strict version control
  • a simple coding and formatting system
  • user identification
  • contributions released immediately to everyone to read
  • possibility to attach discussions of the content. Each page has a discussion page for this purpose.
  • good working environment for text and figures (also tables, although a bit more complicated)
  • good categorisation and search tools for pages
  • possibility to use templates (a block of content that appears the same way on several pages)


Mediawiki is not especially good at (although it can handle these)

  • copy-pasting contents back and forth from one program to another (formatting problems)
  • storing ready-made documents (not a file management system)

Discussion on content management programs

How to read discussions

Fact discussion: .
Opening statement:

Closing statement: Resolution not yet found.

(A closing statement, when resolved, should be updated to the main page.)

Argumentation:

←--': . B) is a good choice. (type: truth; paradigms: science: defence)

⇤--1:: . no version control, people easily drop out of email lists. --Jouni 10:32, 16 January 2007 (EET) (type: truth; paradigms: science: attack)


Risk-assessment-related functionalities in MediaWiki:

  • Main namespace is an area that contains article-like descriptions of risk-related issues, such as risk assessments, method descriptions etc.
  • Variable namespace contains more structured contributions in the form of. This namespace
    • describes variables and their attributes
    • describes causal links between variables
    • Describes the discussions related to the content of a variable on its Talk (or discussion) page.
    • describes the value judgements related to outcomes or other variables.
    • describes smaller pieces of data (and gives links to larger pieces) and describes how the data was used to derive the estimate of a variable.
    • describes rank correlations between variables (possibly using vine copula method).
    • describes set-item relationships between items. This means that a variable may inherit properties from a more general variable of the same kind.
    • describes other non-causal relationships between variables.
  • Model namespace contains Analytica and other model files. There can be translated into variables in the Variable namespace or updated based on existing variables.

Analytica

See Monte Carlo simulation.

Result distribution database

Result distribution database is an idea of an SQL database for result distributions of the variables. The main idea is that the variables and their distributions are program-independent in this environment. If their precalculated result distributions are stored in a database, the risk models can be analysed based on these results independent on which program was used to produce the results. In addition, if the model runs are done in a coherent way, these distributions form a large joint distribution that can be used to conditionalising, backward inference, and optimising, which are usually difficult tasks.

The main properties are

  • a large SQL database where each variable forms a table
  • each table has a fixed number of rows, which equals to the number of simulations used to calculate the model (in the pilot version, this could be in the order of 5000, but in the real database, it should be in the order of 1,000,000 to 10,000,000.
  • tables indexed by predefined fractiles to speed up the performance


Major problems

  • needs constant updating and simulation (requirements for the hardware)
  • When the structure has been fixed, all models must have the same number of simulations irrespective of model size.

Alternative programs could be

  • Netica, a commercial program for handling joint distributions based on sample files.

Other possible programs

These programs do not exist in KTL, and there is not (yet) active development going on related to these. However, they could be potentiallly useful programs to attach to the platform.

File management system could assist the risk assessment by providing a centralised database of original data that could be used in the modelling.

Value evaluator could be a web-based questionnaire where stakeholders could rank different outcomes or events in the order of their personal preference. These could then be synthesised and used in the valuation process of risk assessment outcomes.

Expert elicitator could be a web-based questionnaire where experts could give their best estimates on various variables. The answers could be synthesised based on expert-specific weights that would be determined based on their previous performance in this task. There are existing methods to do this using extensive interviews, but it has never been tried over internet.


Interfaces needed to get the platform running

Mediawiki and the user interface exists already, as it is a major part of the Mediawiki program. It is continuously developed by a large Mediawiki community. KTL has a specialised ICT person to update and develop Mediawiki used in pyrkilo risk assessments. We have several projects that constantly utilise this interface, and experience is increasing rapidly.

Ana-Wiki interface is about 1) translating Analytica model files into Mediawiki pages as variables, and 2) updating Analytica model files based on the updated contents in Mediawiki variable pages. This development is ongoing in the Beneris project and these interface tools should be available during spring 2007. We are currently testing a pilot version of Ana-Wiki converter.

Database interface is for converting Analytica (or other simulation program, such as R) results into the database. Analytica has functionalities for input/output from/to SQL databases, so this should not be a huge task. However, we do not have practical experience on this. We are actively testing this functionality, and we should have more experience before February 2007.

Result interface is for showing results from the database to the endusers. Although this is a critical thing and should be developed carefully, it can be postponed to a later stage when the other parts are running. However, some kind of pilot interface should be developed rather early so that researchers in the project can test the database and its functionalities.

Data interface, Elicitation interface, and Evaluation interface are only needed if these tools are developed further. These are not crucial thing at the moment.


  • Draft Planning for Next 18 Months (1 Nov 06 – 30 April 08) was removed because it was not the final version. To see the draft, click here.
Retrieved from "https://en.opasnet.org/index.php?title=Joint_method_development_period_in_Kuopio_in_2007&oldid=12783"