Open assessment workshop 2009: Difference between revisions
m (→Schedule) |
(→Daily program: Planned contents, objectives and related material: friday exercises) |
||
Line 148: | Line 148: | ||
====Monday: Introduction, motivation, and basic concepts==== | ====Monday: Introduction, motivation, and basic concepts==== | ||
Morning lecture: '''[[Open assessment in research]] | |||
Objectives: | |||
* | * learn how traditional articles have two distinct parts | ||
* | * Learn how these parts can be organised in a better way | ||
* | * Become exposed to the idea of scientific method. | ||
* | * Identify what is the use of science in policy assessments. | ||
* | * Learn, for a piece of information, you know what is a good object type for it. | ||
* | * Learn to see the world as a collection of information pieces. | ||
* | * See your own work/research as a part of a global mass collaboration project. | ||
* | * Learn that it is possible to do the whole research process (idea - research plan - execution of a study - writing articles) in Opasnet. | ||
* | * Learn that the impact of you research may be higher in an open system. | ||
* See how Opasnet can be used in a practical case study (assessment, research). | * See how Opasnet can be used in a practical case study (assessment, research). | ||
Line 172: | Line 170: | ||
*[[Peer review]] | *[[Peer review]] | ||
Afternoon lecture: '''[[Assessments - science-based decision support]], part I | |||
Introduction to | Objectives: | ||
* Learn the basics of an assessment: question/info need, and the three-part structure: decision - factors - outcome. | |||
* Learn how an information need is transferred into a research question. | |||
* Learn the importance and implications of boundaries in defining an object (assessment, variable) | |||
* Learn the roles of participants and intended users. | |||
* Learn to participate in discussions about a new assessment (Tapas) and its framing. | |||
Introduction to case assessment and exercises: | |||
* Societal decision making constrained by GHG emission reduction goals - optimization of health impacts and costs | * Societal decision making constrained by GHG emission reduction goals - optimization of health impacts and costs | ||
** international climate policy → city-level decision making → individual adaptation | ** international climate policy → city-level decision making → individual adaptation | ||
Line 185: | Line 188: | ||
* Friday: what is a good assessment (in terms of science/practical need) - quality of content, applicability, efficiency → core product, packaging, logistics and marketing, production process | * Friday: what is a good assessment (in terms of science/practical need) - quality of content, applicability, efficiency → core product, packaging, logistics and marketing, production process | ||
Objectives: | |||
* learn about the outline of the workshop week | |||
* | * know the basics of the case study | ||
* know objectives of each session | |||
* identify which exercise groups are most interesting for you → group formation | |||
Opasnet pages related to Monday afternoon topics: | Opasnet pages related to Monday afternoon topics: | ||
Line 199: | Line 200: | ||
* [[Issue framing]] | * [[Issue framing]] | ||
* [[Scoping an assessment]] | * [[Scoping an assessment]] | ||
====Tuesday: variable development==== | ====Tuesday: variable development==== | ||
Lectures: '''Variables - evolving interpretations of real-world phenomena''' | |||
Objectives: | |||
* | * Learn the basic structure of an information object. | ||
* | * Learn what variables are: independent, eternally developing descriptions of reality. | ||
* | * Learn that the ultimate criterion of a variable is the truth. | ||
* | * Learn the differences and similarities of a variable, assessment, study, and encyclopedia article. | ||
* | * Learn how to use the variable definition as your information storage. | ||
* | * Learn how to make a new synthesis when new information comes up (in Bayesian language: to update your prior). | ||
* | * Learn how to handle variables that are partially or totally overlapping. | ||
Exercises: | |||
{| class="FCK__ShowTableBorders" {{prettytable}} | {| class="FCK__ShowTableBorders" {{prettytable}} | ||
|- | |- | ||
Line 258: | Line 258: | ||
|} | |} | ||
Objectives: | |||
* Learn to build the definition based on the scope, and compute the result based on the definition. | |||
* | * Practical experience about topics you learned in the morning. | ||
* | |||
Opasnet pages related to Tuesday topics: | Opasnet pages related to Tuesday topics: | ||
Line 274: | Line 273: | ||
====Wednesday: variable development in collaboration==== | ====Wednesday: variable development in collaboration==== | ||
Lectures: '''Science necessitates collaboration''' | |||
Objectves: | |||
* | * Learn why collaboration is better than working alone. | ||
* | * Learn to apply the scientific method, and the importance of it in open assessment. | ||
* | * Learn the structure of formal argumentation. | ||
* | * Learn how to collaborate | ||
Exercises (all groups): | |||
# Choose (as a group) a variable that another group developed on Tuesday (see list of variables in the table above). Scrutinize the variable carefully and, according to your expertise and understanding, create statements and arguments about the variable content. Document the statements as well as the arguments for against your group comes up with on the discussion page of the variable following the formal argumentation structure. It is also possible to first collect comments informally, analyze the comments and then reorganize them in the argumentation format. Consider and argue in particular: | # Choose (as a group) a variable that another group developed on Tuesday (see list of variables in the table above). Scrutinize the variable carefully and, according to your expertise and understanding, create statements and arguments about the variable content. Document the statements as well as the arguments for against your group comes up with on the discussion page of the variable following the formal argumentation structure. It is also possible to first collect comments informally, analyze the comments and then reorganize them in the argumentation format. Consider and argue in particular: | ||
#* Is the scope unambiguously defined (what is the question the variable attempts to answer to)? | #* Is the scope unambiguously defined (what is the question the variable attempts to answer to)? | ||
Line 291: | Line 290: | ||
# Prepare a short presentation about the argumentation about your group's variable description invoked and how it was or should be taken into account in further development and use of that variable. | # Prepare a short presentation about the argumentation about your group's variable description invoked and how it was or should be taken into account in further development and use of that variable. | ||
Objectives: | |||
* | * Learn to use formal argumentation and to turn existing discussions into this formalism. | ||
Opasnet pages related Wednesday topics: | Opasnet pages related Wednesday topics: | ||
Line 307: | Line 306: | ||
====Thursday: assessment development ==== | ====Thursday: assessment development ==== | ||
Lecture: '''[[Assessments - science-based decision support]], part II | |||
Objectives: | |||
* become familiar with the information structure of an assessment object | |||
*information structure of assessment | * learn how solutions to assessment problems are found (sought for), the parts of assessment | ||
* | * learn how the assessment solution addresses the need and how need influences the solution | ||
* | * learn how an assessment consists of a network of individual (independent given causal relations) variables | ||
* | * learn how the information content of an assessment can be analyzed, organized and interpreted for producing the results | ||
* | |||
Exercises: | |||
{| class="FCK__ShowTableBorders" {{prettytable}} | {| class="FCK__ShowTableBorders" {{prettytable}} | ||
Line 374: | Line 372: | ||
====Friday: evaluation of assessment performance==== | ====Friday: evaluation of assessment performance==== | ||
Lecture: '''[[Evaluating assessment performance]]''' | |||
Exercises: | |||
{|{{prettytable}} | |||
! Topic | |||
! Task description | |||
|----- | |||
| Quality of content | |||
| Consider the concepts informativeness, calibration and relevance. What is their point of reference, practical need or truth? Which attributes and sub-attributes of assessments and variables they relate to ?Now choose one variable that was developed in Tuesday and Wednesday exercises. What can you say about their informativeness, calibration and relevance? Then consider the case assessment (ET-CL). What can you say about its informativeness, calibration and relevance? What implications this gives about future work regarding the variable and the assessment? Give suggestions for improving their quality of content. Prepare a presentation including a brief introduction to the concepts, evaluation of the quality of content of the variable and the assessment, and suggestions for improving the quality of content, as well as reasoning to back-up your suggestions. | |||
|----- | |||
| Usability and availability | |||
| Consider the concepts usability and availability. What is the reference point they relate to, practical need or truth? Assuming that the information content of a variable or an assessment were complete, correct and exact, what would make it usable and available or not usable or available for someone who would need that particular chunk of information? Now choose one variable that was developed in Tuesday and Wednesday exercises. What can you say about its usability and availability given the purpose and users defined in the scope of the case assessment (ET-CL)? Then consider the case assessment. What can you say about its usability and availability given the purpose and users defined in the scope. What implications this gives about future work regarding the variable and the assessment? Give suggestions for improving their usability and availability. Prepare a presentation including a brief presentation of the concept, why is it an important aspect of assessment performance, evaluation of the usability and availability of the variable and the assessment, and suggestions for improving the usability and availability, as well as reasoning to back-up your suggestions. | |||
|----- | |||
| Acceptability | |||
| Consider the concept acceptability and its disaggregation to acceptability of premises and acceptability of process. Which attributes and sub-attributes of variables and assessments does acceptability relate to? Now choose one variable that was developed in Tuesday and Wednesday exercises. What can you say about its acceptability given the scope of the variable? Then consider the case assessment. What can you say about its acceptability given the scope of the assessment. What aspects are most crucial regarding the acceptability of the assessment? Give suggestions for improving the acceptability of the assessment. Prepare a presentation including a brief introduction to the concept and why it is an important aspect of assessment performance, an evaluation of the acceptability of the variable and the assessment, and suggestions for improving the acceptability, as well as reasoning to back-up your suggestions. | |||
|----- | |||
| Efficiency | |||
| Consider the concept efficiency. How is efficiency dependent on quality of content and applicability? What else influences efficiency of assessments? Now explore the case assessment, how it is designed and being executed, and think about its efficiency. If you were responsible for making the assessment and provided with scarce resources, what would you do in order to get the best outcome? How about if you were responsible for making a somewhat related an partially overlapping assessment, e.g. [[Bioher]], [[Claih]], TAPAS, some other, and again with scarce resources, what would you tell the manager of ET-CL in order to maximize the efficiency of your assessment. How about if you were responsible for a group of overlapping assessments, say ET-CL, [[Claih]] and [[Bioher]] for example, would it change your management decisions? Prepare a presentation including a brief introduction to the concept and why it is an important aspect of assessment performance, an explanation of how you would as an assessment manager would maximize the efficiency in the above mentioned different situations, as well as reasoning to back-up your hypothetical management decisions. | |||
|----- | |||
| Peer review | |||
| Acquaint yourself with how peer review is defined in the context of open assessment. Consider its similarities with and deviations from your prior perception of what peer review is. Which attributes of variables and assessment does peer review (in OA) address and how? Which properties of good assessments does peer review relate to? Take the position of a peer reviewer and review one variable and the case assessment. Would you accept? Prepare a presentation including a brief introduction to the method and how it is used in evaluating assessment performance | |||
|} | |||
Opasnet pages related to Friday exercise: | |||
*[[Performance]] | *[[Performance]] | ||
*[[Acknowledgements]] | *[[Acknowledgements]] |
Revision as of 09:28, 13 February 2009
This page is a encyclopedia article.
The page identifier is Op_en2582 |
---|
Moderator:Nobody (see all) Click here to sign up. |
|
Upload data
|
Open assessment workshop 2009 is the third workshop about new methods in assessments (previously more narrowly on risk assessment), organised by the National Institute for Health and Welfare, (THL), Kuopio, Finland. Its main purpose is to provide the workshop participants a broad perspective to assessments, promote the idea and practice of open assessment, as well as further develop methods to perform open assessments. The workshop has started following the needs of several EU-funded research projects, notably Intarese, Beneris, and Heimtsa. The workshop is mainly targeted to postgraduate students working in the fields of environmental health or risk assessment, but it is also open to anyone interested. If you want to join the workshop, please create a user account (by following the log in/create account link in the right-top corner of the page) with your contact information and add your username to the list on the bottom of this page. There is no deadline for registration nor a registration fee for the workshop, but early registration is highly recommended in order to help both the organizers and the participants to arrange the facilities, accommodation, programs etc.
General information
Time: | 16.2. - 20.2.2009 |
Place: | THL, Kuopio, Finland (Kuopio info) |
Organizer: | National Institute for Health and Welfare (THL), Department of Environmental Health |
See the map below to check the locations of:
- Hotel Savonia - accommodation
- Tietoteknia - lectures and exercices
- Jätkänkämppä - smoke sauna, ice swimming, traditional dinner
- Antikkala - cross-country and downhill skiing, sledding
- Kuopio passenger harbour - Skating tracks
- Kuopio railway station
- Kuopio airport
- Kauppahalli (market hall at market square) - bus stop for line 5 from city center to Hotel Savonia
- National Institute for Health and Welfare, (THL), Department of Environmental Health
You can zoom into the map and by clicking the red pins on the map, an information box indicating the name of the location will appear.
<googlemap width="375" zoom="11" type="map" lon="27.638855" lat="62.903976"> m62.893334, 27.679328, Kuopio, Finland 62.890603, 27.627397, THL 62.866252, 27.648468, Jätkänkämppä smoke sauna 62.890240, 27.635560, Tietoteknia - Lectures and exercises 62.901100, 27.651600, Hotel Savonia - Workshop accommodation 62.891659, 27.678895, Kauppahalli, bus line 5 to Hotel Savonia 63.014833, 27.792149, Kuopio airport 62.895882, 27.679882, Kuopio railway station 62.921720, 27.662373, Antikkala - ski slope, ski tracks and sledding slope 62.886549, 27.697393, Skating tracks </googlemap>
5-day weather forecast for Kuopio by the Finnish Meteorological Institute.
5-day weather forecast for Kuopio (in Finnish) by Foreca.
15-day forecast for Kuopio by Accuweather
Description
The aim of the workshop is to provide the participants a good understanding of how to make the practical information needs of societal action and creation of scientific knowledge meet by making use of the latest methodological developments and modern information and communication technology. This aim is pursued in the workshop lectures and exercises by discussing and considering e.g following questions:
- How to take the practical needs of policy into account in assessments?
- How can scientific validity of assessments be ensured?
- How to increase the applicability and enhance the use of assessment outputs?
- How should impacts be appraised?
- How to make inferencing, information synthesis and communication more effective?
- Overall, what makes a good assessment a good one?
Answers to the questions listed above are sought for by addressing them from the point of view of open assessment. After the workshop, the participants should have:
- understanding and command of concepts, methods application areas of open assessment
- Adequate command of tools applied in open assessment in order to efficiently make use of them in design and execution of assessments and use of assessment outputs
- improved expertise in the environmental health relevant topics considered in the lectures and group work exercises
In addition to everything above, we wish that after the workshop at least some of the objects worked on in the exercises reach such a high quality in their information content that they can be considered to be the most up-to-date and comprehensive source of information regarding the topic in issue.
The core of the workshop program are the practical group work exercises related to a real-life assessment case called Assessment on impacts of emission trading on city-level (ET-CL), aiming to influence the COP-15 meeting to be held in Copenhagen December 2009. The exercises are complemented with lectures and guidance aiming to provide understanding of the necessary concepts and methods as well as sufficient command of the technical tools for carrying out the exercise tasks. The guidance will be provided by workshop organizers, as well as other participants suitably experienced in the question in issue, according to demand as the workshop progresses. Lectures and group work are explained in more detail below.
Schedule
Monday, Tietoteknia 1039 | Tuesday, Tietoteknia 1039 | Wednesday, Tietoteknia 1038 | Thursday, Tietoteknia 1038 | Friday, Tietoteknia 1038 |
---|---|---|---|---|
Introduction to workshop | Lecture 3: Variables - interpretations of real-world phenomena, part I | Lecture 5: Science necessitates collaboration, part I | Lecture 7: Assessments - science-based decision support, part II | Lecture 8: Evaluating assessment performance |
Lecture 1: Open assessment in research | Lecture 4: Variables - interpretations of real-world phenomena, part II | Lecture 6: Science necessitates collaboration, part II | Social activities: Skating on lake Kallavesi | Group work: Assessment performance evaluation |
Lunch | Lunch | Lunch | Social activities: Skating on lake Kallavesi | Lunch |
Lecture 2: Assessments - science-based decision support, part I | Group work: Developing variables | Group work: Developing variables in collaboration | Lunch | Group work presentations |
Introduction to case study | Group work: Developing variables | Group work: Developing variables in collaboration | Group work: Assessment development | Workshop summary and feedback |
Practical arrangements | Group work presentations | Group work presentations | Group work: Assessment development | |
Social activities: dinner and smoke sauna | Social activities: skiing, snowboarding, sledding on [Puijo] | Group work presentations |
Morning sessions begin at 9:00. Lunch is available at Tietoteknia restaurant (at participant's own cost) 11:45 - 12:45., after which the afternoon sessions continue until approximately 16:45. Exceptions to this are Thursday when we have a later lunch time as well as delayed afternoon session due to a skating trip to lake Kallavesi in the forenoon, and Friday when the afternoon session will end already by 14:00 due to travel schedules of some workshop participants.
Tietoteknia restaurant also serves coffee and snacks. It is open 8:30 - 15:00 and lunch is served 11:00 - 13:30.
Lectures
The lectures are conducted in a form that attempt to emphasize practicality of discussed topics as well as dialogue among the lecturers and participants. The whole set of lectures comprise a discussion that continues and develops throughout the workshop week and guides the participants from considerations regarding the purpose of assessments to detailed contents of individual variables and on to a synthesis and an evaluation of overall outcomes of assessments. The lecture topics can be seen in the schedule above and the daily program description below. More detailed lecture contents can be found on a separate lecture page for each topic (follow the links to get to the lecture pages).
NOTE: The first lecture on Monday morning, Open assessment and science, is targeted also to people that do not have the possibility or interest to participate in the whole workshop. The opening lecture will consider open collaboration in scientific research, and the focus during the rest of the week will be mainly on assessments as science-based information support for societal decision making. It is thus possible to attend only the first lecture as an independent, separate part of the workshop program. It is also possible to continue to follow the rest of the workshop program after the first lecture without any prior registration.
Group work exercises
There are four exercise sessions, one every day from Tuesday to Friday, each consisting of first working in small groups on specific tasks and then presenting the outcomes of that work and lessons learned to other groups. On Tuesday and Wednesday the exercises focus on developing the contents of a chosen set of variables belonging to the Assessment on impacts of emission trading on city-level (ET-CL). On Thursday the exercises focus on certain aspects of the ET-CL case assessment. The last exercise on Friday is about evaluating the goodness of the assessment and its parts in their current form. The groups will be the same in Wednesday's and Thursday's exercises, but can be reconfigured for Thursday and Friday exercises.
The specific tasks for each group focus on one particular object, or a part of that object (a variable, an assessment attribute or group of variables located in Opasnet). The idea is to develop the information content by applying certain specified methods of open assessment. Tentative lists of objects and methods is presented below. The tasks are designed to be focused enough so that the group will be able to get a good grasp of the issue at hand, familiarize themselves adequately with the method(s) to be used, and produce some concrete output on the corresponding Opasnet pages. Methodological and tool support as well as substantial expertise will be provided according to demand by workshop organizers as well as other workshop participants already familiar with the tools, methods and issues in question.
The expected outcome of these exercise tasks consists of multiple aspects:
- improved information content of corresponding variable, assessment, method, or other pages in Opasnet
- increased understanding among the group members of both the phenomena that the objects describe and methods applied
- understanding conveyed to other groups through group work presentations
Monday: Introduction, motivation, and basic concepts
Morning lecture: Open assessment in research
Objectives:
- learn how traditional articles have two distinct parts
- Learn how these parts can be organised in a better way
- Become exposed to the idea of scientific method.
- Identify what is the use of science in policy assessments.
- Learn, for a piece of information, you know what is a good object type for it.
- Learn to see the world as a collection of information pieces.
- See your own work/research as a part of a global mass collaboration project.
- Learn that it is possible to do the whole research process (idea - research plan - execution of a study - writing articles) in Opasnet.
- Learn that the impact of you research may be higher in an open system.
- See how Opasnet can be used in a practical case study (assessment, research).
Opasnet pages related to Monday morning topics:
Afternoon lecture: Assessments - science-based decision support, part I
Objectives:
- Learn the basics of an assessment: question/info need, and the three-part structure: decision - factors - outcome.
- Learn how an information need is transferred into a research question.
- Learn the importance and implications of boundaries in defining an object (assessment, variable)
- Learn the roles of participants and intended users.
- Learn to participate in discussions about a new assessment (Tapas) and its framing.
Introduction to case assessment and exercises:
- Societal decision making constrained by GHG emission reduction goals - optimization of health impacts and costs
- international climate policy → city-level decision making → individual adaptation
- Monday: scope of assessment - what, why, for whom? → from need to assessment problem
- Tuesday: variables - individual parts of assessments
- Wednesday: mass collaboration in practice - web-based guided formal argumentation
- Thursday: definition of assessment - how and why? → suggested solution to assessment problem
- Friday: what is a good assessment (in terms of science/practical need) - quality of content, applicability, efficiency → core product, packaging, logistics and marketing, production process
Objectives:
- learn about the outline of the workshop week
- know the basics of the case study
- know objectives of each session
- identify which exercise groups are most interesting for you → group formation
Opasnet pages related to Monday afternoon topics:
- Performing an assessment
- Defining the purpose of an assessment
- Defining the users of an assessment
- Issue framing
- Scoping an assessment
Tuesday: variable development
Lectures: Variables - evolving interpretations of real-world phenomena
Objectives:
- Learn the basic structure of an information object.
- Learn what variables are: independent, eternally developing descriptions of reality.
- Learn that the ultimate criterion of a variable is the truth.
- Learn the differences and similarities of a variable, assessment, study, and encyclopedia article.
- Learn how to use the variable definition as your information storage.
- Learn how to make a new synthesis when new information comes up (in Bayesian language: to update your prior).
- Learn how to handle variables that are partially or totally overlapping.
Exercises:
Variable | Task | Examples of available data sources |
---|---|---|
PM2.5 exposure-response function | Developing the contents of the variable. Acquaint yourself with the variable concept and the structure of a variable. With the purpose of the case assessment in mind, first determine the scope of the variable and then consequently definition and result based on both the provided existing variables as data sources and your own expertise. Specific things to consider: a) how to synthesize data from different sources? b) what additional data sources could be used to improve the variable? c)why could not either of the available existing variables be used instead of creating a new variable? d) what do you think about the quality of your variable (as the best estimate for the phenomenon it describes)? Prepare to present your variable to other groups and explain how your thinking evolved in the process of developing the variable. | Existing variables Concentration-response to PM2.5 and ERF of PM2.5 on mortality in general population |
Emission factors for burning processes | Developing the contents of the variable. Acquaint yourself with the variable concept and the structure of a variable and consider the role of this variable in the case assessment. Dig out the necessary pieces of information from the provided FIRE database according to the needs of the assessment. Needed indexes and locations include:
Having done the above, consider possible other data sources that could be used to improve the variable, e.g. in terms of making the variable more suitable also for other assessments. Prepare to present your variable to other groups and explain how your thinking evolved in the process of developing the variable. |
|
Population of Helsinki metropolitan area | Developing the contents of the variable. With the purpose of the case assessment in mind, first determine the scope of the variable and then consequently consider definition and result. Check the available population data from different sources, consider their differences and make a synthesis out of them. Specific things to consider include:
Prepare to present your variable to other groups and explain how your thinking evolved in the process of developing the variable. |
|
DALY weights | Developing the contents of the variable. Acquaint yourself with DALY concept, in particular regarding how the weights are estimated. Consider how the existing variable could be improved by including uncertainties in the weight estimates, e.g. making a shift from point estimates to distributions. Improve the variable accordingly if possible. Prepare to present your variable to other groups and explain DALY weights, their role in estimating DALYs, and how including uncertainty in DALY weights affects DALY estimates. | WHO Global Burden of Disease website |
Cardiopulmonary mortality in Helsinki metropolitan area | Developing the contents of the variable. With the purpose of the case assessment in mind, first determine the scope of the variable and then consequently consider definition and result. Search for the needed information from Opasnet base and apply it in developing the variable. |
|
Objectives:
- Learn to build the definition based on the scope, and compute the result based on the definition.
- Practical experience about topics you learned in the morning.
Opasnet pages related to Tuesday topics:
- Designing variables
- Plausibility test
- Applying general information
- Result domain
- Executing variables
- Calculating health impacts
- Life table
Wednesday: variable development in collaboration
Lectures: Science necessitates collaboration
Objectves:
- Learn why collaboration is better than working alone.
- Learn to apply the scientific method, and the importance of it in open assessment.
- Learn the structure of formal argumentation.
- Learn how to collaborate
Exercises (all groups):
- Choose (as a group) a variable that another group developed on Tuesday (see list of variables in the table above). Scrutinize the variable carefully and, according to your expertise and understanding, create statements and arguments about the variable content. Document the statements as well as the arguments for against your group comes up with on the discussion page of the variable following the formal argumentation structure. It is also possible to first collect comments informally, analyze the comments and then reorganize them in the argumentation format. Consider and argue in particular:
- Is the scope unambiguously defined (what is the question the variable attempts to answer to)?
- Does the definition provide the sufficient means to answer the question adequately?
- Is the result an exact and explicit logical outcome of the definition?
- Does the result appear reasonable and meaningful?
- Go back to the variable you (as a group) developed on Tuesday and scrutinize the argumentation that another group has created about the contents of your variable. React to the argumentation by counter arguing or by improving the variable contents as seen necessary.
- Prepare a short presentation about the argumentation about your group's variable description invoked and how it was or should be taken into account in further development and use of that variable.
Objectives:
- Learn to use formal argumentation and to turn existing discussions into this formalism.
Opasnet pages related Wednesday topics:
- Open participation
- Evaluating scientific claims
- Dealing with disputes
- Discussion
- Help:Argumentation
- Index conversion function
- Merging models with different grids
- Synthesising information about continuous variables
- Using non-predetermined spatial disaggregation
Thursday: assessment development
Lecture: Assessments - science-based decision support, part II
Objectives:
- become familiar with the information structure of an assessment object
- learn how solutions to assessment problems are found (sought for), the parts of assessment
- learn how the assessment solution addresses the need and how need influences the solution
- learn how an assessment consists of a network of individual (independent given causal relations) variables
- learn how the information content of an assessment can be analyzed, organized and interpreted for producing the results
Exercises:
Exercise topic | Task |
---|---|
Meeting the needs of intended use (primary: WWViews, COP-15) | Consider the intended use, its needs and the scope of the assessment e.g. in light of the following questions:
Document your statements, arguments and comments on the discussion page, and make amendments directly to the assessment page as necessary. Also present your findings to other groups including a brief explanation of the identified need and how it is addressed. |
Answering the assessment questions - variable network | This task is about developing the definition of the case assessment in terms of the variables belonging to the assessment. Explore the list of variables, their types (decision, indicator, other) and the causal diagram representation of the assessment as well as the Analytica model. Given the scope of the assessment, consider e.g. following questions:
Document statements, arguments and comments on the discussion page, and make amendments directly to the assessment page as necessary. Also present your findings to other groups including a brief explanation of the causal network of variables and its relation to the defined scope of the assessment. |
Answering the assessment questions - BBN analysis | Acquaint yourself with Bayesian belief networks and consider how they can be used in assessments. In particular look into:
Analyze a simple Uninet model, provided to you by the workshop organizers, covering a part of the case assessment. By conditioning different variables try to find the optimal decision option. You can also update the model with new variable results found in Opasnet and repeat the analysis. You can document statements, arguments and comments on, or make amendments to, related pages in the Opasnet if relevant. Present your findings to other groups including a brief introductory explanation of how Bayesian nets can be used in the case assessment. |
Answering the assessment questions - Value of information (VOI) analysis | Acquaint yourself with the Value of information method (VOI). In particular look into:
Analyze a simple Analytica model, provided to you by the workshop organizers, covering a part of the case assessment. Calculate VOI for different variables in the model. You can also update the model with new variable results found in Opasnet and repeat the analysis. Present your findings to other groups including a brief introductory explanation of how VOI can be used in the case assessment. |
Scenarios | Acquaint yourself with the way the concept Scenario is defined in the context of open assessment. Then consider the scenarios set for the case assessment e.g. in light of the following questions:
Document statements, arguments and comments on the discussion page, and present your findings to other groups. |
Opasnet pages related to related Thursday exercise:
- Drawing a causal diagram
- Improving the result of a variable
- Bayesian network
- Value of information
- Reporting an assessment
- Operationalising large structural changes to existing systems
- Variable transfer protocol
Friday: evaluation of assessment performance
Lecture: Evaluating assessment performance
Exercises:
Topic | Task description |
---|---|
Quality of content | Consider the concepts informativeness, calibration and relevance. What is their point of reference, practical need or truth? Which attributes and sub-attributes of assessments and variables they relate to ?Now choose one variable that was developed in Tuesday and Wednesday exercises. What can you say about their informativeness, calibration and relevance? Then consider the case assessment (ET-CL). What can you say about its informativeness, calibration and relevance? What implications this gives about future work regarding the variable and the assessment? Give suggestions for improving their quality of content. Prepare a presentation including a brief introduction to the concepts, evaluation of the quality of content of the variable and the assessment, and suggestions for improving the quality of content, as well as reasoning to back-up your suggestions. |
Usability and availability | Consider the concepts usability and availability. What is the reference point they relate to, practical need or truth? Assuming that the information content of a variable or an assessment were complete, correct and exact, what would make it usable and available or not usable or available for someone who would need that particular chunk of information? Now choose one variable that was developed in Tuesday and Wednesday exercises. What can you say about its usability and availability given the purpose and users defined in the scope of the case assessment (ET-CL)? Then consider the case assessment. What can you say about its usability and availability given the purpose and users defined in the scope. What implications this gives about future work regarding the variable and the assessment? Give suggestions for improving their usability and availability. Prepare a presentation including a brief presentation of the concept, why is it an important aspect of assessment performance, evaluation of the usability and availability of the variable and the assessment, and suggestions for improving the usability and availability, as well as reasoning to back-up your suggestions. |
Acceptability | Consider the concept acceptability and its disaggregation to acceptability of premises and acceptability of process. Which attributes and sub-attributes of variables and assessments does acceptability relate to? Now choose one variable that was developed in Tuesday and Wednesday exercises. What can you say about its acceptability given the scope of the variable? Then consider the case assessment. What can you say about its acceptability given the scope of the assessment. What aspects are most crucial regarding the acceptability of the assessment? Give suggestions for improving the acceptability of the assessment. Prepare a presentation including a brief introduction to the concept and why it is an important aspect of assessment performance, an evaluation of the acceptability of the variable and the assessment, and suggestions for improving the acceptability, as well as reasoning to back-up your suggestions. |
Efficiency | Consider the concept efficiency. How is efficiency dependent on quality of content and applicability? What else influences efficiency of assessments? Now explore the case assessment, how it is designed and being executed, and think about its efficiency. If you were responsible for making the assessment and provided with scarce resources, what would you do in order to get the best outcome? How about if you were responsible for making a somewhat related an partially overlapping assessment, e.g. Bioher, Claih, TAPAS, some other, and again with scarce resources, what would you tell the manager of ET-CL in order to maximize the efficiency of your assessment. How about if you were responsible for a group of overlapping assessments, say ET-CL, Claih and Bioher for example, would it change your management decisions? Prepare a presentation including a brief introduction to the concept and why it is an important aspect of assessment performance, an explanation of how you would as an assessment manager would maximize the efficiency in the above mentioned different situations, as well as reasoning to back-up your hypothetical management decisions. |
Peer review | Acquaint yourself with how peer review is defined in the context of open assessment. Consider its similarities with and deviations from your prior perception of what peer review is. Which attributes of variables and assessment does peer review (in OA) address and how? Which properties of good assessments does peer review relate to? Take the position of a peer reviewer and review one variable and the case assessment. Would you accept? Prepare a presentation including a brief introduction to the method and how it is used in evaluating assessment performance |
Opasnet pages related to Friday exercise:
- Performance
- Acknowledgements
- Peer review
- Quality assurance and quality control
- Quality evaluation criteria
Practicalities
Facilities
NOTE: Lecture and exercise location has changed!!!
The main activities throughout the workshop week take place at the seminar halls 1038 and 1039 in the Tietoteknia building (see the map in the beginning of the page to check the location). There will be a wireless internet connection available for all participants. Each participant is recommended to bring along a laptop computer. If this is not possible for you, please inform the organizers about this and we will see what we can do about it.
Remote participation
If there should be any people who would like to participate in the workshop, but cannot make it to Kuopio to attend the workshop in person, we try to organise a web-conference for the lecture sessions using Skype. The group work will be worked on using the Opasnet website, so also outsiders can relatively easily take part in that as well. It is also possible to join a group remotely and communicate with group members through Skype. This way you may also participate from wherever you are, as most major things - except the social program - will be made available via the Internet. Add your name on the list of remote via internet participants at the end of this page if you want to follow the workshop, but are unable to attend in person.
Note! Our Skype username is: open.risk
You can add that to your Skype contacts preferably already before the workshop
More information regarding how to participate remotely in practice will be provided here before beginning of the workshop.
Social activities
Activities arranged as part of the workshop program:
- Tuesday evening: smoke sauna, ice swimming and dinner at Jätkänkämppä
- Wednesday evening: skiing, snowboarding, sledding etc. at Puijo (Antikkala)
- Thursday forenoon: skating on lake Kallavesi exploiting the Finland Ice Marathon skating tracks
Equipment and prices for social activities:
- Bring your swimming suit if you want to try ice swimming on Tuesday (if you forget, the national sport of Finland, skinny-dipping, is also allowed). The price of smoke sauna is 11€ and dinner 19€, so in total the cost of Tuesday activities is 30€.
- On Wednesday, ski boots, skis, poles, snowboards etc. for both cross-country and downhill will be available for rental at the Antikkala lift station. Cross-country set 10€/2h, downhill ski set or snowboard 11€/2h, ski lift ticket 12€/2h. More rental price info at: Puijon Rinteet web-page. You may also bring your own equipment with you. Remember hat, gloves and warm, but comfortable clothing.
- Skating equipment will be available for rental on Thursday. Long-blade trekking skates cost 7€/2h and skating boots to fit in the skate bindings cost another 7€/2h. Also some blades that can be bound to your trekking/hiking shoes are available for 7€/2h. More rental info (e.g. helmets, elbow and knee pads) at: Leo's skating service web-page. Again hat, gloves and warm, but comfortable clothing is necessary.
It is still possible to adjust the social activities program. If you have wishes or suggestions, go ahead and let us know by writing them down on the discussion page
Please note that the social activities are voluntary and the costs of activities will be borne by the participants themselves. Participating in the social activities happens at each participant's own risk, the participants are not insured against any accidents by the workshop organizers. It is recommended that each participant makes sure before the workshop that they have a valid insurance that covers also the free-time activities during the workshop.
Examples of other activities, sites etc. available:
- Finland Ice marathon, a skating event on natural ice of lake Kallavesi 18-22 February, 2009
- Puijon laturetki - a cross country skiing event
- Tahko, a ski resort approximately 1 hour drive northeast from Kuopio
- Puijo tower, the most famous attraction of Kuopio
- Rauhalahti Spa, a spa hotel in Kuopio
- Live music e.g. at Henkka or Kuopio Music Centre
Accommodation
We have a block reservation for the workshop participants at Hotel Savonia see map. The hotel is the same as where most of the 2008 workshop participants stayed.
Room prices for workshop participants:
- Single room 80 euros/night
- Double room 96 euros/night
If you need accomodation, please (in addition to mentioning it in the registration below) inform the organizers about the desired room type and duration of your stay. If you wish to stay in Kuopio over the weekend before or after the workshop, the above-mentioned workshop room rates apply also for weekend nights.
Getting around in Kuopio
Arrival/departure by air
The distance from Kuopio airport to the city is about 17 km. There are two options to get from the airport to the hotel - bus and taxi. Kuopio airport transport connections
Bus:
Airport buses (5 €/trip) operate (by Linja-Karjala Oy) from all domestic flights, except for the flight AY519, arriving 00.25! The bus leaves from the airport approximately 5-15 minutes after the flight has arrived (the bus will wait for the passengers to collect their luggage before leaving). Get off the bus at Market Square in the center of Kuopio. From the Market Square it is possible to walk to the hotel (distance 2,4 km) or alternatively you can take a bus line 5 (2,70 €/trip) that goes by the hotel. Line 5 buses leave from the Market Square close to the Market Hall on the south end of the square. The timetable (unfortunately only in Finnish) for line 5 can be seen here.
Airport buses operate to all domestic flights. Bus to the airport leaves from Market Square (front of Anttila department store, east end of the square, Puijonkatu 19) 55 minutes before flight departure.
Taxi:
After each flight arrival there will be taxis waiting outside the airport. You may choose the airport taxi service (18 €/person), which means that the driver will wait until the car is full (in practice 2-3 persons/taxi) before leaving and then takes each passenger to their destination. Alternatively you may choose the normal taxi service (about 30 €/trip). If there are 2 people or more sharing a normal taxi it will be cheaper to take the it instead of the airport taxi service. If there are no taxis waiting at the airport you can order one by calling +358 17 106 400.
When going to the airport, make your airport taxi bookings (18 €/person) at least 2 hours before your scheduled departure by calling +358 17 106 400. The taxi will come to pick you up from agreed location approximately 1 hour before the scheduled departure time. Again, if there are 2 or more people sharing the taxi, it is also possible to order a normal taxi from the same taxi center number +358 17 106 400.
Arrival/departure by train
Kuopio railway station is located close to the center. It is possible to walk to the hotel or alternatively walk to Market Square and take bus line 5 going by the hotel. Line 5 buses leave from the Market Square close to the Market Hall on the south end of the square. The timetable (unfortunately only in Finnish) for line 5 can be seen here. Getting from hotel to the railway station happens similarly, but vice versa. You can also take a taxi to the hotel in front of the railway station.
Attendees
Please list your name and organisation here: (Requires login)
No accommodation needed:
- Jouni Tuomisto, THL, Risk research group
- Juha Villman, THL
- Päivi Meriläinen, THL
- Mikko Pohjola, THL
- Juuso Uski, Toxicology student, University of Kuopio
- Virpi Kollanus, THL
- Pauliina Ahtoniemi, THL
- Olli Leino, THL
- Marko Tainio, THL
- Pasi Sorsa, THL
Accommodation needed:
- Eva Kunseler, PBL
- Anne Knol, RIVM
- Audrey de Nazelle, CREAL
- Tek-Ang Lim, InVS
- Danielle Vienneau, IC
- Patrycja Gradowska, TUDelft
Participation via the Internet:
- Katy Walker
- Alex Kuhn
- Tam T Pham
- Adeyanju kazeem Alade, Scientific Computing Student, University of kuopio.
- Aruya Omoarebun Richard, Msc Scientific Computing, University of Kuopio.
Useful links while preparing for the workshop
- Glossary is a collection of important concepts and terms and their explanations
- See Help:Contents for guidance on using Opasnet (this site, running on Mediawiki software platform).
- We recommend participants to familiarise themselves with the following concepts:
- A free 15-day trial version of Analytica or an Analytica player for viewing the models from Lumina Decision Systems website on this site.
- Guidebook is a collection of links to theory and method pages related to the workshop.
- Lectures from workshop 2008:
- Lecture 1: Introduction to Open Risk Assessment (ORA) and the workshop (also in Finnish!)
- Lecture 2: General assessment framework
- Lecture 3: Information structure of ORA: assessment products and their relations
- Lecture 4: Information structure of ORA: defining variables
- Lecture 5: participating in assessments: argumentation
- Lecture 6: participating in assessments: moderation and quality control
- Lecture 7: evaluating assessment performance
- Lecture 8: assessing uncertainties
Keep an eye on this page, this is the official website of the workshop. It will be updated as new information comes available.