Open assessment method
| Moderator:Nobody (see all) Click here to sign up. |
|
|
| Upload data
|
This page is about the specifics of the open assessment method. To read a general description, see open assessment.
- Open assessment (previously also known as pyrkilo) is a method that attempts to answer the following research question and to apply the answer in practical assessments:
- How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?
- Open assessment can also refer to the actual making of such an assessment (precisely: open assessment process), or the end product of the process (precisely: open assessment product or report). Usually, the use of the term open assessment is clear, but if there is a danger of confusion, the precise term (open assessment method, process, or product) should be used. In practice, the assessment processes are performed using Internet tools (notably Opasnet) among traditional tools. Stakeholders and other interested people are able to participate, comment, and edit its contents already since an early phase of the process. Open assessment is based on a clear information structure and scientific method as the ultimate rule for dealing with disputes. R↻
Question
How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?
Answer
- There must be a clear information structure.
- To operationalise the information structure, there must individual objects that each belong to a group of objects defined by a universal object. For discussion about what these objects should be, see Universal object.
- All objects must be subject to the scientific method. The scientific method is a method of discovering knowledge about the natural world based in making falsifiable predictions (hypotheses), testing them empirically, and developing peer-reviewed theories that best explain the known data (Wiktionary).
- To operationalise the scientific method, pragma-dialectical argumentation theory is applied for discussions.
- There must be a website, [Opasnet structure|Opasnet]], which contains the functionalities needed by the method and the contents produced by the method.
Rationale
- Tapscott and Williams provide insight into why mass collaboration work is a better way of producing information products than traditional systems.
- James Surowiecki describes how a group of people are in some conditions better in predicting things than an expert.
- Jimmy Wales has demonstrated the power of mass collaboration in his Wikipedia.
A website is needed for collaboration about the contents of assessments. Collaboration may happen in one of many languages. Assessments contain numerical estimates and computation, and the results must be stored into a place where they can be easily accessed and downloaded for further use. In addition, a file management system is needed for data, model, and other files.
- There must be a clear and universal information structure.
- The method must be based on individual objects that have a research question. The essence of each object is to try and find such an answer to the question that holds against scientific criticism.
Critical questions (asked by CII students in 2011):
- Who decides about content?
- Is it peer reviewed?
- How do you know about the expertise of a contributor?
- How can you consider all contributions acceptable a priori?
- How can you put drafts out?
- People will learn erroneous information.
Answers: it is all about learning, not about having the perfect information available in the beginning. Also it is a process that improves. Find an example of a page that has poor scientific quality but is useful.