Open management trial

From Opasnet
Jump to: navigation, search

Open management trial is a research study type that aims to randomise policy actions into open and non-open groups and then follow the impacts of this intervention.


What is a good study design for an open management trial?


It is good to start from a policy program that contains at least dozens of separate actions. An example is op_fi:Hiilineutraali Helsinki 2035.

  • Each action gets a follow-up page in Google Docs or similar web-workspace. Each page contains the same structure and the relevant information about each action.
  • Pages are randomised into two groups: open group is opened for everyone to read and comment, possible even edit; the closed group is visible only to the dedicated group of authorities who are responsible for the respective action. If there is an action that must be opened or kept closed, it is removed from the randomisation and subsequent analyses but is otherwise treated like any other action.
  • A key part of the trial is to follow different kinds of information produced about the actions. These include but are not limited to
    • Follow-up information of the action performed by the responsible authority.
    • Public discussion about the action.
    • Acceptability of the action among the public, as measured by content analysis of discussions, surveys, or other means.
    • Occurrence of issues raised in the discussions when looked from the decision-making materials such as the minutes of councils and committees responsible for implementing the action.
    • Impacts on the explicit objectives of the action. e.g. climate emissions after mitigation actions.
  • The actions should not be treated as separate pages but rather as a network of issues that jointly form the policy program. Therefore, different issues must be connected to the relevant actions, understanding that one issue may link to several actions.
    • Rationale for the action.
    • Objectives of the action.
    • Estimated and realised costs of implementing the action.
    • Plan for evaluating the progress of the action, and results from the evaluation as implementation proceeds.
    • Public discussion about the action or a related issue.


The network of issues described above should be described in a Wikibase. However, at the moment there are no technical facilities for that.

The method of open management trial could be extended beyond policy programs. This can be done by letting people in an organisation list and describe actions that could be analysed in a way described above. Those actions that fulfill the study criteria would be randomised into open or closed group and followed in the trial. The recruitment of actions could be done on a continuous basis. This is not necessary with policy programs that have dozens of actions starting at a single point in time.

A key point in such a trial is the continuous evaluation and management of the actions, in aim to improve their implementation and effectiveness. The randomised groups of open and closed evaluation processes could tell how much openness affects the work process and eventual outcomes.

Open statistical analyses could help in studying the results of a trial. Open statistical analysis means that the data to be studied is published or openly described in detail, so that it is possible to design statistical analyses without seeing the (sensitive) data itself. Github or another platform is used to co-create analysis codes.

See also

Open tools for implementing and evaluating city policies: