Insight network: Difference between revisions

Revision as of 04:54, 1 May 2019

This page is a knowledge crystal of subtype method. The page identifier is Op_en3861
Moderator:Jouni (see all)

Upload data {{#opasnet_base_link:Op_en3861}}

Insight networks are graphical representations of a particular situation, where the objects described are causally related to each other. In addition, the diagrams contain non-causal elements such as value judgements or inferences based on data. Insight networks utilise the ideas of directed acyclic graphs, but they have additional features.

Question

What notation is simple and flexible enough so that it can be used to represent all major issues related to a policy situation? It must be usable in both graphical and data formats.

Answer

+ Show code - Hide code

# This is code Op_en3861/ on page [[Insight network]]

############### Create global variables at file global.R

library(OpasnetUtils)

objects.latest("Op_en3861", code_name="insightNetwork") # [[Insight network]] insightNetwork

# If you want to use alternative objects, fetch them here.

insightNetwork <- EvalOutput(insightNetwork,verbose=TRUE)

cat("This code uses the following objects:\n")
oprint(insightNetwork@dependencies)

cat("The newtork data comes from these sources:\n")
oprint(meta)

insightTables <- makeInsightTables(meta) # You can update (part of) insightTables by rerunning with (partial) meta.

graphTable <- makeGraphTable(verbose=FALSE) # Produces a table in standard format

gr <- makeInsightGraph(graphTable) # Creates a DiagrammeR graph object

shinyApp(ui, server)

For examples of using insight networks, see op_fi:Ympäristöterveysindikaattori.

Rationale

Process

Insight networks have been described in a scientific article manuscript From open assessment to shared understanding: practical experiences#Insight networks. Objects and their relations used in open policy practice are described on page Open policy ontology.

There is a need for methods facilitating the flow of information and understanding between science and policy. The principle is to describe a risk situation in a formal manner. Insight networks contain items along a causal pathway (or network) from e.g. abatement strategies to emissions to dispersion to exposure to effects. They have been designed to describe also non-causal connections such as non-causal reasoning, values, preferences, and arguments.

These diagrams use graph theory with vertices (or nodes) and arcs (or arrows). They are used to describe and define all the pieces needed for a description of the situation under scrutiny. Diagrams may be produced with any graphics software, providing that calculation functions are not required. If calculations are needed, we recommend the use of R software and OpasnetUtils package.

This is the process how data flows into insight diagrams:

List of data tables of different insight diagrams is found from https://yhteistyotilat.fi/wiki08/x/1oGxAg. It has the following columns:
- Ilmio: Name of the phenomenon. This will become the name of a csv data file.
- Id: Identifier of the phenomenon. This will be used in Oldid of the items and relations.
- Tyyppi: Type of the table. In practice, it defines the columns that the data table has. Different types are listed on #Types of insight network tables.
- URL: Location of the data table. If the URL contains "google.com", it is assumed to be a google sheet. If the type (Tyyppi) is "keskustelu", it is assumed to be an Opasnet page with discussions. Otherwise, it is assumed to be a table on a web page that can be scraped with read_html() function.
- Taulu: If the data is a table on a web page, it is the number of the table on that page. If the data is a discussion, it is the number of discussion; missing value means that all discussions on that page are read.
- Alkurivi: In case of google sheets, it is the first row with actual data.
- Kuvaus: Description of the table, with possible links to relevant description page.

All data tables and discussions are listed, formatted and saved as csv files in a zip file called op_fi:File:Näkemysverkkojen tietotauluja.zip. From there, the data can be accessed from within Opasnet Rtools. (The code scraping web pages does not work in Opasnet, although it is stored there.) Little formatting is done here, mainly the column titles are standardised. But the number and type of columns is not changed.

In the next phase, each csv file is opened, interpreted, and defined as items and relations. This is done in code Op_fi5810/graphs on page op_fi:Ympäristöterveysindikaattori. All these are saved as a DiagrammeR graph, and each topic may be separately selected as a subgraph.

0094: . Are Tehtäväkokonaisuus, Osiotyyppi, JHS-luokka actually types of objects, or are they just indices. Yes, they should be indices and the objects relate to them with "has index". Correct table 4. --Jouni (talk) 20:46, 17 July 2018 (UTC) (type: ; paradigms: science: relevant attack)
0095: . Check the code about renderging graphs and creating a server function that also works on non-web environments on op_fi:Keskustelu:Näkemysverkko#Näkemysverkkoabstrakti vaikuttavuuden tutkimuksen päiville 4.-5.12.2018 --Jouni (talk) 20:46, 17 July 2018 (UTC) (type: ; paradigms: science: relevant attack)

Data

Graphical properties of objects and relations

Graphical properties of objects and relations(-)
Obs	Property	Value	Parameter	Result	Description
1	default	default	node.shape	circle	Default values unless something else is specified
2	default	default	node.style	filled
3	default	default	node.sides	4
4	default	default	node.skew	0
5	default	default	node.fillcolor	white
6	default	default	node.fontsize	11
7	default	default	node.height	0.5
8	default	default	node.width	0.5
9	default	default	node.color	brown
10	default	default	node.penwidth	2
11	default	default	node.fontcolor	black
12	default	default	node.distortion	0
13	default	default	edge.color	grey
14	default	default	edge.fontsize	10	Not currently used
15	default	default	edge.fontcolor	grey
16	default	default	edge.style	dotted
17	default	default	edge.penwidth	2
18	default	default	edge.arrowsize	1	Not currently used
19	type	unknown	node.fillcolor	yellow	This formatting is used if there are undefined objects
20	type	unknown	node.color	green
21	type	substance	node.shape	circle	Substantive type object
22	type	substance	node.fillcolor	skyblue2	Substantive type object
23	type	knowledge crystal	node.color	gold	Knowledge crystal type object (including ovariables and key ovariables)
24	type	option	node.color	palevioletred	Option for a decision
25	type	option	node.fillcolor	white	Option for a decision
26	type	index	node.shape	polygon	Index or other classifying determinant
27	type	index	node.sides	4
28	type	index	node.skew	0.5
29	type	index	node.fillcolor	purple1
30	type	index	node.height	0.3
31	type	graph	node.shape	polygon	Index or other classifying determinant
32	type	graph	node.sides	3
33	type	graph	node.fillcolor	pink
34	type	assessment	node.shape	polygon	Assessment
35	type	assessment	node.sides	8
36	type	assessment	node.fillcolor	purple1
37	type	stakeholder	node.shape	hexagon	Stakeholder type object
38	type	stakeholder	node.fillcolor	khaki1	Stakeholder type object
39	type	stakeholder	node.width	0.8	Stakeholder type object
40	type	method	node.shape	polygon	Method type object
41	type	method	node.sides	6	Method type object
42	type	method	node.fillcolor	purple1	Method type object
43	type	process	node.shape	pentagon	Process type object
44	type	process	node.fillcolor	purple1	Process type object
45	type	action	node.fillcolor	#009246	Process type object, dark green
46	type	action	node.shape	rectangle	Decision type object
47	type	task 1	node.color	brown	Illustration of the responsible organisation of the task
48	type	task 2	node.color	yellow	Illustration of the responsible organisation of the task
49	type	task 3	node.color	blue	Illustration of the responsible organisation of the task
50	type	task 4	node.color	green	Illustration of the responsible organisation of the task
51	type	task 5	node.color	red	Illustration of the responsible organisation of the task
52	type	decision	node.fillcolor	red	Decision type object
53	type	data	node.shape	rectangle	Data type object
54	type	data	node.fillcolor	gold	Data type object
55	type	objective	node.shape	diamond	Objective type object
56	type	objective	node.fillcolor	yellow	Objective type object
57	type	objective	node.width	0.8	Objective type object
58	type	publication	node.fillcolor	gray	Publication type object
59	type	statement	node.shape	polygon	Argument type object
60	type	statement	node.sides	4	Argument type object
61	type	statement	node.width	0.8	Argument type object
62	type	statement	node.distortion	-0.5	Argument type object
63	type	true statement	node.fillcolor	gold	Argument type object
64	type	false statement	node.fillcolor	gray	Argument type object
65	type	fact opening statement	node.fillcolor	lightskyblue1	Argument type object. Discussion start
66	type	value opening statement	node.fillcolor	palegreen1	Argument type object
67	type	fact closing statement	node.fillcolor	skyblue	Argument type object. Discussion end
68	type	value closing statement	node.fillcolor	springgreen	Argument type object.
69	type	fact discussion	node.fillcolor	skyblue	Argument type object. Not neede?
70	type	value discussion	node.fillcolor	springgreen	Value judgement type object. Not needed?
71	type	risk factor	node.color	pink	Additional information about object class
72	type	indicator	node.color	brown	Additional information about object class
73	type	indicator	node.fillcolor	gold	Additional information about object class
74	type	tactical indicator	node.fillcolor	#00d7a7	Additional information about object class light green
75	type	operational indicator	node.fillcolor	#9fc9eb	Additional information about object class light blue
76	type	strategic indicator	node.fillcolor	#9fc9eb	Additional information about object class
77	type	strategic indicator	node.shape	diamond	Additional information about object class
78	type	arviointikriteeri	node.color	orange	Not quite clear what criteria objects are: indicators or value statements, or something else
79	type	task	node.color	green	Additional information about object class
80	type	data	node.color	orange	Additional information about object class
81	type	health organisation	node.color	yellow	Additional information about object class
82	Relation	causal link	edge.color	black	Causal link
83	Relation	causal link	edge.style	solid	Causal link
84	Relation	positive causal link	edge.fontcolor	green	Causal link
85	Relation	negative causal link	edge.fontcolor	red	Causal link
86	Relation	participatory link	edge.color	purple	Participatory link
87	Relation	participatory link	edge.style	dashed	Participatory link
88	Relation	operational link	edge.color	black	Operational link
89	Relation	operational link	edge.style	dashed	Operational link
90	Relation	evaluative link	edge.color	green	Evaluative link
91	Relation	relevant attack	edge.color	red	Attacking argument
92	Relation	relevant defense	edge.color	green	Defending argument
93	Relation	relevant comment	edge.color	blue	Commenting argument
94	Relation	irrelevant argument	edge.color	gray	Invalid argument
95	Relation	argumentative link	edge.style	dotted	Argumentative link
96	Relation	argumentative link	edge.penwidth	4	Argumentative link
97	Relation	referential link	edge.color	red	Referential link
98	Relation	referential link	edge.style	dashed	Referential link

Calculations

Insight network 2.0

An updated version should improve the

a) context sensitivity (referring to primarily to objects within own context but secondarily to those from another context),
b) making graphs by default from a single context rather than a full list of contexts from a meta table,
c) compatibility with cytoscape.js,
d) merging ready-made graphs meaningfully,
e) have a reasonable intermediate object format that contains all data needed, such as
- tables for nodes and edges, compatible with Diagrammer, Cytoscape.js, AND Gephi.
- metadata for display, such as seeds, steps, object types to ignore, whether to show labels etc. Or should these just be implemented on the graph?

What should be done?

Fetch the data table by scrape or other function and with data about URL, table, and initial row.
Use splizzeria and fillprev if needed.
Interpret columns based on a vector of column numbers (with possibly 1+2 notation to paste columns) to create the standard columns. If this is done in an ovariable formula, there is no need for a specific function.
- Context
- Item
- type
- label
- rel
- Object
- Description
- Reldescription
- URL
- Result (dummy, always 0)
Create missing node rows from objects. Do NOT assume context.
Create URL from permanent resource location trunk and the identifier (where does the identifier come from?)
Item ja label laitetaan pötköön ja haetaan mätsi. Tulos onrow-pötköstä.
Create an ovariable from the table.
Add meta to the ovariable with formatting data.
- insightGraph:
  - seed
  - removenodes
  - formatting (character vector with possible entries: Hide node labels, Hide edge labels, Show legend nodes, Remove branches only)
  - ignoreobj
  - steps

(NOT NEEDED? Create Oldid if does not exist from context and numbering)
If a relation is presented as item, the formatting is applied to the ring.

Combine graph objects

Find items without context. Match them with items with the same Item (label) that do have a type.

Tuplarelaatiot, voidaanko kategorisesti poistaa?

Out <- rep(NA, length(find)) For(x in cond,) For(i in 1:length(find) Tmp<-id[context==contextfind(i))])[Match(find(i), df$cond(x)(df$context==contextfind(i))] pitää etsiä id alkuperäisestä taulukosta heti muuten ei toimi Out<- ifelse(isna(out). Tmp,out) )) Sitten sama ioman contekstirajoitusta.

Insight network 1.0

There are three different identifiers for a subject item.

Oldid: a technical identifier typically of format context.number, where number is a sequential number within a context.
Item: the actual name of the item, detailed enough to give a good understanding of its meaning.
label: a short name shown on insight networks. Does not exmplain everything, just enough to distinguish it from other items.

If Oldid is not given, it is created from the context and a number. If label is not given in data, it is truncated from Item.

Object item has one column Object that may contain any of these. The priority is Item > label > Oldid > Object. The last option means that it is assumed that Object refers to a new item that is not mentioned in the Item column.

An insight network is produced in this order (last object mentioned first).

gr: a diagrammer graph with all data and formatting for an insight network. Produced by makeInsightGraph.
makeInsightGraph

Making insight graphs

+ Show code - Hide code

# This is code Op_en3861/formatted on page [[Insight network]]

library(OpasnetUtils)

#' Function formatting creates a formatting table for nodes and edges.
#' The function as no parameters.
#' @return data.frame with ontology terms as rows and formatting properties as columns.

formatting <- function() {  
  ## Replace default setting with additional class info
  
  ## Find all classes for item subclasses
  
  hierItem <- opbase.data("Op_en7783", subset="Item types") # [[Open policy ontology]]
  hierItem <- hierItem[c("Object","English name","Finnish name")]
  colnames(hierItem)[colnames(hierItem)=="Object"] <- "Class" # Contains also other relations than subclass, notably "part of".
  
  ## Find all classes for relation subclasses
  
  hierRel <- opbase.data("Op_en7783", subset="Relation types") # [[Open policy ontology]]. All relations are of type 'has subclass'
  for(i in colnames(hierRel)) hierRel[[i]] <- as.character(hierRel[[i]])
  hierRel <- data.frame(
    Class = rep(hierRel$Class, 2),
    Tmp1 = c(hierRel$`English name`, hierRel$`English inverse`),
    Tmp2 = c(hierRel$`Finnish name`, hierRel$`Finnish inverse`)
  )
  colnames(hierRel) <- c("Class", "English name", "Finnish name")
  
  # Make a single resource list
  hier <- rbind(hierItem, hierRel)
  for(i in colnames(hier)) hier[[i]] <- as.character(hier[[i]])
  
  # Combine language versions of resource list
  hier <- unique(data.frame(
    Class = c(hier$Class, rep(hier$`English name`, 2)),
    Item = c(rep(hier$`English name`, 2), hier$`Finnish name`),
    stringsAsFactors = FALSE
  ))
  
  out <- hier
  tmp <- out$Class
  for(i in 1:6) {
    tmp <- hier$Class[match(tmp, hier$Item)]
    out <- rbind(
      out,
      cbind(
        Class = tmp,
        Item = hier$Item
      )
    )
  }
  hier <- out[!is.na(out$Class),]
  
  # Replace default settings with property-specific settings
  
  # First fetch the graphical styles of properties from [[Insight network]]
  
  prop_gen <- opbase.data(
    "Op_en3861", # [[Insight network]]
    subset="Graphical properties of objects and relations"
  )
  
  tmp <- prop_gen[grepl("edge", prop_gen$Parameter) & prop_gen$Value!="default",]
  tmp$Parameter <- gsub("edge","node",tmp$Parameter)
  prop_gen <- rbind(prop_gen, tmp)

  # Create a data.frame with all item * parameter combinations.
  # This will be filled with item-specific graph settings
  
  prop_spec <- merge(
    data.frame(Resource = unique(c(hier$Class, hier$Item))),
    prop_gen[prop_gen$Property=="default",c("Parameter","Result")]
  )
  
  for(i in 1:nrow(prop_gen)) { 
    if(prop_gen$Property[i] != "default") { 
      # Names of items that should have the property replaced
      tst <- unique(hier$Item[hier$Class==prop_gen$Value[i]])
      prop_spec$Result[prop_spec$Resource %in% tst & prop_spec$Parameter==prop_gen$Parameter[i]] <- prop_gen$Result[i]
    }
  }
  
  prop_spec$Result <- as.character(prop_spec$Result)
  
  formatted <- reshape(prop_spec, idvar="Resource", timevar="Parameter", direction="wide")
  colnames(formatted) <- gsub("Result.", "", colnames(formatted))
  
  #  > colnames(formatted)
  #  [1] "Resource"        "node.shape"      "node.sides"      "node.skew"      
  #  [5] "node.fillcolor"  "node.fontsize"   "node.height"     "node.width"     
  #  [9] "node.color"      "node.penwidth"   "node.fontcolor"  "node.distortion"
  #  [13] "edge.color"      "edge.fontsize"   "edge.fontcolor"  "edge.style"     
  #  [17] "edge.penwidth"   "edge.arrowsize" 
  
  for(i in c(
    "node.sides",
    "node.skew",
    "node.fontsize",
    "node.height",
    "node.width",
    "node.penwidth",
    "node.distortion",
    "edge.fontsize",
    "edge.penwidth",
    "edge.arrowsize" 
  )) formatted[[i]] <- as.numeric(formatted[[i]])
  
  return(formatted)
}

formatted <- formatting()

objects.store(formatted)
cat("Data.frame formatted stored.\n")

+ Show code - Hide code

# This is code Op_en3861/makeGraph on page [[Insight network]]
library(OpasnetUtils)

#' Making insight network graph object
#' 
#' makeGraph is a function for taking an insight ovariable and making a graph object.
#'
#' @param a is data.frame defining nodes and edges with at least columns: Oldid, type, Item, label, Relation, Object, Description. Other columns for nodes such as URL are allowed.
#' @return two data.frames: nodes_df and edges_df that are directly given as parameters for DiagrammeR::create_graph.

makeGraph <- function(ova, ...) {
  require(OpasnetUtils)
  require(DiagrammeR)
  
  if(!exists("formatted")){
    objects.latest("Op_en3861", code_name="formatted") # [[Insight network]] formatted
  }
  if(!exists("chooseGr")) {
    objects.latest("Op_en3861", code_name="chooseGr") # [[Insight network]] chooseGr
  }
  
  if("ovariable" %in% class(ova)) {
    a <- ova@output
    meta <- ova@meta$insightnetwork
  } else {
    a <- ova
    meta <- NULL
  }
  
  for(i in 1:ncol(a)) {
    a[[i]] <- gsub("[\"']", " ", a[[i]])
  }
  
  # Fill in missing labels, Items, and object nodes
  
  a$label <- ifelse(is.na(a$label),substr(a$Item,1,30), a$label)
  a$Item  <- ifelse(is.na(a$Item),a$label, a$Item)
  
  # Find nrow that matches the Object based on Item or label.
  tst <- rep(1:nrow(a),2)[match(a$Object, c(a$Item, a$label))]
  
  # Use Item as Object identifier when possible
  hasobj <- !(is.na(a$Object) | a$Object=="") # Rows of data.frame a that have Object
  a$Object[hasobj] <- a$Item[tst][hasobj]
  
  # Find objects that have not been defined
  newbies  <- ifelse(is.na(tst), a$Object,NA)
  newbies <- newbies[!is.na(newbies)]
  
  if(length(newbies)>0) {
    a <- orbind(
      a,
      data.frame(
        Item=newbies,
        label=substr(newbies,1,30),
        stringsAsFactors = FALSE
      )
    )
  }
  
  nodes <- a[!(duplicated(a$Item) | is.na(a$Item) | a$Item==""),]
  nodes$tooltip <- paste0(
    nodes$label, ". ",
    ifelse(nodes$label == nodes$Item, "", paste0(nodes$Item, ". ")), 
    ifelse(is.na(nodes$Description), "", paste0("\n", nodes$Description)),
    " (", nodes$Context, "/", nodes$id,")"
  )
  nodes <- merge(nodes, formatted, by.x="type", by.y="Resource")
  colnames(nodes) <- gsub("node.","",colnames(nodes))
  nodes <- nodes[!grepl("edge.", colnames(nodes))]
  nodes$id <- 1:nrow(nodes)
  
  # Create edges and flip unpreferred relations to their inverse relations
  
  inver <- opbase.data("Op_en7783", subset="Relation types")
  for(i in colnames(inver)) inver[[i]] <- as.character(inver[[i]])
  inve <- data.frame(
    rel = c(inver$`English name`,inver$`Finnish name`),
    inve = c(inver$`English inverse`,inver$`Finnish inverse`),
    stringsAsFactors = FALSE
  )
  
  edges <- a[!(is.na(a$Object) | a$Object=="") , ]
  flip <- edges$rel %in% inve$inve
  tmp <- edges$Item
  edges$Item[flip] <- edges$Object[flip]
  edges$Object[flip] <- tmp[flip]
  edges$rel[flip] <- inve$rel[match(edges$rel, inve$inve)][flip]
  edges$from <- match(edges$Item, nodes$Item)
  edges$to <- match(edges$Object, nodes$Item)
  edges$label <- edges$rel
  edges$labeltooltip <- paste0(edges$label, " (",edges$Context, "/",edges$id, ")")
  edges <- merge(edges, formatted, by.x="rel", by.y="Resource")
  colnames(edges) <- gsub("edge.","",colnames(edges))
  edges <- edges[!grepl("node.", colnames(edges))]
  edges$id <- 1:nrow(edges)
  
  gr <- create_graph(
    nodes_df=nodes,
    edges_df=edges
  )
  if(!is.null(meta)) {
    gr <- chooseGr(gr, input=meta)
  }
  
  return(gr) 
}

objects.store(makeGraph)
cat("Function makeGraph stored.\n")

+ Show code - Hide code

# This is code Op_en3861/chooseGr on page [[Insight diagram]].
library(OpasnetUtils)

#' Function chooseGr takes a diagrammer graph and selects s subgraph based on topic, labels, steps from selected nodes etc.
#' @param gr diagrammer graph
#' @param input list of arguments to be used in selection
#' @seeds ovariable where @data has columns Topic to be chosen and Node for Oldid's to select.
#' @return diagrammer graph where node_selection contains the selected nodes

chooseGr <- function(gr, input, seeds=NULL, verbose=FALSE) {
  if(!is.null(seeds)) seeds <- match(seeds@data$Node[seeds@data$Topic==input$topic], gr$nodes_df$Oldid)
  nods <- union(c(
    seeds,
    match(input$addnodes, gr$nodes_df$label)),
    match(input$addnodesByid, gr$nodes_df$id)
  )
  nods <- nods[!is.na(nods)]
  gr <- deselect_nodes(gr, get_selection(gr))
  gr <- select_nodes_by_id(gr, nods)
  if(input$steps>0) {
    for(i in 1:input$steps) {
      gr <- deselect_nodes(gr,match(input$removenodes, gr$nodes_df$label))
      if(nrow(gr$node_selection)>0) {
        gr <- trav_both(gr,add_to_selection = TRUE)
      }
    }
    if("Remove branches only" %in% input$formatting) {
      gr <- select_nodes_by_id(gr,match(input$removenodes, gr$nodes_df$label))
    } else {
      gr <- deselect_nodes(gr,match(input$removenodes, gr$nodes_df$label))
    }
  }
  if("Show legend nodes" %in% input$formatting) {
    gr <- select_nodes_by_id(gr, match(seeds@data$Node[seeds@data$Topic=="Selitykset"], gr$nodes_df$Oldid))
  }
  gr <- deselect_nodes(gr, match(input$ignoreobj, gr$nodes_df$type))
  if(verbose) cat("Selected nodes: ", gr$nodes_df$label[gr$nodes_df$id %in% gr$node_selection[[1]]])
  return(gr)
}

objects.store(chooseGr)
cat("Ovariable chooseGr stored.\n")

Function insightJSON fetches a JSON file of an insight network through a REST API. Works on own computer only.

+ Show code - Hide code

# This is code Op_en3861/insightJSON on page [[Insight network]]

#' This function fetches an insight network data as JSON through REST api and makes a graph
#' @param URL URL for the insight network data
#' @return a diagrammer graph object

insightJSON <- function(URL) {
  require(OpasnetUtils)
  require(DiagrammeR)
  require(jsonlite)
  
  objects.latest("Op_en3861", "formatted") # [[Insight network]] formatted
  
  tst  <- fromJSON(url(URL))
  nodes <- tst$data$nodes
  nodes <- data.frame(
    type = paste(nodes$indicator_level, nodes$type),
    Item = nodes$name,
    oldid = nodes$id,
    label = substr(nodes$name,1,30),
    tooltip = paste0(nodes$name,". ",nodes$id),
    URL = gsub("aplans.api","hnh",gsub("v1/","",nodes$url)),
    stringsAsFactors = FALSE
  )
  nodes$type <- gsub("NA ","",nodes$type)
  
  nodes <- merge(nodes, formatted, by.x="type", by.y="Resource")
  nodes$id <- 1:nrow(nodes)
  nodes <- nodes[!grepl("edge.",colnames(nodes))]
  colnames(nodes) <- gsub("node.","",colnames(nodes))
  nodes <- nodes[c("id",setdiff(colnames(nodes),"id"))]
  
  edges <- tst$data$edges
  edges$from <- match(edges$from, nodes$oldid)
  edges$to <- match(edges$to, nodes$oldid)
  edges$oldid <- edges$id
  edges$rel <- gsub("_", " ", edges$effect_type)
  edges$label <- edges$rel
  edges$tooltip <- paste0(edges$rel, " (", edges$confidence_level,") ", edges$id)
  edges <- merge(edges, formatted, by.x="rel", by.y="Resource")
  edges$id <- 1:nrow(edges)
  edges <- edges[!grepl("node.",colnames(edges))]
  colnames(edges) <- gsub("edge.","",colnames(edges))
  edges <- edges[c("from","to",setdiff(colnames(edges),c("from","to")))]
  
  gr <- create_graph(nodes, edges)
  return(gr)
}

#objects.store(insightJSON) # NOT STORED.
#cat("Function insightJSON stored.\n")

Format tables

+ Show code - Hide code

# This is code Op_en3861/splizzeria on page [[Insight network]]
library(OpasnetUtils)

#' Split cells contents into vectors
#' 
#' splizzeria function takes a data.frame and splits entries in cells of certain columns into separate rows. The idea is to make entries easier.
#' 
#' @param df data.frame to be splitted
#' @param cols names of columns that have the splittable contents
#' @param split splitting character that separates individual entries in the cells 
#' @return data.frame with the same columns but (possibly) more rows than df.

splizzeria <- function(
  df,
  cols,
  split=","
) {  
  require(reshape2)
  for(i in cols) {
    d <- as.character(df[[i]])
    d[d==""] <- NA # Because "" is incorrectly strsplitted
    d <- melt(strsplit(d, split=split), value.name=i)
    df$L1 <- 1:nrow(df)
    df <- merge(df[colnames(df)!=i], d)
    df[[i]] <- trimws(df[[i]])
    df[is.na(df[[i]]),i] <- ""
    df$L1 <- NULL
  }
  return(df)
}

objects.store(splizzeria)
cat("Function splizzeria stored.\n")

+ Show code - Hide code

# This is code Op_en3861/fillprev on page [[Insight network]]
library(OpasnetUtils)

#' Filling empty cells
#' 
#' fillprev fills empty cells in a data.frame by using content from the previous row.
#' 
#' @param df data.frame to be filled
#' @param cols vector of column names or positions to be filled.
#' @return Returns a data.frame with the same shape as df.

fillprev <- function(df, cols) {
  out <- df
  for(i in cols) { 
    for(j in 2:nrow(out)) {
      if(out[j,i] %in% c("", NA)) out[j,i] <- out[j-1,i]
    }
  }
  return(out)
}

objects.store(fillprev)
cat("Function fillprev stored.\n")

Shiny server

+ Show code - Hide code

# This is code Op_en3861/ui on page [[Insight network]]
library(OpasnetUtils)

################ Create Shiny user interface at file ui.R

ui <- function(request) {
  basicPage(
    sidebarLayout(
      sidebarPanel(
        selectInput("topic","Topic",unique(seeds@data$Topic)),
        selectizeInput(
          "addnodes",
          "Add seed nodes by label",
          sort(gr$nodes_df$label),
          selected = NULL,
          multiple = TRUE,
          options = NULL
        ),
        selectizeInput(
          "addnodesByOldid",
          "Add seed nodes by Oldid",
          sort(gr$nodes_df$Oldid),
          selected = NULL,
          multiple = TRUE,
          options = NULL
        ),
        selectizeInput(
          "removenodes",
          "Remove branches starting from nodes",
          sort(gr$nodes_df$label),
          selected = NULL,
          multiple = TRUE,
          options = NULL
        ),
        selectizeInput(
          "ignoreobj",
          "Ignore these object types",
          sort(unique(gr$nodes_df$type)),
          selected=NULL,
          multiple=TRUE,
          options=NULL
        ),
#        selectizeInput( # Commented out because does not have a final idea of how it should be implemented.
#          "ignorerel",
#          "Ignore these edge types",
#          sort(unique(gr$edges$rel)),
#          selected=NULL,
#          multiple=TRUE,
#          options=NULL
#        ),
        checkboxGroupInput("formatting", "Format graph", c(
          "Hide node labels",
          "Hide edge labels",
          "Show legend nodes",
          "Remove branches only"
        )),
        sliderInput("steps", "Number of steps:",  
                    min = 0, max = 5, value = 0),
        bookmarkButton()
      ),
      mainPanel(
#        p("Used object types: "), textOutput("objs"),
#        p("Used edge types: "), textOutput("rels"),
#        as.character(textOutput("remobj")),
#        strsplit(as.character(textOutput("remobj")),split=","),
#        html_nodes(textOutput("remobj"), css="div.shiny-text-output"),
        grVizOutput("plot1")
      )
    )
  )
}

objects.store(ui)
cat("Function ui stored (makeUi is depreciated and not stored). Usage: shinyApp(ui, server, enableBookmarking = 'url')\n")

+ Show code - Hide code

# This is code Op_en3861/server on page [[Insight diagram]].
library(OpasnetUtils)

objects.latest("Op_en3861", code_name="chooseGr") # [[Insight network]] chooseGr

#### Create shiny server at file server.R

server <- function(input, output, session) {
  output$plot1 <- renderGrViz({
    gr2 <- chooseGr(gr = gr, input = input, seeds = seeds)
#    gr <- deselect_nodes(gr, union( # This should find the INTERCEPT of from is selected AND to is selected and rel  %in% ignorerel.
# However, it is more complicated than that, because we may not want both from and to to disappear, only the one who is further away from core.
# Therefore, we may want to perform this inside the step loop in the same way as cutting branches.
#      tmp$edges_df$from[tmp$edges_df$rel %in% input$ignorerel],
#      tmp$edges_df$to[tmp$edges_df$rel %in% input$ignorerel]
#    ))

    gr2$nodes_df$label <- gsub("(.{1,18})(\\s|$)", "\\1\n", gr2$nodes_df$label) # Cut labels to max 18 characters long on one line (except if a word is longer)
    # Alternative possibility is to use strwrap function from {base} or stri_wrap from stringi.
    if("Hide node labels" %in% input$formatting) gr2$nodes_df$label <- ""
    if("Hide edge labels" %in% input$formatting) gr2$edges_df$label <- " "

    grViz(generate_dot(transform_to_subgraph_ws(gr2)))
  })
#  output$objs <- reactive({
#    tmp <- grr()
#    sort(unique(tmp$nodes_df$type[tmp$node_selection$node]))
#  })
#  output$rels <- reactive({
#    tmp <- grr()
#    sort(unique(tmp$edges_df$rel[union(
#      tmp$edges_df$id[tmp$edges_df$from %in% tmp$node_selection$node],
#      tmp$edges_df$id[tmp$edges_df$to %in% tmp$node_selection$node] #   )]))
#   })
}

objects.store(chooseGr, server)
cat("Functions chooseGr, server stored. Note! ChooseGr comes from its own code. Usage: shinyApp(ui, server, enableBookmarking = 'url')\n")

Scrape functions

These functions were be placed in the OpasnetUtils package, which is maintained in Github. To use the code, install a new version of the package by running R code

devtools::install_github("jtuomist/OpasnetUtils")

Codes Op_en3861/scrape.discussion, Op_en3861/scrape.functions, and Op_en3861/scrape.assessment on this page are outdated.

Copy descriptions to ovariables

The function assessmentDescriptions scans through an assessment ovarible that has all relevant assessment objects as parents. Dependencies slot may also have additional information, such as the following.

Name: name of parent (obligatory)
Ident: Opasnet page identifier and code name where the parent object can be loaded (e.g. Op_en7748/hia). Note: This is typically the code for the whole assessment, not the individual codes for the objects.
Token: Token for the model run where the parent object can be loaded (e.g. xxNsLw5hWdM6xyYp)
Description: A short description about what the object is. This is typically shown when cursor hovers over the object on an online insight diagram.
Page: Opasnet page identifier for the object's knowledge crystal page, which contains the research question, answer, and description of the object, together with discussion, if any. Typically this is empty for ovariables, because this information can be found from ovariable@meta slot and there is no need to duplicate it here.
Child: An object to which this object links. This is typically needed for objects such as graphs and data.frames that do not contain this information in their own structure, unlike ovariables. The direction of a relation is away from this object because then this object is the subject in triple sentences and can be given other parameters as well in other columns. A typical sentence is "graph describes ovariable", but for illustrative purposes this is inversed on insight networks so that the arrow points from an ovariable to a graph ("ovariable is described by graph").
Other columns are allowed.

+ Show code - Hide code

# This is code Op_en3861/ on page [[Insight network]]

assessmentDescriptions <- function( # Adds descriptions to ovariables.
  assessment # Ovariable or assessment whose dependencies will be given descriptions.
) {
  dep <- assessment@dependencies
  for(i in dep$Name) {
    tmp <- get(i)
    if("ovariable" %in% class(tmp)) {
      desc <- as.character(dep$Description[dep$Name==i])
      if(is.null(tmp@meta$Description)) {
        tmp@meta$Description <- desc
      } else {
        tmp@meta$Description <- paste0("1) ", desc, ". 2) ", tmp@meta$Description)
        warning("There already was ", i, "@meta$Description. The two were concatenated.\n")
      }
      assign(i, tmp, envir = .GlobalEnv)
    }
  }
  return(cat("Done.\n"))
}

Old notation

⇤--#: . Look at the table below together with Open policy ontology and merge. Decide which things should be on this page and which should be on the other. --Jouni (talk) 06:55, 24 April 2018 (UTC) (type: truth; paradigms: science: attack)

Object	Colour code in Analytica	Comments
General variable	8R3B (automatic)	This is a deterministic function of the quantities it depends on.
Chance variable	11L4B (autom)	This is a variable which is uncertain and uncontrollable (in a direct sense).
Data-driven variable	3R1B	A general variable where the result is mostly driven by data (observations or literature).
Author judgement variable	4R2B	A general variable where the result is mainly driven by author judgement (estimates with poor or no data).
Decision variable	9L3B	This is the variable that a decision-maker has the power to control. The decision variable should always be at the top of the chain of causality, even if this is a subchain i.e. it should not have any parent variables. Essentially the decision variable should be regarded as a decision that has to be made; since many factors affect all decisions it is not (in the case of INTARESE) an efficient use of resources to attempt to model what leads a decision-maker to make his decision.
Indicator	1R3B (autom)	This is a variable of special interest. One of the indicators in an assessment may be the quantitative criterion that you are trying to optimize. A particularly important variable in relation to the interests of the intended users of the assessment output (i.e. it must be a means of effective communication of assessment results). It must be in causal connection to the endpoints of the assessment and thus address causality throughout the full chain. It should reflect the use/purpose of the assessment. It should address and be adapted according to the target audience. It should be the ‘leading component’ in the assessment process.
Value judgement variable	8L4B	A preference or value that a person or a group assigns to a particular condition or state of the world.
Index (or dimension)	5R2B (autom)	This identifies the dimensions of the variable to which it is linked. Note that these dimensions do not have to be numeric, but can also be classes etc.
Risk assessment	8R3B (autom)
Scope	6R1B	The scope of the object
Conclusion	6L3B	A conclusion of the risk assessment (Result/Conclusion attribute).
Module	6R3B (autom)	A group of variables that are put together for illustrative or other practical reasons.
Data	2L3B (autom)	Contents of the Definition/Data attribute of a variable. If the Result attribute of a variable is used as Data for another variable, the first variable is called a proxy, and this node is used in the diagram. If an arrow or line is drawn between these objects, it must be noticed that this is NOT a causal link but an inference link. The direction of the arrow would be from the proxy to the variable.
Argument	8R2B	A piece of argumentation related to an object (variable, risk assessment, or class)
Formula	9L3B	Contents of the Definition/Formula attribute of a variable.
Class	1L2B	A class object (a set of objects that share a particular property).
Function	4R2B (autom)	A special kind of class. The particular property that is shared contains a full description of the Scope and the Definition attributes with given parameters.
Causal arrow		This states a causal relationship (or influence) of one variable onto another. Note that causal arrows can only exist between two arrows; any arrows to or from non-causal objects are non-causal inference arrows.
Non-causal arrow		This states an inference relationship between two objects. This means that the object where the arrow starts from is in the Data attribute of the other object. It is thus used to infer something about the value of the result of the latter object. Either object can be a variable or a non-variable. Note that Analytica is only able to show one kind of arrows, so in some cases the nature of the arrow (causal or inference) must be concluded from the context.

Previous notations

Insight networks have previously been called pyrkilo diagrams, extended causal diagrams, and factor-effect-value networks. These names are no longer in active use. An archived version of the notation can be found from an earlier version of this page.

References

Insight network: Difference between revisions

Revision as of 04:54, 1 May 2019

Contents

Question

Answer

Rationale

Process

Data

Graphical properties of objects and relations

Calculations

Making insight graphs

Format tables

Shiny server

Scrape functions

Copy descriptions to ovariables

Old notation

Previous notations

See also

References

Navigation menu

@@ Line 767: / Line 767: @@
 ==== Scrape functions ====
-These functions will be placed in the OpasnetUtils package. For now, it must be manually copied to your code.
+These functions were be placed in the OpasnetUtils package, which is [https://github.com/jtuomist/OpasnetUtils/blob/master/R/scrape.R maintained in Github]. To use the code, install a new version of the package by running R code
-<rcode name="scrape.discussion" label="Initiate function scrape.discussion (for developers only)" embed=1>
+ devtools::install_github("jtuomist/OpasnetUtils")
-# This is code Op_en3861/scrape.discussion on page [[Insight network]]
-library(OpasnetUtils)
-#' Scrape structured (pragma-dialectical) discussions
+Codes Op_en3861/scrape.discussion, Op_en3861/scrape.functions, and Op_en3861/scrape.assessment on this page are outdated.
-#'
-#' scrape.discussion is a function that takes a discussion in Opasnet and converts it to a standard graph table.
-#'
-#' @param pagename name of the page to read.
-#' @param wiki name of the Opasnet wiki for the page
-#' @param Url string for the url of the page to be used as link in insight networks
-#' @param n numbers of discussions on the page to scrape. If NULL, all discussions on the page will be read.
-#' @return a list of two data.frames. The first has standard headings for a graph table. The second has columns id, type, title, content, sign, target, type, paradigm, relation, and Result; and is ready for creating ovariables from arguments.
-scrape.discussion <- function(pagename, wiki = "", Url="", n=NULL) {
-  require(rvest) # html_nodes
-  require(xml2)
-  require(reshape2)
-  # Find the previous argument that is one level higher, i.e. the target argument, and the overtarget.
-  findtarget <- function(rel, disc, discItem) {
-    level <- rep(0,nrow(rel))
-    for(j in 0:20) {
-      level[rel$rel_id %in% html_attr(html_nodes(disc, css=paste(paste(rep("dd", j), collapse=" ")," .argument")),"rel_id")] <- j
-    }
-    test <- 1:length(level)
-    rel$Object <- rep("", length(level)) # Object is target of argument, overtarget is target of Object
-    for(j in test) {
-      rel$Object[j] <-  rel$Item[max(c(-Inf,test[level == level[j]-1 & test < j]))]
-      rel$overtarget[j] <- rel$Item[max(c(-Inf,test[level == level[j]-2 & test < j]))]
-    }
-    rel$Object[is.na(rel$Object)] <- discItem[1]
-    rel$overtarget[is.na(rel$overtarget) & level == 1] <- discItem[1]
-    return(rel)
-  }
-  discall <- html_nodes(
-    read_html(
-      opasnet.page(pagename,wiki)
-    ), css="div.discussion"
-  )
-  if(is.null(n)) n <- 1:length(discall)
-  out <- data.frame()
-  for(k in n) { # Go through each discussion
-    disc <- discall[[k]]
-    resd <- html_text(html_nodes(disc, css=".resolved"), trim=TRUE) # Not used
-    type <- tolower(trimws(substr(html_text(html_nodes(disc, css="font.type")),1,5)))
-    type <- gsub("arvok", "value", type)
-    type <- gsub("fakta", "fact", type)
-    context <- html_attr(disc, "id")
-    discItem <- paste(context,c("openingStatement","closingStatement"),sep=".")
-    # Make unique rel_id for argument-target relations.
-    nods <- html_nodes(disc, css=".argumentation .argument")
-    for(l in 1:length(nods)) {
-      xml_attr(nods[[l]], "rel_id") <- l
-      nodtmp <- html_nodes(nods[[l]], css=".paradigm")
-      xml_attr(nodtmp, "rel_id") <- l
-    }
-    # Make data.frame for opening and closing statements
-    tmp <- data.frame(
-      Context = context,
-      rel_id = discItem,
-      type = paste(type,c("opening statement","closing statement")),
-      Item = discItem,
-      label = c(".",html_text(html_nodes(disc, css="b.title"), trim=TRUE)),
-      rel = c("produces",""),
-      Object = c(discItem[2],""),
-      Description = c(
-        html_text(html_nodes(disc, css=".openingStatement"), trim=TRUE),
-        html_text(html_nodes(disc, css=".closingStatement"), trim=TRUE)),
-      URL = paste(Url, context, sep="#"),
-      sign = "",
-      overtarget = "",
-      paradigm = "science",
-      stringsAsFactors = FALSE
-    )
-    out <- rbind(out, tmp)
-    ######## rel: Find parameters that are unique within the id-target relations
-    ######## (one argument may show up several times)
-    #   "id"  "title" "type"  "content" "sign"  "target"   "overtarget"
-    arg_id <- html_attr(html_nodes(disc, css=" .argument"), "id") # argument id
-    rel <- data.frame(
-      Context = context,
-      rel_id = html_attr(html_nodes(disc, css=" .argument"), "rel_id"), # relation id
-      type = html_text(html_nodes(disc, css=".argument i.type ")),
-      Item = arg_id,
-      label = html_text(html_nodes(disc, css=" .argument .title")),
-      Description = html_text(html_nodes(disc, css=" .argument .content")),
-      URL = paste(Url, arg_id, sep="#"),
-      sign = html_text(html_nodes(disc, css=" .argument .sign a:first-of-type")),
-      #     refs = html_text(html_nodes(nods, css="sup.reference"), trim=TRUE),
-      stringsAsFactors = FALSE
-    )
-    rel <- findtarget(rel, disc, discItem) # add Object and overtarget
-    ######### para: find parameters that are unique in id-target-paradigm triples
-    #  "id"        "paradigm"  "relation"  "relevance" "selftruth" "truth"
-    para <- data.frame(
-      rel_id = html_attr(html_nodes(disc, css=" .argument .paradigm"), "rel_id"),
-      paradigm = html_text(html_nodes(disc, css=" .argument .paradigm")),
-      rel = html_text(html_nodes(disc, css=" .argument .relation")),
-      stringsAsFactors = FALSE
-    )
-    out <- rbind(out, merge(rel,para))
-  }
-  out$label <- ifelse(out$label %in% c(".",""),substr(out$Description, 1, 30),out$label)
-  out <- out[order(out$label, decreasing=TRUE),]
-  #  for(i in 1:ncol(out[[1]])) out[[1]][[i]] <- gsub("['\"]", "", out[[1]][[i]])
-  ###########################################
-  # Make inferences about validity and truth based on different paradigms
-  # Add rows from science paradigm to all other paradigms if missing
-  default <- merge(
-    unique(out["paradigm"]),
-    out[out$paradigm=="science",colnames(out)!="paradigm"]
-  )
-  default <- rbind(out,default)
-  out <- default[!duplicated(default[c("Context","rel_id","paradigm")]),]
-  # Initial situation: all nodes are true and active
-  out$true <- TRUE
-  out$relevant <- TRUE
-  out$active <- TRUE
-  tmp2 <- out # Temporarily store the data.frame
-  # Function to test for truth, relevance, activity, orphan, type, and relation
-  nodetest <- function(dat, type, relation) {
-    return(
-      dat$type==type & # nodes that are about truth of target
-        dat$rel==relation & # nodes that have the relation of interest (usually attack)
-        dat$true & # nodes that are true
-        dat$active & # nodes that are active
-        !dat$Item %in% dat$Object[dat$active] & # nodes that are orphans
-        dat$relevant # node-object pairs that are relevant
-    )
-  }
-  # Start inference from orphan nodes (nodes that do not have parents)
-  out <- data.frame()
-  for(i in unique(tmp2$paradigm)) {
-    tmp <- tmp2[tmp2$paradigm==i,]
-    while(any(tmp$active)) { # Handles threads until all nodes and relations have been handled
-      # Find nodes that make successful truth attacks
-      # Successful truth attacks invalidate their target nodes
-      tmp$true[tmp$Item %in% tmp$Object[nodetest(tmp, "truth","attack")]] <- FALSE
-      # In paradigm personaltine, invalidate all arguments by Tine Bizjak
-      if(i=="personaltine") {
-        tmp$true[grepl("Tine Bizjak",tmp$sign)] <- FALSE
-      }
-      # Find nodes that make successful relevance attacks
-      tst <- nodetest(tmp,"relevance","attack")
-      # Successful relevance attacks invalidate their target-overtarget relations
-      tmp$relevant[paste(tmp$Item,tmp$Object,sep="-") %in% paste(tmp$Object,tmp$overtarget,sep="-")[tst]] <- FALSE
-      # Remove orphan nodes from active nodes
-      tmp$active[!tmp$Item %in% tmp$Object[tmp$active]] <- FALSE
-    }
-    out <- rbind(out, tmp)
-  }
-  # Post-processing: adjust type and rel so that graphs plot correctly
-  tst <- out$type %in% c("truth","relevance")
-  out$type[tst] <- ifelse(out$true, "true statement", "false statement")[tst]
-  out$rel[tst] <- paste(ifelse(out$relevant,"relevant","irrelevant"),out$rel)[tst]
-  return(out)
-}
-objects.store(scrape.discussion)
-cat("Function scrape.discussion stored.\n")
-</rcode>
-<rcode name="scrape.functions" label="Initiate scrape functions (for developers only)" embed=1>
-# This is code Op_en3861/scrape.functions on page [[Insight network]]
-library(OpasnetUtils)
-#' Scrape google sheets
-#'
-#' scrape.gssheet is a function for scraping google sheets and making data.frames
-#'
-#' @param page character vector with the URL of the google sheet
-#' @param firstrow atom vector with the number of the first data row on the gsheet
-#' @return a data.frame
-scrape.gssheet <- function(page, firstrow) {
-  require(gsheet)
-  out <- gsheet2tbl(page)
-  if(firstrow>2) out <- out[-(1:(firstrow-2)) , ]
-  return(out)
-}
-#' Scrape tables on a webpage
-#'
-#' scrape.webpage is a function for scraping data from a table on a webpage.
-#'
-#' @param page URL of the page to be scraped.
-#' @param table number of table on the page to be scraped.
-#' @return a data.frame
-scrape.webtable <- function(page, table) {
-  require(rvest)
-  out <- html_table(read_html(page),fill=TRUE)[[table]]
-  return(out)
-}
-objects.store(scrape.gssheet, scrape.webtable)
-cat("Functions scrape.gssheet, scrape.webtable stored.\n")
-</rcode>
-<rcode name="scrape.assessment" label="Function scrape.assessment (for developers only)" embed=1>
-# This is code Op_en3861/scrape.assessment on page [[Insight network]]
-library(OpasnetUtils)
-#' Create URLs for objects
-#'
-#' makeurl creates a unique URL based on wiki page id and object name. Used to create hyperlinks to the knowledge crystal pages.
-#' @param page wiki_page_id: character vector format "Op_en7748"
-#' @param name name of the knowledge crystals: character vector
-#' @return character vector of URLs
-makeurl <- function(
-  page,
-  name
-) {
-  if(is.null(page)|is.null(name)) return(NA) else
-    out <- paste0(
-      c(
-        en = "http://en.opasnet.org/w/index.php?curid=",
-        fi = "http://fi.opasnet.org/fi/index.php?curid="
-      )[substr(page, 4,5)], # is it en or fi?
-      substr(page, 6,11), # id
-      "#", gsub(" ", "_", name)
-    )
-  out <- ifelse(is.na(page)|is.na(name), NA, out)
-  return(out)
-}
-#' Scrape ovariables and other objects in an assessment
-#'
-#' scrape.assessment makes standard data.frame for insight diagram out of all ovariables, odecisions, and data.frames and their dependencies in the global environment.
-#'
-#' @param assessment ovariable that contains other assessment objects as dependencies
-#' @param objectives names of ovariables that are objectives in the model
-#' @return a list of two data.frames. The first one is for making insight diagrams, the second for making discussion ovariables for analysis (the latter part not implemented yet)
-scrape.assessment <- function(
-  assessment,
-  objectives = character()
-) {
-  #  require(DiagrammeR)
-  require(OpasnetUtils)
-  nod <- data.frame()
-  ova <- character()
-  dec <- character()
-  dat <- character()
-  plo <- character()
-  dep <- assessment@dependencies
-  URLass <- assessment@meta$wiki_page_id
-  #### Find all objects (decisions, ovariables, data.frames and graphs)
-  objs <- list()
-  for(i in dep$Name) {
-    cl <- class(get(i))
-    if("ovariable" %in% cl) ova <- c(ova, i)
-    if("odecision" %in% cl) dec <- c(dec, i)
-    if("data.frame" %in% cl) dat <- c(dat, i)
-    if(any(c("ggplot","dgr_graph") %in% cl)) plo <- c(plo, i)
-    if(is.vector(get(i))) objs <- c(objs, list(get(i))) else objs <- c(objs, get(i))
-  }
-  names(objs) <- dep$Name
-  ###### Add all decisions and options
-  for(i in dec) {
-    deci <- objs[[i]]@dectable
-    if(nrow(deci)>0) {
-      tst <- !duplicated(deci[c("Option","Decision")])
-      nod <- rbind(
-        nod,
-        data.frame(
-          #          Oldid = paste("Opt", match(i, dec), sep=""),
-          type = "option",
-          Item = as.character(deci$Option[tst]),
-          rel = "is option for",
-          Object = as.character(deci$Decision[tst]),
-          Description = if(is.null(deci$Description)) NA else
-            as.character(deci$Description[tst]),
-          URL = makeurl(URLass, deci$Option)[tst],
-          stringsAsFactors = FALSE
-        )
-      )
-      tst <- !duplicated(deci[c("Decision","Variable")])
-      nod <- rbind(
-        nod,
-        data.frame(
-          #          Oldid = paste("Dec", match(i, dec), sep=""),
-          type = "decision",
-          Item = as.character(deci$Decision[tst]),
-          rel = "affects",
-          Object = as.character(deci$Variable[tst]),
-          Description = if(is.null(deci$Description)) NA else
-            as.character(deci$Description[tst]),
-          URL = makeurl(URLass, deci$Decision)[tst],
-          stringsAsFactors = FALSE
-        )
-      )
-    }
-  }
-  ####### Add all ovariables
-  for(i in ova) {
-    obj <- objs[[i]]
-    nam <- if(length(obj@name)>0) obj@name else ""
-    nod <- rbind( # Add dependencies
-      nod,
-      data.frame(
-        #        Oldid = paste("Ova",match(i,ova),sep=""),
-        type = "ovariable",
-        Item = nam,
-        rel = "is affected by",
-        Object = if(nrow(obj@dependencies)==0) NA else as.character(obj@dependencies$Name),
-        Description = if(is.null(obj@meta$Description)) NA else
-          obj@meta$Description,
-        URL = makeurl(obj@meta$wiki_page_id, nam),
-        stringsAsFactors = FALSE
-      )
-    )
-    tmp <- colnames(obj@output)[  # Add indices
-      obj@marginal &
-        !grepl("Source$", colnames(obj@output)) &
-        ! colnames(obj@output) %in% c("Iter") # deci not tested because of error
-      ]
-    if(length(tmp)>0) {
-      nod <- rbind(
-        nod,
-        data.frame(
-          #          Oldid = paste0("Ova",match(i,ova),tmp),
-          type = "index",
-          Item = tmp,
-          rel = "is index for",
-          Object = nam,
-          Description = NA,
-          URL = makeurl(URLass, tmp),
-          stringsAsFactors = FALSE
-        )
-      )
-    }
-  }
-  ### Add graphs
-  if(length(plo)>0) {
-    nod <- orbind(
-      nod,
-      data.frame(
-        #        Oldid = paste0("Plo", 1:length(plo)),
-        type = "graph",
-        Item = plo,
-        stringsAsFactors = FALSE
-      )
-    )
-  }
-  # V(dag)$Size[V(dag)$name == i] <- nrow(obj@output)     # Size
-  # vertex.size = log(V(dag)$Size)+2, # Vertex size is (non-linearly) relative to rows in output.
-  ### Add data.frames
-  if(length(dat)>0) {
-    nod <- orbind(
-      nod,
-      data.frame(
-        #        Oldid = paste("Dat", 1:length(dat), sep=""),
-        type = "data",
-        Item = dat,
-        stringsAsFactors = FALSE
-      )
-    )
-  }
-  # Retype objectives and assessments if available
-  nod$type[nod$Item %in% objectives] <- "objective"
-  nod$type[nod$Item %in% assessment@name] <- "assessment"
-  # Add info from Page column if available and doesn't exist already
-  if(!is.null(dep$Page)) {
-    tst <- is.na(nod$Description)
-    nod$Description[tst] <- as.character(dep$Description[match(nod$Item, dep$Name)][tst])
-    tst <- is.na(nod$URL)
-    nod$URL[tst] <- makeurl(
-      dep$Page[match(nod$Item[tst], dep$Name)],
-      dep$Name[match(nod$Item[tst], dep$Name)]
-    )
-  }
-  tst <- is.na(nod$Description)
-  nod$Description[tst] <- nod$type[tst]
-  #Add info (with force) from type column in dep if available
-  if(!is.null(dep$type)) {
-    tst <- dep$Name[!(is.na(dep$type) | dep$type=="")]
-    tst <- tst[tst %in% nod$Item] # Necessary because some names may be missing
-    nod$type[match(tst,nod$Item)] <- as.character(dep$type[match(tst,dep$Name)])
-  }
-  #Add info (with force) from label column in dep if available
-  nod$label <- NA # Add column anyway because makeInsightTables requires standard columns
-  if(!is.null(dep$label)) {
-    tst <- dep$Name[!(is.na(dep$label) | dep$label=="")]
-    tst <- tst[tst %in% nod$Item] # Necessary because some names may be missing
-    nod$label[match(tst,nod$Item)] <- as.character(dep$label[match(tst,dep$Name)])
-  }
-  # Add relations defined by hand on assessment@dependencies
-  tst <- dep$Name[!(is.na(dep$Child) | dep$Child=="")]
-  if(length(tst)>0) { # Does nothing if column Child does not exist
-    nod <- orbind(
-      nod,
-      data.frame(
-        #        Oldid = as.character(nod$Oldid[match(tst,nod$Item)]),
-        Item = as.character(tst),
-        rel = c(
-          rep("indirectly affects",3),
-          "is data for",
-          "describes"
-        )[match(nod$type[match(tst,nod$Item)],c("ovariable","key ovariable","objective","data","graph"))],
-        Object = as.character(dep$Child[match(tst,dep$Name)])
-      )
-    )
-  }
-  return(nod)
-}
-objects.store(scrape.assessment, makeurl)
-cat("Functions scrape.assessment, makeurl stored.\n")
-</rcode>
 ==== Copy descriptions to ovariables ====