Uploading to Opasnet Base

This page is a knowledge crystal of subtype method. The page identifier is Op_en4590
Moderator:Jouni (see all)
Upload data {{#opasnet_base_link:Op_en4590}}

Uploading to Opasnet Base helps you understand what data could and should be updated to Opasnet Base and what the recommended data structures and formats are. For technical instructions how to use the current upload software, see Opasnet Base connection. For a general description about the database, see Opasnet Base and for technical details about the database, see Opasnet Base structure. For details about downloading data, see Opasnet Base UI.

Question

What data could and should be updated to Opasnet Base and what are the recommended data structures and formats?

Answer

There are several different tools to upload data to Opasnet Base. These are first listed and described here, and then more details are given either here or on a separate page.

1. Table2Base is a functionality that uploads a table from an Opasnet page to Opasnet Base. <t2b> tag is used to make the table. By default, the table content replaces any previous content in Opasnet Base every time when the table is changed and the page saved. Currently, there can be any number of explanation columns but only one observation column in a t2b table. It will be made possible to have a freely chosen number of also observation columns.
2. Opasnet Base Connection for R is a group of R functions that can easily upload data from models run in the R environment.
3. Opasnet Base Import is a tool for uploading data in CSV or XLS files to the Opasnet Base. Registered Opasnet users can use this tool by following the "Upload data" link in the top right corner box on any variable (or other data storing) page. This is explained on more detail on this page.
4. Table2Base functionality can also be extended to upload data from a questionnaire form on an Opasnet page. This way, the data itself does not stay on the page and can therefore only be seen from the base. Currently, this functionality uses Google forms to store data (see Widget:Google Form), and therefore the data is not stored in Opasnet Base but as a Google document instead. A version that would store data into the base are under construction.

Data

• At the first data was modified from text file to excel file. At the same time the first ID column was removed (see figures 1 and 2),

Fig 1. Example raw data in text format.

Fig 2. Text data is transformed to excel format. First id column has been removed.

• Secondly we went to the opasnet page where the table should be uploaded and take "Upload data". (The page should be variable or method page!!)
• In the Opasnet Base Import page all the inputs depends on the data.
  • Define Unit (If you have several units write e.g. "misc"
  • If data has only one column with results, take mark into "First row contains the indices"
  • If data has many columns with results, you have to define Indices and Locations (see Fig 3) Indices are all indices of data (last one is indice for Locations) in the table and Locations are name of result columns. All names must be separated with comma.
  • If data is in csv format define CSV inputs.
  • Browse data from files
  • Upload

Fig 3. Inputs of example data.

• Confirm uploading (Fig 4.)

Fig 4. Preview of data rows and confirming of uploading.

• Browse uploaded data with Opasnet Base UI (See Fig 5.).

Fig 5. Data uploaded successfully.

• In the data page "Show the result" shows the data from OpasnetBase (Fig 6)

Fig 6. Data page is ready.

Rationale

This section describes what the inputs are and the different ways they can be formatted for a successful upload.

What are the topics about which data can be uploaded into the Base? Well, basically any topic that provides useful information for any decision-making situation that has societal relevance. This sounds like a very wide definition, and it is. The data may be about which car models are environmentally friendly. It may be about pollutants in food. It may be new ideas about a societally just value-added tax.

What are, then, the data structures that are allowed? Although not all structures are allowed, almost any data can be easily transformed into the structure that is used in Opasnet Base. The data must be formatted as a two-dimensional table with one observation at each row. Cells of the table may contain either text or numerical values. Columns contain either explanations (or independent variables in statistics) or the actual observations (or dependent variables in statistics). This difference is important. Explanations are things that are fixed before the actual observation, while observations are those that are actually measured or observed. See the following table as example.

An example of a data table.

City	Year	Sex	Body mass index BMI (kg/m2)	Blood cholesterol (mM)
London	2010	Male	20	3.31
London	2010	Male	25	6.83
London	2010	Female	20	5.55
London	2010	Female	30	5.42
London	2010	Female	25	4.19
New York	2010	Male	22	3.33
New York	2010	Male	26	5.84
New York	2010	Female	28	5.67
New York	2010	Female	26	4.52
New York	2010	Female	24	5.67

The table seems unambiguous at the first glance, but it is impossible to interpret it correctly without knowing, which columns are explanations and which are observations. This may be a study performed in London and New York in 2010, where random people were asked for a blood test. In this case, city and year are explanations, and other columns are observations. However, the data may as well be summary statistics from a larger study, where the studied individuals were grouped in these cities based on their sex and body mass index (BMI), and the mean cholesterol in each group is the only observation column. (The data implies that London used BMI groups 5 kg/m² wide, while New York used BMI groups kg/m² wide, but you cannot know based on the data only.)

However, if you know which columns are explanations and which are observations, you can actually deduce many important aspects of the design of the study from which the data came. Of course many things must be explained elsewhere like how people were selected and what the base population studied was.

Another important feature of the data is, whether it is deterministic or probabilistic. With deterministic data, it is assumed that each row is an independent piece of information. With probabilistic data, it is assumed that there are random draws from a pool of potential observations (like an urn full of balls with different colours, each having the same probability of being picked). However, all observations are not picked from the same pool, but from several pools uniquely defined by the explanation columns.

Let's assume that we look at the table above and learn that it is a probabilistic data with two explanation columns. We then know that people were randomly picked from either London or New York in 2010 (explanation columns are always the first ones on the left side). In London, two happened to be males and three females, with measured BMIs and cholesterol levels. Thus, there are five observations from the pool defined by "London 2010", and also five observations from "New York 2010." These are numbered 1..5. If the data is deterministic, the observation number is 0 for all rows.