Uploading to Opasnet Base

This page is a knowledge crystal of subtype method. The page identifier is Op_en4590
Moderator:Jouni (see all)
Upload data {{#opasnet_base_link:Op_en4590}}

Uploading to Opasnet Base helps you understand what data could and should be updated to Opasnet Base and what the recommended data structures and formats are. For technical instructions how to use the current upload software, see Opasnet Base connection. For a general description about the database, see Opasnet Base and for technical details about the database, see [[Opasnet Base structure).

Scope

What data could and should be updated to Opasnet Base and what are the recommended data structures and formats?

Definition

Inputs

This section describes what the inputs are and the different ways they can be formatted for a successful upload.

Data

The data must be formatted as a two-dimensional table with one observation at each row. Columns contain either explanations (or independent variables in statistics) or the actual observations (or dependent variables in statistics). This difference is important. Explanations are things that are fixed before the actual observation, while observations are those that are actually measured or observed. See the following table as example.

An example of a data table.

City	Year	Sex	Body mass index BMI	Blood cholesterol
London	2010	Male	20	3.31
London	2010	Male	25	6.83
London	2010	Female	20	5.55
London	2010	Female	30	5.42
London	2010	Female	25	4.19
New York	2010	Male	22	3.33
New York	2010	Male	26	5.84
New York	2010	Female	28	5.67
New York	2010	Female	26	4.52
New York	2010	Female	24	5.67

The table seems unambiguous at the first glance, but it is impossible to interpret it without knowing, which columns are explanations and which are observations. This may be a study performed in London and New York in 2010, where random people were asked for a blood test. In this case, city and year are explanations, and other columns are observations. However, the data may as well be summary statistics from a larger study, where the studied individuals were grouped in these cities based on their sex and body mass index (BMI), and the mean cholesterol in each group is the only observation column. (The data implies that London used 5-unit groups for BMI while New York used 2-unit groups, but you cannot know based on the data only.)

However, if you know which columns are explanations and which are observations, you can actually deduce many important aspects of the design of the study from which the data came. Of course many things must be explained elsewhere like how people were selected and what the base population studied was.

Another important feature of the data is, whether it is deterministic or probabilistic.

8.7.2010 Jouni Tuomisto If the variable is deterministic, Obs should be 0. This must be in all upload methods. They are corrected accordingly, see Indexify.

Some key parts of the Opasnet Base connection

Overview

This module saves original data or model results (a study or a variable, respectively) into the Opasnet Base. You need your Opasnet username and password to do that. You must fill in all tables and fields below before the process can be completed.

If an object with the same Ident already exists in the Opasnet Base, the information will be added to that object. Before you start, make sure that you have created an object page in the Opasnet wiki for each object (study or variable) you want to upload.

The first row must contain the names of the columns. These names are used when creating the list of explanations in the Opasnet Base.

You must give your Opasnet username to upload data. The username will be stored together with the upload information.

Inputs

Writerpsswd: You must know the writer password for the Opasnet Base if you are not using the AWP web interface.

• Object info contains the most important metadata about your data.

- Ident is the page identifier in Opasnet. If your study or variable does not already have a page, you must create one. The identifier is found in the metadata box in the top right corner of the Opasnet page. - Number of indices is the number of columns that contain explanatory information (see below). - Parameter name is a common name for all data columns. If omitted, 'Parameter' is used. See below for more details. - If "Probabilistic?" is 1, then each row of the data table is considered a random draw from a data pool. Note that it is assumed that the index values are assumed the same in all rows, and only the index values of the first row are stored. - Append to upload: Typically, each data upload event is given a separate identifier. If you want to continue an existing upload of the same object, you can give the number of that upload, and the new data will be appended.

Key parts of OBC

Findid

This function gets an id from a table. It has the following parameters: in: the property for which the id is needed. In MUST be unique in cond and it must contain index i. table: the table from where the id is brought. The table MUST have .j as the column index, .i as the row index, and a column named 'id'. cond: the name of the field that is compared with in. Cond must be text.

Textify

Changes a number to a text value with up to 15 significant numbers. This bypasses the number formatting problem that tends to convert e.g. 93341 to '93.34K'. If the input is null, the result is .

Input format type 1 (Copy-paste)

The data are copy-pasted into the field 'Observations'. The source of the data can be any spreadsheet or text processor, as long as each column is separated by a tab, and each row by a line break.

Note that the pasted data should be between 'quotation marks'.

Fill in the additional information about the data. These are asked for the object, and also for all the indices and the parameter. Note that is an entry with the identical Ident already exists in the Opasnet Base, this information will NOT be uploaded but the existing information will be used instead. All information should be between 'quotation marks' so that they are not mistakenly interpreted as Analytica node identifiers. - Name: a description that may be longer than an identifier. This is typically identical to the respective page in Opasnet. - Unit: unit of measurement.

Advanced options:

There are two ways of uploading data. A) 'Upload data' is a public format, and all details are openly available. B) 'Upload non-public data' stores the actual data (the values in the parameter columns) into a database that requires a password for reading. However, all other information (including upload metadata and the data in the index fields) are openly available.

Platform: You must choose THL computer if you are not using the AWP web interface.

Object info: - Data source: 1 means that you are copy-pasting data to the 'Observations' field. 2 means that you have a 2D table in an Analytica node. The node must have column index .j (note: it is a local index!) and row index .i. The names of the columns must be in the index .j, and the first row must contain data. 3 means that you have a typical Analytica node with n indices; one of the indices may be Run. The node is transformed into a 2D table using MDArrayToTable. - Analytica identifier is the identifier of the node to be used. The name must be given between 'quotation marks', i.e. as text. - Ident: like in the simplified upload. - Number of indices: like in the simplified upload if data source 2 is used; for 3, the number of indices comes from the node, and this entry is ignored. - Parameter name: like in the simplified upload if data source 2 is used; for 3, the parameter is implicit, and this entry is ignored. - Probabilistic?: like in the simplified upload if data source 2 is used; for 3, if this entry is 1, the sample mode is used and the full distribution is saved, if the entry is not 1, the mid mode is used. - Append to upload: like in the simplified upload.