File:Opasnet base connection.ANA: Difference between revisions
(uploaded a new version of "File:Opasnet base connection.ANA": The complete model was updated to reflect the updated Opasnet Base structure (with act and actobj, and without objinfo etc). Reading from the Base works fine, also uploading mostly ) |
(uploaded a new version of "File:Opasnet base connection.ANA": Several bugs corrected. Now seems to work OK. In addition, act.series_id moved to actobj.series_id because it is object-specific.) |
(No difference)
|
Revision as of 09:30, 13 January 2010
Moderator:Jouni (see all) |
|
Upload data
|
This model is an interface for uploading data to and downloading it from the Opasnet base. This file is an editable version of a web interface. For using the file, you need Analytica Enterprise. For using the web interface, you only need a web browser (Mozilla Firefox recommended).
An improved web interface
There is also a web interface for downloading only; this interface produces raw data tables that you can copy-paste to an external software. However, some pieces of data are very large, and you may experience downloading the data, or handling the data afterwards. Therefore, there is a need to improve the interface. The improvement needs are described in this chapter.
The data is stored into the Opasnet base as two-dimensional tables, one for each data object (data objects can be either studies or variables. For details, see the description of the database structure. For example, the data called Morbidity data for Europe consists of a table that is indexed by four indices: Year, Sex, Age, and Country. Indices define the specifics of a particular observation. In addition, there is one parameter (morbidity), which is the field that contains the observations (values of interest).
Often, the user is only interested in a part of the data, e.g. one year of observation, one country, and maybe a few age groups. So, there should be an interface that shows the data in the Opasnet in a few steps, when a user selects a data object.
- The user is shown the list of uploads (versions) of the data object. The default is the newest upload, but the user should be able to choose an older one as well. This is based on the query that is shown on the metadata box as "List of uploads".
- The user is shown the list of indices and locations (possible values). The user can choose any combination of locations with tick boxes beside each location.
- The user is shown a two-dimensional table of data with indices and parameters as fields, and observations as rows. This can then be copy-pasted to other programs. Alternatively, a comma separated file (CSV) is created and a Save as dialog box appears. Now the "The newest upload" link does something like this but does not allow for selecting locations, and the output format is not optimal for copy-pasting data into another software.
- In cases where the data is given as distributions instead of point estimates, the data can also be downloaded as a sample file of distributions.
Development needs
- Now the model interprets the results as numbers if applicable, otherwise text.
- Analytica_node creates an indexname list where local indices start with dot (.Index). The dot should be removed, because in the Opasnet Base there are no local indices.
- Upload the read functions. The indices should have the real names from the Opasnet Base, not just In1, In2 and so on.
- The queries for read functions should be the same queries as in [[Template:Resultlink]].
- The read functions don't work properly. They don't show Testvariable data uploaded 28.8.2009 for some reason.
- The first read function should show data in 2D format; only the next function should turn it into an array. This is because not all data has originally been arrays and would result in hugely multidimensional arrays with a lot of empty cells.
Kehitettävää (by Erkki):
- A) Taulukkodatan syöttö on vielä epävarmaa, toimii välillä ja välillä ei, vailla selkeää johdonmukaisuutta. ==> Epävarma toiminta voisi Juhan mukaan viitata johonkin selain-/flash-/tms.-tekniseen fibaan.
- B) Parametrisarakkeissa olevan puuttuvan datan käsittely näyttää keskeneräiseltä, koska 'Null' ei nähtävästi kelpaa arvoksi lainkaan (ko. rivit jäävät kokonaan pois), ja 'Undefined'-arvot AWP muuntaa nolliksi, mikä johtaa harhaan.
- C) Kun menen katsomaan http://base.opasnet.org/, niin ko. study näkyy kylläkin pudotusvalikossa, mutta "post"-painikkeella ei palaudu mitään dataa. Juha varmaan fiksailee tuon kun ehtii.
This should be the Objects node (there is a bug): <anacode> Index j:= ['id','Ident','Name','Unit','Objtype_id','Acttype_id','Page','Wiki_id', 'Who','Comments']; index i:= concat(concat(N_vars,indices),['Run']); var Ident:= if @i <= size(N_vars) then Object_info[Info='Ident', @N_vars=@i] else if @i=size(i) then else i; var a:= Index_info[indices=i, Ind_info=j]; a:= if a=null then else a; var e:= Object_info[N_vars=i, Info=j]; a:= if e=null then a else e; var b:= sum(findintext(wikis,ident)*@wikis,wikis); var c:= if b=0 then else wikis[@wikis=b]; c:= if b=0 then '2664' else selecttext(ident,1+textlength(c)); var d:= Object_info[Info='Probabilistic?']; d:= if (d=1 or d='Yes' or d='Y' or d='yes' or d='y') then ' (probabilistic)' else ' (deterministic)'; d:= d[@N_vars=@i]; a:= array(j,[ @i, ident, if i ='Run' then 'Observations: ' & max(Data_table[.j='Obs'],Data_table.i) {&d}& ', Cells: '&size(cells.i) &', Indices: ' &size(loccells.i)/size(Cells.i) & ', Analytica '&Analyticaedition&', ('&Analyticaplatform& '), Version: '&textify(Analyticaversion) else a, a, if @i=1 then 1 else if @i=size(i) then 9 else 6, if @i=1 then 11 else 1, if i='Run' then '2817' else c&, if b=0 then '1' else b&, opasnet_username, a]); a:= if a = null then else a </anacode>
There is still a problem that the first variable uploads OK, but the others don't. I haven't found the reason for this yet.
See also
- Previous version image:RDB connection.ANA and image:RDB reader.ANA were only for either reading or writing data.
File history
Click on a date/time to view the file as it appeared at that time.
Date/Time | Dimensions | User | Comment | |
---|---|---|---|---|
current | 10:26, 21 August 2010 | (80 KB) | Jouni (talk | contribs) | Major simplifying revisions: lot of memory-saving improvements. Two input formats rejected. Intermediate nodes merged to save memory. Bugs corrected. |
19:30, 12 July 2010 | (92 KB) | Jouni (talk | contribs) | Now it is quick also with probabilistic data, as I changed while-do loop to for x[]:= b loop. In addition, the obs in a deterministic table is the row number, while before it was just zero. | |
13:42, 12 July 2010 | (92 KB) | Jouni (talk | contribs) | As expected, Cellrow node was too heavy with large data. This part was streamlined. Now it is quick and simple with deterministic data, but takes a long time with large probabilistic data (little memory need, though). | |
22:22, 11 July 2010 | (92 KB) | Jouni (talk | contribs) | Still some bugs found but it seems to work now. | |
05:05, 10 July 2010 | (107 KB) | Jouni (talk | contribs) | Model tested with several options and now it always works! This is now version 2.0. All descriptions were moved to file:Opasnet base connection.ANA (technical) and Uploading to Opasnet Base (method) and updated. | |
14:56, 9 July 2010 | (121 KB) | Jouni (talk | contribs) | Tested and now it works except bugs were found from W_Actobj and Cell, because they depend on uploaded tables and the update too fast. Maybe a static node in between solves this problem. | |
23:05, 8 July 2010 | (136 KB) | Jouni (talk | contribs) | Update finalised and first tests are successful. The code has become much simpler, it can even been understood by outsiders. | |
14:11, 8 July 2010 | (126 KB) | Jouni (talk | contribs) | Code streamlined. All different upload types seem to work to Data_table phase but not further. Added observise function to work with table-based prob data. | |
22:50, 7 July 2010 | (108 KB) | Jouni (talk | contribs) | Index info was rejected because it caused trouble with updating, was complex, and didn't bring anything essential. Some bugs removed. Should work now but haven't tested yet. Code starts to be simple and beautiful. Stuff moved to Old parts. | |
15:13, 7 July 2010 | (105 KB) | Jouni (talk | contribs) | Uploading part streamlined and corrected. Not all upload methods worked properly. There is still some problem. |
You cannot overwrite this file.
File usage
The following 3 pages use this file:
- File:RDB connection.ANA (file redirect)
- File:RDB reader.ANA (file redirect)
- File:Transferring to result database.ANA (file redirect)