Advanced Data Import (Templates)

Data Import Templates allow you to define how to upload data in a file. These can be saved for re-use or applied one-off for a data file. This allows you to upload files and create new Datasets immediately, without having to define rules for upload each time.

Data Import Templates are made using the following steps.

1

Upload File

To upload your file(s) you can click on “Upload File” in the “Add New Source” section. You can add one or multiple files.

Data Import Template Inital

2

Add Files and Tabs to a Workspace

Each file and tab (if using .xlsx) will be show in the left menu. You can add each file into the workspace by clicking “add”. Inside a file are ‘data regions’ – which are the areas where your data are located. The below image shows four files uploaded (left) and a data region (right).

Data Import Workspace

If your file has multiple data regions, such as those described in Data and File Layouts you can add the file from your workspace any number of additional times to capture additional data regions.

Extract Data

3

Define File layouts

Each data region will be automatically detected and shown to the left, with a preview of the final Dataset shown to the right. The below image shows a tabular layout as described in Data and File Layouts.

Data Region Tabular

If you have transposed data you can select the “Unpivot” option. Unpivot can operate on one or many rows and columns in the data region. For a transposed table you want to select everything except the first column using the “Selection Bounds”, as the first column’s values will become the new table’s column names.

Data Region Unpivot

Merge data

After you have defined your data file and data region you can define how they are mereged together to create a single Dataset. Merging of data regions occurs by matching values in each selected column to values in a selected column in another data region (join). You can merge by one pair of columns or by many pairs of columns.

If using many pairs of columns all conditions must be met for the data to be merged, else they will appear as unconnected rows in the final dataset. Merging by matching the columns should be used when different variables exist across data files and need to be merged together, resulting in a wider table (outer joining).

Data Merge

You can alternatively add a data region to your final Dataset as rows. This can be used when each data region contains the same kind of data, resulting in a longer table (concatenation). Once you are done merging all data regions into one, you can select “Finalize Dataset”.

Add Details and Finalize

Once in the finalize step, you can enter a name and click “Create Dataset”. After creating the Dataset you can enter the Dataset Name, Dataset Description, Link to ELN entry, and Tags. Completing this step will lead you to Dataset details page. You can now use this Dataset to create an analysis.

(Optional) Creating a Template

Alternatively you can select “Create Dataset and Template”.

Create Template

First you will be asked to name and describe the template. You additionally can define the template description and change the name for the required files. Changing the name lets other users know which file (or type of file) is expected. You can also use this to provide a short hint on what kind of file should be provided. The “file group” option should be used when the number of expected files is greater than 1. When supplying multiple files, all files of that type will be concatenated into one result, and then have merge rules applied as specified in the template.