Batch Import of Data¶
The Importing Data user guide details how the application can be used to import data from a single file, either an Excel spreadsheet or a .csv file. However, you may also wish to import data from a collection of files in one batch using the same previously created data model. The Data Importer Tool allows the user to save a template detailing how to import data from a file, and this template may then be used to guide the import of data from other files in bulk using the Bulk Data Importer tool.
The Bulk Importer tool can be opened in three ways:
from the Welcome tab.
from double-clicking an Import Record in the Data Manager.
from a right-click context menu action on selected File entities in the Data Manager.
Limitations of the tool¶
The same limitations of the Data Importer Tool apply to the Bulk Data Importer tool. All files must conform to the same data orientation and format defined by the import template used for the batch import operation.
Model and Template selection¶
To import data from a set of files, the correct data model and import template must be selected. You may either double click on an Import Record in the Data Manager to open the bulk importer tool with the correct Import Record already selected, or select the data model and Import Record from drop-down menus in the tool.
After selecting an import template from the second drop-down menu, a summary of the template, including whether the entities are arranged column-wise or row-wise, the row/column containing the Entity names, the number format, and a list of importable FactKinds is displayed on the top right hand side of the Bulk Data Importer Tool.
If the Bulk Importer is opened from the Open in bulk importer right-click action in the Data Manager, the selected files will appear in the Files list in the tool. Otherwise, files to import can be added by dragging and dropping them from the Data Manager. The supported file types are the same as for the Importer tool: Excel spreadsheets (.xls, .xlsx and .xlsm), as well as CSV files (.csv) and text files (.txt).
Importing data and verifying results¶
When the list of files to import has been finalized the data import procedure may be triggered via the Start Import/Start Import Dry-run button.
If the checkbox for dry-run is enabled, triggering the import will attempt to detect FactKinds and Entities within each file and create Facts where data can be matched to an Entity, however no objects will be saved to the database. This mode is useful to verify the results of the import before saving data to the database. If the checkbox is not enabled, then triggering the import will also attempt to save all successfully imported data to the database.
If the import template does not completely match the file being imported, a warning dialog will pop up asking you to review the proposed matches
The results of the import or import dry-run will be displayed in the bottom table in a file by file manner. Each row in the results table details the number of successfully created Entities and Facts and also displays a status symbol indicating if data could be successfully imported from that file.
When importing data from a file, there are a number of errors that may prevent data from being imported. These errors generally fall into two classes:
Template errors are caused when the file to import does not conform to the data orientation or format specified by the selected import template and/or when the data in the file can not be matched to the selected data model. This class of errors prevents all data from a file from being imported. For example, if the file to import does not have Entity names in the row or column specified by the import template, then no Entities will be detected in the file and thus no data will be imported. This is an example of a Template error.
Data errors are caused by malformed data, generally due to a data type mismatch with the detected FactKind. This class of errors only prevents specific cells from being imported. For example, if the detected FactKind for a cell is "Tensile Strength (MPa)" which should have a float data type, and the cell containing data instead has a value "N/A" (a string type), then the Fact for that cell can not be created.
If data can not be imported from a given file due to a Template error, then either a new import template should be created from the Data Importer Tool, or a different data model should be used for import.