Skip to main content

Data sources

process-data-source

Data sources are the core elements of Qalita, they are referenced in the platform, the data are never copied and stored in the platform.

Add a source

Adding a source is currently only possible via the command line qalita.

When you add a source, it creates a qalita-conf.yaml configuration file in the ~/.qalita/ folder, which contains the source connection information.

Example:

version: 1
sources:
- config:
path: /home/aleopold/desktop/data/heart/
description: 11 clinical features for predicting heart disease events.
id: 1
name: Heart Failure Prediction Dataset
owner: admin
owner_id: 1
reference: true
sensitive: true
type: file
validate: valid
visibility: internal

In this example, the source is a local file, but it could be a database, a remote file, a data stream, etc.

PropertyDescription
versionConfiguration version
sourcesList of sources
configSource configuration, properties vary according to source type.
Unique source identifier, obtained by the platform after finalizing source referencing with the qalita source push command.
nameSource name
ownerName of user owning the source
owner_idIdentifier of the user who owns the source
typeType of source
If source is a reference, allows comparison with other sources.
sensitiveIf the source is sensitive, special authorization is required to access it see authorizations
validateIf the source is valid, this means that the customer has access to the source, and it can be referenced on the platform.
Visibility of source, can be public, internal or private. Restricts access to different user profiles on the platform.

Source types

Sources can be of any type, as long as packages are available to handle them. QALITA Platform provides no limitations on source types.