Importing Data with the Ingest API

What is Data Ingest?

Data Ingest is a feature in SysTrack that allows cloud customers to import external user or device data into SysTrack from CSV files and make it available through report views in dashboards, groups, and queries. Users may also refer to this process as data ingestion or data ingress. A typical use case is enriching SysTrack data with business-owned attributes such as department, asset tag, purchase date, employee ID, or other fields that are maintained outside SysTrack.

IMPORTANT: Currently, the Data Ingest feature is API-only. All operations run through REST endpoints, and no dedicated UI is available. This guide describes the recommended workflow, from schema design through audit review.

How Ingest Works

The Ingest API uses a staged process to provide validation and feedback that helps prevent failed or incorrect imports:

1. You define the target schema for users or devices and send it to SysTrack.

2. You send a CSV file to SysTrack that matches the defined schema.

3. SysTrack analyzes the uploaded file and reports any differences between the CSV and the current schema.

4. You review the differences and decide how to proceed. This step ensures that you validate schema changes and data updates before ingestion.

5. You explicitly call the process endpoint to start ingestion.

6. SysTrack ingests the data. After the first successful ingest run, the data is available through the RPT_EnrichedDevices and RPT_EnrichedUsers report views.

7. SysTrack associates each ingested device with its WGUID, optimized to run during off-hours to minimize impact.

8. Ingested users do not need to be associated with SysTrack data. Use email as a unique identifier to map user data.

You can use the Ingest API to update your imported data as often as needed.

Data Ingest Limitations

• Only CSV files are supported.

• You can upload up to 2,000 bytes of metadata per row in a CSV file.

• A CSV file can contain rows for up to 125% of your current device or user count.

• The schema must be in the JSON format.

Prerequisites

Before you start, ensure you have:

• A valid Ingest API key for authentication. See Manage API Keys.

• A CSV file with data you are importing: users or devices.

• For devices, a key column type defined for mapping:

  • Use SystemName when device rows should be matched by system name.

  • Use Serial when device rows should be matched by serial number.

• A validated column design:

  • Column names contain only letters, digits, and underscores.

  • Supported data types are int, double, datetime, guid, boolean, and string.

  • String columns have a length value greater than 0.

Step 1: Build the Schema

If you do not have a saved schema, create one from your CSV file during the following steps or use the provided examples.

Start From the Matching Template

• If importing devices, start from IngestAPI_Devices_Template.csv.

• If importing users, start from IngestAPI_Users_Template.csv.

{
   "keyColumn": {
      "name": "SystemName",
      "keyType": "SystemName"
   },
   "columns": [
      {
         "name": "SystemName",
         "type": "string",
         "length": 100
      },
      {
         "name": "AssetTag",
         "type": "string",
         "length": 50
      },
      {
         "name": "DepartmentGuid",
         "type": "guid"
      },
      {
         "name": "SupportTier",
         "type": "int"
      },
      {
         "name": "PurchasePrice",
         "type": "double"
      },
      {
         "name": "PurchaseDate",
         "type": "datetime"
      },
      {
         "name": "IsLoaner",
         "type": "boolean"
      }
   ]
}

{
   "columns": [
      {
         "name": "Username",
         "type": "string",
         "length": 100
      },
      {
         "name": "EmployeeId",
         "type": "int"
      },
      {
         "name": "Department",
         "type": "string",
         "length": 50
      },
      {
         "name": "HireDate",
         "type": "datetime"
      },
      {
         "name": "IsManager",
         "type": "boolean"
      }
   ]
}

Draft the Schema from Your CSV Headers

1. Create a JSON file for your schema in a text editor.

2. Add one schema column entry for each CSV header.

3. Infer type per column using real sample values:

   • boolean: values like true or false.

   • int: whole numbers only.

   • double: decimal numbers.

   • datetime: ISO-like timestamps such as 2025-01-15T00:00:00Z.

   • guid: GUID format values.

   • string: anything else.

4. For each string column, set length to a safe maximum expected value.

Add Device Key Mapping Explicitly (Devices Only)

1. Choose the matching strategy:

   • SystemName if your CSV has stable system name values.

   • Serial if your CSV has stable serial values.

2. Set keyColumn.name to that exact CSV header.

3. Make sure that same column is also present in columns with the required type.

To get valid device key values, use the RPT_System_Config report view:

• For SystemName, use the SystemId column.

• For Serial, use the SerialNumber column.

Copy the values exactly as SysTrack stores them. In some environments, SystemId contains a fully qualified name. Do not shorten it. Devices without configuration inventory do not appear in RPT_System_Config.

Match the User Identifier Format (Users Only)

For user imports, SysTrack does not validate or change the user identifier value. You can ingest any format, for example email, UPN, or DOMAIN\user. To match ingested user data with SysTrack data in dashboards, groups, and queries, use the same format as the SysTrack column that you match it with.

Prepare the CSV File

Prepare the CSV file for the Ingest API so that its headers match the schema and its values match the column types. A file that does not meet these requirements can fail processing.

CSV File Format

Save the CSV file with UTF-8 encoding and use a comma as the delimiter. Excel in regions that use a decimal comma saves CSV files with semicolons as delimiters. The Ingest API reads such a file as one column.

CSV Header Names

CSV header names must match the column names in the schema. Matching ignores letter case. The Ingest API does not remove spaces from header names. A header with a space before or after the name, for example ' PurchasePrice ', does not match the schema column. In this case, the upload response lists the name with spaces under added and the schema column under deleted, and processing fails.

A common cause is a space after each comma in a header line that you typed, or spaces in Excel header cells.

CSV Values

The Ingest API keeps spaces in values. In string columns, spaces count toward the column length. In the device key column, remove spaces before and after the values, or the device rows do not match.

Every cell in an int, double, datetime, guid, or boolean column must have a value. An empty cell in these columns fails processing, even if the upload response does not show a problem. Cells in string columns can be empty.

Values in number, date, and boolean columns must use these formats:

IMPORTANT: In a double column, the Ingest API can accept the value 1,345 as 1.345 without an error. In a datetime column, dates are read in US order, so 03/04/2022 is read as March 4.

Save the CSV File from Excel

Excel saves each cell in a CSV file as it is shown on the screen. Before you save the file from Excel, format the columns:

1. Format number columns as Number with the required decimal places, and clear Use 1000 Separator (,).

   Do not use the Comma Style, Accounting, or Currency formats. These formats add thousands separators and padding spaces.

2. Format date columns with a custom format, for example yyyy-mm-dd.

3. Before you paste the device key values, format the device key column (SystemName or Serial) as Text. Otherwise, Excel removes leading zeros and changes long numbers to scientific notation.

4. Save the file as CSV UTF-8 (Comma delimited).

Step 2: Submit the Schema

• PUT /ingestapi/tableType/devices/schema

• PUT /ingestapi/tableType/users/schema

• SysTrack accepts the schema update.

• Old ingested records remain (if any).

Step 3: Upload the CSV File

• POST /ingestapi/tabletype/devices/csv

• POST /ingestapi/tabletype/users/csv

1. Send the CSV body with the text/csv content type.

2. Review the response sections:

   • added

   • deleted

   • existing with isChanged: false and isChanged: true

• The upload succeeds.

• The response shows a schema-difference analysis that indicates whether you can continue.

• Upload does not automatically start processing.

• If you upload a second CSV before processing the first, it replaces the first file.

Step 4: Decide Whether to Adjust the Schema

• The upload response reflects the expected structure.

Step 5: Start Processing

• POST /ingestapi/tabletype/devices/process

• POST /ingestapi/tabletype/users/process

1. Send the process request.

2. Save the returned ingestId. It is required to track the run result.

• The response returns a new ingestId.

• If processing cannot start, the response explains whether a run is already in progress, no schema exists yet, or the uploaded CSV file is missing.

• If the endpoint reports that the file is already being processed, do not start another run. Use the ingestion ID from the error message to monitor the existing run.

Step 6: Monitor the Run Status

• GET /ingestapi/status/{ingestId} (see Get ingest status)

1. Poll the endpoint until the status reaches a final state.

2. If the run fails, start it again.

• You know whether the run completed successfully or failed.

NOTE: The RPT_EnrichedDevices and RPT_EnrichedUsers report views are available only after the first ingest run for that table type completes successfully. Until then, dashboards, groups, and queries that use these views can show errors. Before you use the views, make sure that the run status shows a successful run.

Step 7: Remove Uploaded or Current Ingest Data When Needed

• DELETE /ingestapi/tabletype/devices

• DELETE /ingestapi/tabletype/users

• The current ingest data for the selected table type is removed.

Step 8: Review Ingest History

• GET /ingestapi/history (see Get ingest history)

1. Review recent runs.

2. Filter by category to focus on users or devices.

3. Filter by startTime and endTime when investigating a specific time range.

• You can confirm start time and end time of each run.

Step 9: Review Audit Records

• GET /auditapi/ingest/actions (see Get ingest action audit records)

• GET /auditapi/ingest/actions/{actionType} (see Get ingest action audit records by type)

1. Review all ingest-related actions in a time range.

2. Filter by runBy for operator-specific auditing.

3. Filter by actionType.

• You can see who ran ingest actions, from which IP address, and when.

Ingest API Troubleshooting

Schema Errors

Upload Errors

Process Errors