Create and configure a Flat File data source

The Flat File data source is a general connector to fetch data from a publicly accesible google sheet, CSV or TXT file into Data Manager . To use it, you create a Flat file job.

You’ll first add a data source, then configure it before running the fetch using the Adapters client or by initiating a Fetch from Cloud from within the UI.

Add a Flat File data source

To add a data source from a flat file (publicly accessible google sheet, CSV or TXT):

1. From the app switcher, select Assets.

2. Select Data Manager in the sidebar.

3. Select Object classes.

4. Select the Show data sources button beside the desired object class

5. Select + Add.

6. Under General, select Flat Files.

7. Click Next.

8. Complete all required (*) fields before clicking Configure your data source.

Configure a Flat File data source

In the Flat File Configuration screen, configure details about the data source:

Data Manager fields

You need to specify the following information from Data Manager:

• Mode of connection - Set whether you want to consume data using a cloud to cloud fetch or using the Data Manager adapters client.

• Data source name – the name of the data source used to run, transform, map, cleanse, and import the data.

• Enabled – leave this toggle enabled. Disabled jobs cannot be run.

• Visible – enable this if the data source is to be visible in the Object classes view.

• Short display name – a unique name as a data source label.

• Data source type – what type of data the tool is providing. For example, Assets, CMDB, user location. The data source type must be available for selection and can’t be created inline.

• File path – full path (including file name and extension) where the flat file is located. For example:

• macOS: /data-manager-files/CMDBComputers.csv

  • Windows: C:\data-manager-files\CMDBComputers.csv

• Wildcard – enable if multiple flat files with the same column structure should be combined (for example, data-file-*.csv consumes data-file-1.csv, data-file-2.csv, and so on).

  • When using a wildcard, the File path excludes filename and extension (it is just the directory path).

• Skip line – enter an integer if the header row is not on the first line. Default is 0.

• Delimiter – the character separating values in the file. Default is comma ,.

• Qualifier – a unicode character that wraps a string value so it is treated as a single value (for example, "Microsoft Windows, XP, SP2"). The most common qualifier is the double quote ".

• End with delimiter – enable only if each line in your flat file ends with a delimiter character.

The Mode of connection will define the following Data Manager field.

Data Manager Client

• Import data from - where the data is coming form: File path or Web address/es.

  • File path – full path (including file name and extension) where the flat file is located. For example:

    • macOS: /data-manager-files/CMDBComputers.csv

    • Windows: C:\data-manager-files\CMDBComputers.csv

  • Wildcard – enable if multiple flat files with the same column structure should be combined (for example, data-file-*.csv consumes data-file-1.csv, data-file-2.csv, and so on).

    • When using a wildcard, the File path excludes filename and extension (it is just the directory path).

Available for both Data Manager Client or Cloud to cloud.

• Web address/es - Enter up to 10 HTTPS web addresses, separated by a semicolon (;). Each site must have public access. The total size of all files cannot exceed 250 MB.

Notes:

• The publicly available web address/es can not required authorization details.

• For Google sheet web address/es a URL will normally look like https://docs.google.com/spreadsheets/d/11-6l-3P0EprlPTzlEtGdUVI0PUwYj-sMYBKMbqW2bROs/edit?gid=0#gid=0 we remove edit?gid=0#gid=0 from the end and append export?format=csv

Column Mappings

The Column Mappings section defines how each column in the file is interpreted:

• Source column name – the column name in the file.

• Source column type – data type for this column (string, date, integer, decimal, etc.).

• Date format – appears when the source column type is a date. Select the date format that matches the data; leave empty for non‑date columns.

• Destination column name – the column name to use inside Data Manager.

Any secure values (for example, passwords, secrets) entered while creating a job are stored encrypted and displayed as ********************. You can delete and re‑enter them when editing a data source.

You cannot map CSV/TXT data to a NULL column for a Flat File. The data type for such a column is set to string by default. If you need another type, use a Transform function (for example, Set Column type) to convert the data.

Generate column mappings automatically

Use the Auto-generate button to automatically create column mappings:

1. Copy the header row from your file (including delimiters).

2. Paste it into the Auto‑generate field.

3. Select + Auto-generate.

Create column mappings manually

To define mappings manually:

1. In Column Mappings, select + Add mapping for each column.

2. Fill out Source column name, Source column type, Date format (if required), and Destination column name.

To clear all mappings

If you need to adjust the mappings, or you want to start again. This is possible with the Clear all mappings button.

1. From the app switcher, select Assets.

2. Select Data Manager in the sidebar.

3. Select Object classes.

4. Select the Show data sources button beside the desired object class.

5. Select the ellipses (…) beside the data source to edit.

6. Select Edit data source.

7. Scroll to the bottom and select Clear all mappings.

8. You can now re-create your mapping automatically or manually.

9. Select Save.