Cleanse your data

This task requires Data Manager admin permissions.

The Cleanse process filters duplicate, erroneous, or unneeded information from your data. Cleansing runs after a data source is fetched but before merging. It can be triggered manually from any data source configuration screen, but is usually triggered via the Cleanse and Import client from the command line or through a scheduled task where the Cleanse and Import client is ran.

Before cleansing:

• All data sources for the object class must be Mapped.

Note: It is recommended that when a new data source is added you work through all phases (fetch, transform (if required), map, cleanse and merge) before adding another one.

Cleanse your data manually

Cleanse a data source

1. From the sidebar, select Assets.

2. In Data Manager, select Object classes.

3. Select the Show data sources button beside the Object class.

4. Select the data source you wish to cleanse.

5. Select Modify cleansing rules.

6. Select Run cleanse.

7. Select Start to initiate a cleanse.

   • Cleansing starts and results update automatically.

   • If you leave the page, cleansing continues in the background.

You can view results in Data Manager > Object Classes > … (beside the object class) > Cleansing statistics, where you see how each cleansing rule affected your data.

Cleanse using the Cleanse and Import client (macOS)

Before you start:

• All data sources for the object class must be Mapped.

• You must have a Cleanse and Import token and your Workspace ID.

• Wrap any parameter with spaces in double quotes.

Steps

1. Open Terminal.

2. Navigate to the clients:

   cd /Users/Shared/assets_data_manager

3. Select the appropriate client:

   cd osx-arm64/assets-cleanse-and-import-client # or cd osx-x64/assets-cleanse-and-import-client

4. (Optional) View help:

   ./dm-cleanseimport --help

5. Run cleanse:

   ./dm-cleanseimport --run "cleanse" --workspace-id <workspaceId> --token <your-cleanse-import-token>

6. Optional parameters:

   • Run Cleanse and Import together: --run "full"

   • Limit by data source type: --type "My Data Source Type Name"

   • Limit by data source: --ds "My Data Source"

   • Limit by object class: --object "Compute"

   • Proxy configuration: --proxy, --proxy-user, --proxy-pass, --proxy-default-credentials, --bypass-invalid-certificate

A success message indicates completion and any errors encountered.

Cleanse using the Cleanse and Import client (Linux)

Steps are the same as macOS, with Linux‑specific client folders:

Optional parameters (--run "full", --type, --ds, --object, proxy options) are the same as above.

Cleanse using the Cleanse and Import client (Windows/DOS)

1. Open an elevated command prompt.

2. Navigate to the clients:

   cd Users\Public\assets_data_manager cd win-x64\assets-cleanse-and-import-client rem or cd win-x86\assets-cleanse-and-import-client

3. (Optional) View help:

   dm-cleanseimport.exe --help

4. Run cleanse:

   dm-cleanseimport.exe --run "cleanse" --workspace-id <workspaceId> --token <your-cleanse-import-token>

5. Optional parameters:

   • Run Cleanse and Import together: --run "full"

   • Limit by data source type: --type my_data_source_name

   • Limit by data source: --ds my_data_source

   • Limit by object class: --object Compute

   • Proxy options: --proxy, --proxy-user, --proxy-pass, --proxy-default-credentials, --bypass-invalid-certificate

A success message indicates completion and any errors encountered.