What are the requirements and limits of Data Manager?

Although Assets Data Manager is built to handle very large data sets, it does have a maximum load.

Each of these limits will have potential workarounds that may involve modifications to your local system, or may involve modifying how your data is structured in Data Manager.

What is the minimum disk space required?

The Cleanse and Import Client runs on your local system. Because this is a middleware client running locally, the amount of data it can handle is entirely dependent on your system hardware and configuration.

If you are loading large files or adding data frequently you will need to ensure that you have enough disk space to hold your data. To save disk space, consider removing files that have been processed more than 14 days ago.

If you are using data from tools or databases, but not files, the disk space requirements may be reduced.

For example:

If you have 10 CSV files that are less than 100mb, you will need 100mb * 14 days = 1.5gb in available disk space (on top of any OS requirements).

What is the minimum memory required?

The Cleanse and Import Client runs locally on your system, if you load large files or add data frequently you will need to ensure that you have enough RAM to run the client.

The memory requirements may be reduced if you are using data from files such as CSV but not tools or databases.

For example:

If you are using Flat Files and have 75,000 objects, we recommend having at least 8gb of memory available.

If you are using SQL or APIs and have 175,000 objects, we recommend having at least 8gb of memory available.

Some adapters also work where all data is loaded into memory, so the impact on memory should be reviewed and adjusted during the first load, as required.

The sizing guide below should help you determine the amount of storage you’ll need to account for.

Data Manager Sizing Guide

How long will an import take to complete?

Assets Data Manager runs in the cloud within Jira Service Management. The size of an import - and the time it will take to complete - is based upon the following three factors:

1. The number of attributes in the import.

2. The number of data sources to be imported.

3. The total number of records to be imported, calculated by adding together the total records for each data source.

If you find that your import is slow or unreliable when importing, there are a number of steps you can take to make it faster:

1. Reduce the number of attributes - In most cases, not all attributes from all data sources are needed. To reduce the size of your import, you can reduce the total number of attributes that are Imported by assigning them to <ignore> when you Configure your attribute mapping.

2. Add more cleansing rules - Cleansing rules help reduce the size of an import by removing duplicate or erroneous records.

3. Reduce the number of data sources - In some cases, not all Data Sources may be needed. Review your list of Data Sources and remove any that are not required.

What is the maximum number of Important attributes that are allowed?

Important attributes are attributes that are compared against data sources to ensure that they are correctly and consistently represented in your data. You may select any attribute as important by checking the Important checkbox when editing that attribute in the Attributes screen.

When an important attribute is Imported, it is compared against data sources. If the data for an attribute matches all the compared sources, it is given the Verified flag.

What is the maximum number of tags that are allowed?

Tags are customisable and searchable metadata that can be attached to Data Manager objects. Across all active object classes in Data Manager, you can use a maximum of 56 tags.