Cleansing rules

Cleansing rules are automations that can be added to the cleanse operation to eliminate missing, erroneous, or duplicate information from your data. cleansing rules are linked to object classes.

There are two types of cleansing rules, which differ in how they remove data:

• Exclude and remove cleansing rules – rules that exclude and remove data from the Data Manager object.

• filter cleansing rules – rules that filter data from the Data Manager object. Filtered records are denoted with a ? for that data source when they are viewed as part of an advanced search.

Two cleansing rules are required for each data source:

• Exclude null or empty primary keys – This rule excludes objects with primary keys that either have no value (NULL). This rule is linked to the Reason Null or Empty primary key. This rule is typically the first rule in the list of rules.

• Remove primary key duplicates – This rule removes primary keys that are found to have duplicates. This rule is linked to the Reason Force Cleaned. This rule is typically the last rule in the list of rules.

If there are duplicates and there’s a need to keep the most recent record (for example, updated or created last), then add the cleansing rule Filter old duplicates, which will keep the appropriate record.

In total, there are six default cleansing rules. To see the default cleansing rules based on the object class and data source, go to Data Manager > Object classes > … > default cleansing rules.

Transforms that run during the Transform operation and the cleansing rules that run during the cleanse operation in Data Manager are not the same.