How to bulk delete Assets objects efficiently

Summary

Assets currently allows technically infinite (read that as "a lot") number of objects to be bulk deleted. Basically, whatever the number of objects returned in the IQL search falls within the scope of the delete operation.

This way of handling bulk deletion causes:

• The UI progress "busy" animation stops even when the deletion is progressing in the background

• Indeterministic impression of results (sometimes 150 objects will be deleted all at once, sometimes it will still show 10 objects left since the background task is still ongoing and index is not yet updated)

• Database locking exception (viz., OptimisticLockException since user might think that the delete operation missed out on few objects and run a second pass)

This has been recorded as an improvement area for Assets: JSDSERVER-7306 - Improve Insight Bulk Delete operation, and with 5.7 improvements have been released.

This Article intends to describe a few different ways of handling bulk delete operations more efficiently.

Environment

JSM datacenter with Asset Management

Diagnosis

This intends to be a "How to" article. However, as an Assets user/manager/admin one might run into any of the issues described in the summary section.

Cause

Depending on the type of deletion (all objects in an object type, some objects based on IQL etc.) Insight's bulk delete operation includes very heavy sequential operations on the DB viz., Delete the attribute values → delete the attributes → delete the objects → update all referenced objects to the deleting objects. Since there's no way of knowing all the references, currently insight does a scan of ALL OBJECTS and then updates the attributes of the existing objects referencing the deleting objects.

Solution

Here are a few techniques you can use to more efficiently bulk delete massive number of objects without the risk of being error prone:

Bulk Deletion via CSV import:

• Create an Import Configuration for the particular objecttype(s) you wish to delete with the following settings:

  • Create a CSV with a header row (matching the attributes of the objecttype)

  • Add just one row of actual data

  • Configure a CSV import for this file, and create an Object Type map for each Object Type you wish to delete all objects within

  • Make sure to set Missing Objects: Remove, with the Threshold 0 Synchronizations.

  • Map the CSV column (data locator) to your Object Type Label. (if there are any mandatory Attributes - map something into these (add column to the CSV, or map the same single column, depending on the Attribute's Type)

Now when you run this CSV import, it'll use the import engine to remove all the existing objects as we've configured Insight to think of them as "missing" objects. Once the import finishes, you'll have just one object created and you can delete that object manually.

Bulk Deletion via Assets Automation:

We can also use the Assets Automation engine to pick up objects we wish to delete and then use Assets REST API to delete them. Create an Automation rule like the following:

• When: These deletions are mostly one time activity. Please select the cron expression accordingly. Remember to disable the rule when not needed

• IF: Enter the AQL which returns the objects you wish to delete

• THEN: REST API http request: http(s)://<BASE_URL>/rest/insight/1.0/object/${Key}