Platform Maintenance needs overhaul- Cleanup is single threaded activity not able to keep up with cleanup needs of large environments

Idea - BO Cleanup or platform cleanup needs to be multithreaded running multiple batches in parallel, should have ability to identify the data once and cleanup the identified data rather than identifying the cleanup data every day, should have ability to skip the deletes with 0 records and have option to target specific tables for targeted cleaning as per business needs.

Here are current problems with Platform Maintenance - 

• BO Cleanup is single‑threaded, non‑scalable and sequential process.
• Creation of IBS_SPEC_CA_DELETE table every iteration takes 45 mins almost everyday from 10 hours of cleanup time.
• BO cleanup does the cleanup sequentially running delete on tables returning 0 records which is highly inefficient.
• BO Cleanup when running in "Clean obsolete associations" mode runs for 5-6 hours and do not respect the clean timeout thus running in business hours frequently.
• No table‑level targeting or staged cleanup is supported in the cleanup.
• Cleanup cannot keep up with daily data growth for our largest market.
• Configuration tuning in terms of batch size or increasing memory alone is insufficient to address the issue.
• Manual SQL cleanup is unsupported and risky
• Ongoing backlog causes workflow failures and operational risk for the business.
• BO Cleanup can only be run in the non-business hours and as it locks tables during delete affects 24 * 7 applications.

why it is useful ?

• BO Cleanup should be optimized to fix the above problems like it should be able to pickup multiple batches and process through them in parallel, skip tables with 0 records to delete etc.

Who would benefit from it ?

• Large Tririga implementations where large number of records are marked for deletion every day, this will help keep the cleanup up to the data rather than building over time.

how should it work ?

• It should address the above mentioned problems. Ultimately, we should be able to predict its behaviour and optimize it to be able to keep up with the cleanup needs of large Tririga implementations.