The Importance of Resilient Odoo Crons
by Muhammad Faizan
Odoo crons are scheduled tasks that run automatically at regular intervals to perform various background jobs, such as sending emails, generating reports, updating records, and more. These tasks are critical for the smooth functioning of an Odoo instance, and any issues with cron jobs can lead to disruptions in business operations.
Resilient Odoo crons refer to the ability of the system to handle and recover from cron failures or errors without causing significant disruptions to the system or business operations. This requires a robust and reliable cron management system that can detect, diagnose, and resolve cron failures in a timely and efficient manner.
Some of the key factors that contribute to resilient Odoo crons include proper monitoring and logging of cron jobs, proactive maintenance and optimisation of the system, and the use of best practices and industry standards for cron management. With resilient Odoo crons in place, businesses can ensure that their critical background tasks are running smoothly and efficiently, minimising the risk of downtime and ensuring uninterrupted operations.
Pitfalls
Use case 1: Create an invoice for a subscription every month.
I. The current implementation lacks robustness, meaning that if one subscription fails, all subscriptions fail.
II. The implementation is monolithic, which means that in case of a timeout, all data needs to be processed again, potentially resulting in an inconsistent state if emails have already been sent.
III. If an invoice is skipped one month, it will never be invoiced for that month.
IV. There is a high risk of concurrent updates due to long transactions updating data.
V. Running the function twice in the same month can result in two invoices being generated for the same subscription.
Use case 2: Synchronise contacts with any external tool every day.
I. It always processes all the data, which can cause issues as the amount of data grows.
II. There is a risk of timeout, and if it occurs, the function will retry forever in a monolithic manner.
III. If the function runs twice on the same day, it will synchronise twice, causing unnecessary duplication of work.
IV. To avoid these issues, the function should be set to run only once a day, and if the service is unavailable at that specific time, no synchronisation should occur that day.
Features of Resilient Odoo Crons
Idempotent: An operation that can be repeated without affecting the outcome beyond the original application.
- To ensure idempotency, it is important to make sure that data is only processed once.
- Keeping track of processed records helps ensure that data is not processed multiple times.
- With idempotency, it is safe to retry operations if they fail, as the operation will not cause any additional changes beyond the initial application.
Incremental: Incremental processing involves splitting data into small batches and saving it at the end of each batch.
- It's only possible if the process is already idempotent.
- Incremental processing can reduce transaction time, lower concurrent update risk and timeouts will be resolved eventually.
- However, data consistency at commit can be a concern and commits can be annoying for automated testing.
- Running the process more often than usual can help to avoid timeouts.
Robust: Prevent an issue on a single record from leading to a general failure.
- A crash and a timeout are not the same, and a retry is not possible in both cases.
- Ensure data consistency by rolling back when necessary.
- All correct records should be processed without any exceptions.
- Record processing may fail without generating any error, so it is important to identify the ones that failed using filters.
Complete: Ensure that all records are processed and none are left unprocessed.
- Ensure that every record is processed at some point and that no records are left behind.
- If a record is not processed, make sure it will be retried in the future to avoid leaving it behind.
- Process all records that can be correctly processed, and don't skip any that can be handled.
- Design your processing to be idempotent, so that repeating the processing on the same records won't cause problems.
- Be aware that processing problematic records can cause an infinite loop, and have a strategy to handle that situation.
- If too many records are failing repeatedly, it's important to have a mechanism in place to prevent infinite retries and avoid overwhelming the system.
Verbose: Log what is happening.
- When something went wrong.
- When processing is long.
- Processing records with an issue does not fail silently anymore.
- Don't bloat the logs.
Testable: Make automated testing easy.
- Tests should be designed to be rolled back, as it allows for easier testing and isolation of issues.
- Rolling back tests is not possible with commit, as committing changes to the database would make it difficult to undo changes made during testing.
- It is important to make it possible to disable commit during testing, in order to ensure that changes made during testing do not affect the production database.
- With the ability to roll back tests and disable commit during testing, there is no excuse for not thoroughly testing code before deploying it to production.
- By default, the behavior during testing should be to commit changes, unless otherwise specified to disable the commit.
Conclusion
- A resilient Cron requires specific characteristics to ensure it functions correctly.
- The Cron must be idempotent, incremental, robust, and complete.
- Additionally, it should be verbose and testable to enable easy debugging and automated testing.