During WordPress development, we frequently face limitations that prevent us from completing tasks due to execution time and/or memory constraints.
To overcome these limitations, we need to divide our tasks into smaller ones, often triggered by Cron jobs. To know more about Cron, please refer this article about Cron.
Each Cron execution initiates a new PHP process, and unless we establish a way to communicate the status of the work completed in the previous process, we may encounter difficulties.
WordPress applications commonly use the database to communicate job state or other common information between processes.
Another method to share information is through the file system, but this is discouraged in the enterprise WordPress ecosystem because many leading hosting companies operate WordPress in containers. Therefore, relying on the file system can be unreliable.
You can refresh your basics with this wiki page.