https://meltedinhex.com/tags/distributed-tasks-in-python/ Recent content in Distributed Tasks in Python on Melted in Hex https://meltedinhex.com/images/og-social.png https://meltedinhex.com/images/og-social.png Hugo en-us Thu, 12 Jan 2017 23:30:00 +0530 https://meltedinhex.com/posts/distributed-processing-using-celery-in/ Thu, 12 Jan 2017 23:30:00 +0530 https://meltedinhex.com/posts/distributed-processing-using-celery-in/ <p>Celery is an asynchronous task queue based on distributed message passing. Tasks are executed concurrently on one or more worker servers using multiprocessing, Eventlet or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).</p> <p><strong>Architecture:</strong></p> <p><a href="https://meltedinhex.com/images/distributed-processing-using-celery-in/celery_architecture-d8d5269c.jpg"><img loading="lazy" src="https://meltedinhex.com/images/distributed-processing-using-celery-in/celery_architecture-d8d5269c.jpg"></a></p> <p>Fig1 : Celery architecture</p> <p>The main part of this architecture is the broker (transporter), which handles all the task processing.</p> <p>The client sends tasks to the broker, and the broker uses round robin to distribute those tasks to workers.</p>