In this talk, the basic mechanisms of Celery and Docker-Swarm will be explained. With Docker-Swarm , a cluster will be built upon two Raspberry Pi machines. Hadoop entry-level "Word Count" program will be re-writen in Python and executed parallelly via Celery on the cluster. An example of distributed system modeling nerual-network will also be explained.

Hope this talk to induce some thoughts: 1) Hadoop/Spark are complicated; do Python's users have alternative tools to do distributed parallel computing? 2)Docker is hot; what does Docker actually mean to us? What kind of computational environment can Docker-Swarm provide? 3) With Celery and Docker-Swarm, how can a distributed system be built?

In the territory of Python, Celery is a famous distributed task queue framework, its Canvas mechanism is specially powerful in building complex workflow. Celery can function in distributed environment and go perfectly along with Docker-Swarm: Docker-Swarm provides a cluster environment and the worker containers to sustain Celery; worker containers and processes and be dynamically scaled and expanded to fulfill Celery's need, work together parallelly to accomplish the computation. With Celery and Docker-Swarm, an example will be provided, in which a cluster of 8 containers will be built upon two Raspberry Pi machines. Hadoop entry-level "Word Count" program will be re-writen in Python and executed parallelly via Celery on the cluster. Regarding concurrent/distributed computing, IoT, microservices, serverless-computing are all well known paradigms. Based on MQTT, IBM Bluemix is one of the leaders in this field, however, free of charge only for 30 days. With Celery and Docker-Swarm, and with respect to Bluemix, we can actually build a private IoT-liked system in minutes, free of any charge. To provide an example, a distributed system using Celery and Docker-Swarm modeling a neural-network will be explained.