Saturday 11:30 a.m.–noon
Room 203 #pyconjp_203Running Dask in the Cloud
Shane Cousins
- Audience level:
- Intermediate
- Category:
- Distributed Computing
- Video:
- https://youtu.be/fJafLN8oCf0
Description
Is pandas running out of memory? Have you wanted to more easily process large scale data?
dask may be what your looking for.
Here I'll introduce the dask framework and talk about how you can get it running in the cloud
Abstract
Dask is a general purpose Spark-like big data computing framework that allows you to take advantage of Numpy/Pandas/Scikit-learn level complex algorithms, written in Pure Python.
This talk provides a brief introduction of dask and focuses on how a dask cluster can be setup in the cloud to get you started with migrating your pandas/numpy workflows to more easily work with and process larger datasets.