Published July 23, 2019
| Version v1
Presentation
Open
Turning HPC Systems into Interactive Data Analysis Platforms using Jupyter and Dask
Description
This talk demonstrates how to use Dask and Jupyter on large high-performance computing (HPC) systems to scale and accelerate large interactive data analysis tasks -- effectively turning HPC systems into interactive big-data platforms. We will introduce dask-jobqueue which allows users to seamlessly deploy and scale dask on HPC clusters that use a variety of job queuing systems such as PBS, Slurm, SGE, and LSF. We will also introduce dask-mpi, a Python package that makes deploying Dask easy from within a distributed MPI environment.
Files
interactive-supercomputing-dask-jupyter.pdf
Files
(10.3 MB)
Name | Size | Download all |
---|---|---|
md5:78989fa4f379ffc630a221b2c5c841c6
|
10.3 MB | Preview Download |