Python for Scientific Computing

Python for Scientific Computing#

Authors#

Susana Bueno, Maria Montagna, Sergio Orlandini

Description#

Python is an increasingly widespread language in various contexts and it is conquering an important space even within the scientific community. Thanks to its extreme readability, it constitutes an important candidate for the writing and management of highly complex algorithms. Its greater limitation compared to the traditionally most used languages in the scientific and technical computing field is the limited performance. However, a wide variety of high quality scientific libraries is available today in Python and allows, through low-level tailored implementations, to make a vast number of highly optimized algorithms usable while maintaining the simplicity of the Python language. In this course, which assumes the knowledge of the fundamental elements of the language, we will discuss the fundamental elements of the most used scientific libraries using Python giving the student a look at the correct setting to be given to a calculation-oriented Python code. To improve understanding, the different modules of the course are immediately put into practice in hands-on sessions in which students and teachers can interact directly on simple but significant concrete problems proposed in the exercise.

By the end of the course each student should be able to:

  • know the most important numerical libraries available in Python

  • write a Python program/module using the most important Python numerical libraries

  • understand the best practices for programming scientific applications in Python

Target audience#

Researchers and programmers who want to use Python to write and manage scientific applications.

Pre-requisites#

  • Basic knowledge of the Python programming (e.g. from the Cineca course Introduction to Python programming).

  • Knowledge of Jupyter Notebook environment.(For more details please visit https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/what_is_jupyter.html)

Download/Clone repository#

git clone https://gitlab.hpc.cineca.it/cineca-hpyc/python-scientific-2023.git

Agenda:#

Day 1

09:00 - 09:30  Registration
09:30 - 10:00  Introduction to Scientific Python Computing
10:00 - 10:30  Cluster Introduction
10:30 - 11:00  Matplotlib
11:00 - 13:00  Numpy I
13:00 - 14:00  Lunch break
14:00 - 16:00  Numpy II
16:00 - 17:30  Hands-on

Day 2

09:30 - 12:00  Pandas
12:00 - 13:00  Hands-on
13:00 - 14:00  Lunch break
14:00 - 16:00  Scipy
16:00 - 17:30  Hands-on

Day 3

09:30 - 12:00  Speed-up Python
12:00 - 13:00  Hands-on
13:00 - 14:00  Lunch break
14:00 - 16:00  cudf/dask
16:00 - 17:30  Python packaging

Hands-on Sessions#

Hands on sessions can be held either locally on your laptop or on CINECA cluster Marconi100, using Jupyter notebooks.

On Marconi100#

To open a notebook directly on a compute node a double ssh tunnel is required following the steps below:

  1. On localhost (your laptop) open an ssh session to m100 login node (i.e: login.m100.cineca.it) from the shell (for Windows users use Putty) with the command:

ssh USERNAME@login.m100.cineca.it

All of you has already received a personal USERNAME (a08traXY) and a password.

Once you are on m100 login, submit an interactive job to get a compute node:

srun -N 1 --ntasks-per-node=4 --gres=gpu:1 -A tra23_PythSci -p m100_sys_test -q qos_test -t 04:00:00 --pty /bin/bash

The name of compute node will appear on the prompt once you got it (i.e.: r205n01):

USERNAME@rxxxnxx

Load the specific module containing python libraries:

module load autoload cineca-hpyc/2022.12

All python packages will be available in the module, you don’t need to create a virtual environment.

  1. On another shell from your local machine open a ssh tunnel to login node and from login node to compute node:

ssh -L 9999:localhost:9999 USERNAME@login.m100.cineca.it ssh -L 9999:localhost:9999 -N rxxxnxx

  1. Go back to the shell on the remote host (m100) and open the jupyter notebook on the selected port with the following command:

jupyter notebook --port=9999 --no-browser

  1. To access the notebook, open a browser on localhost and copy and paste the URL that will appear on the shell after you have lanched jupyter.

http://localhost:9999/?token=75f9c6d4611a636b3249cd79fe10b218ab1f1c267d4c53d13

On your laptop#

On your local machine you have to set a virtual environment in your work directory, install all required libraries (see the requirements.txt file to get the list of all the necessary packages for the couse), and activate the environment.

Contents