Nantes Université

Skip to content
Extraits de code Groupes Projets
Vérifiée Valider fdef3a40 rédigé par Benoit SEIGNOVERT's avatar Benoit SEIGNOVERT
Parcourir les fichiers

Update agenda

parent 2b8b1ed7
Aucune branche associée trouvée
Aucune étiquette associée trouvée
Aucune requête de fusion associée trouvée
%% Cell type:markdown id:0ef87507-4419-480b-9e05-2e1444af60d1 tags:
<center><img src="https://s3.glicid.fr/nuts/workshop-banner.svg"/></center>
# Welcome to the NuTS workshop praticals
The praticals were prepared and will be presented by [Leonard Sédoux](https://leonard-seydoux.github.io/) with the help of [Benoît Seignovert](https://benoit.seignovert.fr).
> This Jupyter environment **is already preconfigured** for you and you just have to click on the links above to open the notebooks and edit them.
> If you want to, you can download the notebooks individually with the file explorer (on the left)
> All the notebooks are also available in the [NuTS Gitlab repo](https://gitlab.univ-nantes.fr/nuts/tp/machine-learning).
## Agenda
### Thursday, June 1<sup>st</sup>
#### Atelier 1 - First steps with data preparation (9h-12h)
Here we will investigate the basic statistical properties and some feeling of the data. Do we have sufficient training points? Is the data stationary? What features will be relevant to solve the task at hand? Can we avoid overfitting?
- [Basics of programming with Python and scientific libraries](machine-learning/1_inspection/1_check_basics.ipynb)
- [River load sensor calibration](machine-learning/1_inspection/2_calibration.ipynb)
- [Classification of Iris dataset using various classifiers](machine-learning/2_iris/iris_classification.ipynb)
🍴 Lunch break and posters (12h-14h)
#### Atelier 2: Machine learning approches (14h-18h)
If deep learning is powerful to solve a vast majority of problems, machine-learning approches, if successful, provide more insight about the physics at play for solving problems. We will investigate and criticize several supervised and unsupervised approaches to solve the task of classifying clouds of Lidar points (scenes) to infer the type of structure visible therein.
- [Label a subset of a lidar point cloud](machine-learning/3_lidar/label.ipynb)
- [Three-dimensional lidar data classification of complex natural scenes with multi-scale features](machine-learning/3_lidar/lidar.ipynb)
### Friday, June 2<sup>nd</sup>
#### Atelier 3: Deep Learning with Pytorch (9h-12h)
When the attempts to solve the problem are unsuccessful or if defining features is a challenge because of the input data's dimension, we can also learn the features that best solve the task. Here we will revisit the problem of Lidar with deep-learning approaches and see how we can efficiently design a neural network to solve the classification task.
- [MNIST classification with a fully connected neural network](deep-learning/session_1a_fcnn.ipynb)
- [MNIST classification with a convolutional neural network](deep-learning/session_1b_cnn.ipynb)
- [Understanding and training PhaseNet on a local dataset](deep-learning/session_2_phasnet.ipynb)
🍴 Lunch break and posters (12h-14h)
#### Atelier 4: Deeper dive in artificial intelligence (14h-15h30)
This last class will be a free hands class on various problems. We invite people to bring datasets and tasks of interest so we can sit and try machine-learning solutions for solving them. We will also dive deeper into algorithms, try to improve performances and discuss some good practices in machine and deep learning.
%% Cell type:markdown id:096bcfe7-b208-4567-b9ee-8290b8dc325e tags:
## Explore the JupyterHub configuration
<center><img src="https://s3.glicid.fr/nuts/jupyterhub.png" width="70%" /></center>
%% Cell type:markdown id:503b8801-c0bb-4bb7-aada-bf98fd93ee43 tags:
### Users ressources
%% Cell type:code id:1760d41f-86a6-4b7f-8806-b5557762e7b1 tags:
``` python
from os import environ
from psutil import cpu_count, virtual_memory
```
%% Cell type:code id:f07d76ac-e6d0-4fe5-8130-0fdd36ac815c tags:
``` python
print('Username: ', environ.get('SLURM_JOB_USER'))
print('Node name: ', environ.get('SLURMD_NODENAME'))
print('vCPU: ', environ.get('SLURM_CPUS_ON_NODE'))
print('RAM: ', int(environ.get('SLURM_MEM_PER_NODE', 0)) // 1024, 'GB')
print('Node vCPU: ', cpu_count(logical=True))
print('Node RAM: ', virtual_memory().total // 1024 ** 3, 'GB')
```
%% Output
Username: user-id@domain.fr
Node name: budbud017
vCPU: 6
RAM: 24 GB
Node vCPU: 64
Node RAM: 251 GB
%% Cell type:markdown id:ac43551d-8768-4f9b-9d19-41b6ff9f2660 tags:
### Python packages
%% Cell type:code id:47a04f49-75b4-45ca-977e-a27f4a1af324 tags:
``` python
from platform import python_version
import numpy, matplotlib, seaborn, pandas, torch, sklearn, obspy, seisbench
```
%% Cell type:code id:fd3e0810-01cc-4017-a6e4-25542a63bb84 tags:
``` python
print('Python: ', python_version())
print('Numpy: ', numpy.__version__)
print('Matplotlib: ', matplotlib.__version__)
print('Seaborn: ', seaborn.__version__)
print('Pandas: ', pandas.__version__)
print('PyTorch: ', torch.__version__)
print('Scikit Learn: ', sklearn.__version__)
print('Obspy: ', obspy.__version__)
print('Seisbench: ', seisbench.__version__)
```
%% Output
Python: 3.9.16
Numpy: 1.24.3
Matplotlib: 3.7.1
Seaborn: 0.12.2
Pandas: 1.5.3
PyTorch: 1.13.1
Scikit Learn: 1.2.2
Obspy: 1.4.0
Seisbench: 0.4.0
%% Cell type:markdown id:adc1930d-503c-4caf-b79a-c226b39ddf7d tags:
### PyTorch GPU configuration
%% Cell type:code id:7bc5a46e-caac-4ac6-94c8-efd317a9b8d2 tags:
``` python
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print('Using device:', device)
if device.type == 'cuda':
print('Cuda version:', torch.version.cuda)
for i in range(torch.cuda.device_count()):
print()
print(torch.cuda.get_device_name(i))
print('Memory Usage:')
print('Allocated:', round(torch.cuda.memory_allocated(i) / 1024 ** 3, 1), 'GB')
print('Cached: ', round(torch.cuda.memory_reserved(i) / 1024 ** 3, 1), 'GB')
```
%% Output
Using device: cuda
Cuda version: 11.7
NVIDIA A100-PCIE-40GB
Memory Usage:
Allocated: 0.0 GB
Cached: 0.0 GB
NVIDIA A100-PCIE-40GB
Memory Usage:
Allocated: 0.0 GB
Cached: 0.0 GB
%% Cell type:markdown id:55e02b91-907b-469a-b7e8-6de4d2c16599 tags:
### Seisbench datasets
%% Cell type:code id:e2c6e92d-c58c-473d-bdd0-5dddc4580c0f tags:
``` python
from seisbench.data import Iquique
```
%% Cell type:code id:af776464-eb23-4308-bf43-8244928d5d6b tags:
``` python
data = Iquique()
print(data)
data.metadata.head()
```
%% Output
2023-05-30 17:43:53,481 | seisbench | WARNING | Check available storage and memory before downloading and general use of Iquique dataset. Dataset size: waveforms.hdf5 ~5Gb, metadata.csv ~2.6Mb
Iquique - 13400 traces
index source_origin_time source_latitude_deg \
0 0 2014-05-01T00:52:20.970000Z -19.4527
1 1 2014-05-01T00:52:20.970000Z -19.4527
2 2 2014-05-01T00:52:20.970000Z -19.4527
3 3 2014-05-01T00:52:20.970000Z -19.4527
4 4 2014-05-01T00:52:20.970000Z -19.4527
source_longitude_deg source_depth_km path_back_azimuth_deg \
0 -69.9762 57.95 161.009411
1 -69.9762 57.95 311.334575
2 -69.9762 57.95 329.936310
3 -69.9762 57.95 327.358785
4 -69.9762 57.95 211.759639
station_network_code station_code trace_channel station_location_code ... \
0 7K BOLO EH* NaN ...
1 CX PB08 HH* NaN ...
2 DG IN17 HH* NaN ...
3 DG IN06 HH* NaN ...
4 DG IN10 HH* NaN ...
trace_sampling_rate_hz trace_completeness trace_has_spikes \
0 100.0 1.0 False
1 100.0 1.0 False
2 100.0 1.0 False
3 100.0 1.0 False
4 100.0 1.0 False
trace_start_time trace_P_arrival_sample \
0 2014-05-01T00:52:20.630000Z 2000.0
1 2014-05-01T00:52:19.048393Z 2000.0
2 2014-05-01T00:52:20.740000Z 2000.0
3 2014-05-01T00:52:09.279762Z 2000.0
4 2014-05-01T00:52:09.700000Z 2000.0
trace_S_arrival_sample trace_name_original trace_chunk \
0 3564.0 7K.BOLO.
1 NaN CX.PB08.
2 3551.0 DG.IN17.
3 2697.0 DG.IN06.
4 2761.0 DG.IN10.
trace_component_order split
0 ZNE train
1 ZNE test
2 ZNE dev
3 ZNE train
4 ZNE dev
[5 rows x 24 columns]
%% Cell type:markdown id:b4f41f45-c6b5-4ae1-9506-c6981f91343d tags:
<center><img src="https://s3.glicid.fr/nuts/workshop-footer.svg"/></center>
......
0% Chargement en cours ou .
You are about to add 0 people to the discussion. Proceed with caution.
Terminez d'abord l'édition de ce message.
Veuillez vous inscrire ou vous pour commenter