Merge branch 'CH_master' into 'main'

# Conflicts: # example_demo/prediction.ipynb # example_demo/validation.ipynb

Merge branch 'CH_master' into 'main'
# Conflicts: # example_demo/prediction.ipynb # example_demo/validation.ipynb
34e21044 · Celine HOURCADE · 79a112f7 · cd839a80 · 34e21044 · 34e21044
--- a/example_demo/prediction.ipynb
+++ b/example_demo/prediction.ipynb
@@ -53,7 +53,9 @@
    "        * FR_CHLF_2022004T134407.mseed\n",
    "        * FR_GARF_2022004T134407.mseed\n",
    "        * FR_GNEF_2022004T134407.mseed\n",
-    "        * FR_VERF_2022004T134407.mseed"
+    "        * FR_VERF_2022004T134407.mseed\n",
+    "        \n",
+    "The name of the mseed file should be formatted as \"{network} _ {station} _ {time_event}.mseed\"."
   ]
  },
  {

 %% Cell type:markdown id:0f96ee4d-3f8a-4a95-b815-4da29fbf1e8e tags:

 # Prediction Mode

 %% Cell type:markdown id:0bd9e000-a2ee-4e96-a29f-c105ceec4c62 tags:

 <div class="alert alert-block alert-info">
 <b>This mode is used if the nature/label of the event is not known in advance. </b>
 </div>

 %% Cell type:markdown id:826842f6-e7ce-4d55-86f3-ce97b39b24b2 tags:

 ## Download data

 %% Cell type:markdown id:c02a24e2-742c-4ac8-b11d-8964284db1c3 tags:

 Input data should be in **mseed** format and correspond to raw **60 second 3-component seismograms**.

 %% Cell type:markdown id:a245431a-78d6-4fa1-9550-b52412a04c87 tags:

 Demo data is available in the "mseed" folder.

 %% Cell type:markdown id:eb0b3464-7061-49a8-8eee-da130ad17bae tags:

 To apply the algorithm, we need a folder architecture:
 * mseed_demo
   * 2022004T134407
        * FR_CHLF_2022004T134407.mseed
        * FR_GARF_2022004T134407.mseed
        * FR_GNEF_2022004T134407.mseed
        * FR_VERF_2022004T134407.mseed

+The name of the mseed file should be formatted as "{network} _ {station} _ {time_event}.mseed".
+
 %% Cell type:markdown id:631cae95-a1bc-46f8-8596-b11f0a6ee1b9 tags:

 ## Run prediction

 %% Cell type:markdown id:263b8345-4e16-47a6-b60f-8cba04e9cb69 tags:

 ### From a terminal:

 %% Cell type:markdown id:36d73373-310d-4557-ba3b-3cc62544c703 tags:

 ```
 python run.py --mode pred --data_dir ./mseed_demo --spectro_dir ./spectro_demo --output_dir ./output_demo
 ```

 %% Cell type:markdown id:f928f030-d567-4cd3-98bc-aba6d0c922ec tags:

 ### From a notebook:

 %% Cell type:code id:93f08547-af04-429d-9dd4-220fd3669d30 tags:

 ``` python
 import sys
 sys.path.append('../')
 from data_process import spectro_extract
 from prediction import pred
 ```

 %% Cell type:code id:1228c624-48f4-401d-9d0f-e76397188218 tags:

 ``` python
 data = spectro_extract(data_dir="./../mseed_demo", spectro_dir="./../spectro_demo")
 ```

 %% Output

    Number of events: 9
    *****************
    EVENT 1 / 9
    Number of streams: 14
    *****************
    EVENT 2 / 9
    Number of streams: 10
    *****************
    EVENT 3 / 9
    Number of streams: 8
    *****************
    EVENT 4 / 9
    Number of streams: 8
    *****************
    EVENT 5 / 9
    Number of streams: 11
    *****************
    EVENT 6 / 9
    Number of streams: 6
    *****************
    EVENT 7 / 9
    Number of streams: 7
    *****************
    EVENT 8 / 9
    Number of streams: 4
    *****************
    EVENT 9 / 9
    Number of streams: 10
    Stream 10 / 10

 %% Cell type:code id:26d663ad-ab6b-4c21-8c04-93f147e48256 tags:

 ``` python
 pred(model_dir="./../model/model_2021354T1554.h5", spectro_dir='./../spectro_demo', output_dir="./../output_demo")
 ```

 %% Output

    2022-09-20 16:48:22.650238: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
    To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

    Number of events: 9
    *****************
    EVENT 1 / 9
    Number of station: 15
    Station 3 / 15

    2022-09-20 16:48:22.857009: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)

    *****************
    EVENT 2 / 9
    Number of station: 10
    *****************
    EVENT 3 / 9
    Number of station: 16
    *****************
    EVENT 4 / 9
    Number of station: 8
    *****************
    EVENT 5 / 9
    Number of station: 11
    *****************
    EVENT 6 / 9
    Number of station: 12
    *****************
    EVENT 7 / 9
    Number of station: 7
    *****************
    EVENT 8 / 9
    Number of station: 4
    *****************
    EVENT 9 / 9
    Number of station: 10
    Station 10 / 10

 %% Cell type:markdown id:a059bc71-912e-4b2b-ab4a-0f7bf5c56b5c tags:

 ## Read csv file

 %% Cell type:code id:29e9f0ed-331c-41be-bad4-b9539a132baa tags:

 ``` python
 import pandas as pd
 ```

 %% Cell type:code id:e9c9f987-e2c7-4273-ad37-67aa2dab0d7b tags:

 ``` python
 pred_net_csv = pd.read_csv("../output_demo/prediction_network_level.csv")
 pred_net_csv = pd.DataFrame(pred_net_csv)
 pred_net_csv.head()
 ```

 %% Output

                event  prob_nat  prob_ant  pred         nature
    0  2022004T111040     0.006     0.994     1  Anthropogenic
    1  2022001T213524     0.961     0.039     0        Natural
    2  2022003T080315     0.043     0.957     1  Anthropogenic
    3  2022003T084110     0.019     0.981     1  Anthropogenic
    4  2022004T134407     0.770     0.230     0        Natural

 %% Cell type:code id:b2943f25-c309-488c-8e45-2ef919e6da57 tags:

 ``` python
 pred_sta_csv = pd.read_csv("../output_demo/prediction_station_level.csv")
 pred_sta_csv = pd.DataFrame(pred_sta_csv)
 pred_sta_csv.head()
 ```

 %% Output

                    file_name station  prob_nat  prob_ant  pred         nature
    0  FR_DAUF_2022004T111040    DAUF     0.001     0.999     1  Anthropogenic
    1  FR_LEUC_2022004T111040    LEUC     0.002     0.998     1  Anthropogenic
    2  FR_CRNF_2022004T111040    CRNF     0.014     0.986     1  Anthropogenic
    3   FR_AGO_2022004T111040     AGO     0.001     0.999     1  Anthropogenic
    4  FR_ABJF_2022004T111040    ABJF     0.000     1.000     1  Anthropogenic

--- a/example_demo/validation.ipynb
+++ b/example_demo/validation.ipynb
@@ -53,7 +53,9 @@
    "        * FR_CHLF_2022004T134407.mseed\n",
    "        * FR_GARF_2022004T134407.mseed\n",
    "        * FR_GNEF_2022004T134407.mseed\n",
-    "        * FR_VERF_2022004T134407.mseed"
+    "        * FR_VERF_2022004T134407.mseed\n",
+    "        \n",
+    "The name of the mseed file should be formatted as \"{network} _ {station} _ {time_event}.mseed\"."
   ]
  },
  {
@@ -297,6 +299,16 @@
     "name": "stdout",
     "output_type": "stream",
     "text": [
+      "Station 2 / 11\n",
+      "Station 3 / 11\n",
+      "Station 4 / 11\n",
+      "Station 5 / 11\n",
+      "Station 6 / 11\n",
+      "Station 7 / 11\n",
+      "Station 8 / 11\n",
+      "Station 9 / 11\n",
+      "Station 10 / 11\n",
+      "Station 11 / 11\n",
      "*****************\n",
      "EVENT 2 / 9\n",
      "Number of station: 10\n",

 %% Cell type:markdown id:0f96ee4d-3f8a-4a95-b815-4da29fbf1e8e tags:

 # Validation Mode

 %% Cell type:markdown id:0bd9e000-a2ee-4e96-a29f-c105ceec4c62 tags:

 <div class="alert alert-block alert-info">
 <b>This mode is used if the nature/label of the event is known in advance. </b>
 </div>

 %% Cell type:markdown id:826842f6-e7ce-4d55-86f3-ce97b39b24b2 tags:

 ## Download data

 %% Cell type:markdown id:c02a24e2-742c-4ac8-b11d-8964284db1c3 tags:

 Input data should be in **mseed** format and correspond to raw **60 second 3-component seismograms**.

 %% Cell type:markdown id:a245431a-78d6-4fa1-9550-b52412a04c87 tags:

 Demo data is available in the "mseed" folder.

 %% Cell type:markdown id:eb0b3464-7061-49a8-8eee-da130ad17bae tags:

 To apply the algorithm, we need a folder architecture:
 * mseed_demo
   * 2022004T134407
        * FR_CHLF_2022004T134407.mseed
        * FR_GARF_2022004T134407.mseed
        * FR_GNEF_2022004T134407.mseed
        * FR_VERF_2022004T134407.mseed

+The name of the mseed file should be formatted as "{network} _ {station} _ {time_event}.mseed".
+
 %% Cell type:markdown id:b61cddaf-cf4d-4d46-afc6-85e99d5c7514 tags:

 To apply this mode, a **csv file** import pandas as pd is also required with the associated label for each event.

 %% Cell type:code id:9bcac75b-0166-4346-975b-66b226902bb5 tags:

 ``` python
 import pandas as pd
 ```

 %% Cell type:code id:ec2b7b4d-ccb3-4884-97a6-7e7de00010bb tags:

 ``` python
 pred_net_csv = pd.read_csv("../demo_file.csv")
 pred_net_csv = pd.DataFrame(pred_net_csv)
 pred_net_csv.head()
 ```

 %% Output

                 time  label_cat
    0  2022004T134407          0
    1  2022003T041502          0
    2  2022001T213524          0
    3  2022004T111745          1
    4  2022004T111040          1

 %% Cell type:markdown id:631cae95-a1bc-46f8-8596-b11f0a6ee1b9 tags:

 ## Run validation

 %% Cell type:markdown id:263b8345-4e16-47a6-b60f-8cba04e9cb69 tags:

 ### From a terminal:

 %% Cell type:markdown id:36d73373-310d-4557-ba3b-3cc62544c703 tags:

 ```
 python run.py --mode pred --data_dir ./mseed_demo --spectro_dir ./spectro_demo --output_dir ./output_demo --csv_file ./demo_file.csv
 ```

 %% Cell type:markdown id:f928f030-d567-4cd3-98bc-aba6d0c922ec tags:

 ### From a notebook:

 %% Cell type:code id:93f08547-af04-429d-9dd4-220fd3669d30 tags:

 ``` python
 import sys
 sys.path.append('../')
 from data_process import spectro_extract
 from prediction import valid

 import numpy as np
 ```

 %% Cell type:code id:8ee5e6f3-68f5-4a5b-8d1b-246ef60c32a7 tags:

 ``` python
 events = np.genfromtxt(f'./../demo_file.csv', delimiter=',', skip_header = 1, dtype=str)
 ```

 %% Cell type:code id:1228c624-48f4-401d-9d0f-e76397188218 tags:

 ``` python
 data = spectro_extract(data_dir="./../mseed_demo", spectro_dir="./../spectro_demo", events_list=events)
 ```

 %% Output

    Number of events: 9
    *****************
    EVENT 1 / 9
    Number of streams: 11
    *****************
    EVENT 2 / 9
    Number of streams: 10
    *****************
    EVENT 3 / 9
    Number of streams: 10
    *****************
    EVENT 4 / 9
    Number of streams: 4
    *****************
    EVENT 5 / 9
    Number of streams: 14
    *****************
    EVENT 6 / 9
    Number of streams: 6
    *****************
    EVENT 7 / 9
    Number of streams: 7
    *****************
    EVENT 8 / 9
    Number of streams: 8
    *****************
    EVENT 9 / 9
    Number of streams: 8
    Stream 8 / 8

 %% Cell type:code id:26d663ad-ab6b-4c21-8c04-93f147e48256 tags:

 ``` python
 valid(model_dir="./../model/model_2021354T1554.h5", spectro_dir='./../spectro_demo', output_dir="./../output_demo", event_label=events)
 ```

 %% Output

    2022-09-20 16:48:29.640896: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
    To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

    Number of events: 9
    9
    *****************
    EVENT 1 / 9
    Number of station: 11
    Station 2 / 11

    2022-09-20 16:48:30.027344: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)

+    Station 2 / 11
+    Station 3 / 11
+    Station 4 / 11
+    Station 5 / 11
+    Station 6 / 11
+    Station 7 / 11
+    Station 8 / 11
+    Station 9 / 11
+    Station 10 / 11
+    Station 11 / 11
    *****************
    EVENT 2 / 9
    Number of station: 10
    *****************
    EVENT 3 / 9
    Number of station: 10
    *****************
    EVENT 4 / 9
    Number of station: 4
    *****************
    EVENT 5 / 9
    Number of station: 15
    *****************
    EVENT 6 / 9
    Number of station: 12
    *****************
    EVENT 7 / 9
    Number of station: 7
    *****************
    EVENT 8 / 9
    Number of station: 8
    *****************
    EVENT 9 / 9
    Number of station: 16
    Station 16 / 16

 %% Cell type:markdown id:a059bc71-912e-4b2b-ab4a-0f7bf5c56b5c tags:

 ## Read csv file

 %% Cell type:code id:29e9f0ed-331c-41be-bad4-b9539a132baa tags:

 ``` python
 import pandas as pd
 ```

 %% Cell type:code id:e9c9f987-e2c7-4273-ad37-67aa2dab0d7b tags:

 ``` python
 pred_net_csv = pd.read_csv("../output_demo/validation_network_level.csv")
 pred_net_csv = pd.DataFrame(pred_net_csv)
 pred_net_csv.head()
 ```

 %% Output

                event  label_cat  prob_nat  prob_ant  pred         nature
    0  2022004T134407          0     0.770     0.230     0        Natural
    1  2022003T041502          0     0.988     0.012     0        Natural
    2  2022001T213524          0     0.961     0.039     0        Natural
    3  2022004T111745          1     0.005     0.995     1  Anthropogenic
    4  2022004T111040          1     0.006     0.994     1  Anthropogenic

 %% Cell type:code id:b2943f25-c309-488c-8e45-2ef919e6da57 tags:

 ``` python
 pred_sta_csv = pd.read_csv("../output_demo/validation_station_level.csv")
 pred_sta_csv = pd.DataFrame(pred_sta_csv)
 pred_sta_csv.head()
 ```

 %% Output

                    file_name station  label_cat  prob_nat  prob_ant  pred  \
    0  FR_ABJF_2022004T134407    ABJF          0     0.044     0.956     1
    1  FR_CHLF_2022004T134407    CHLF          0     0.962     0.038     0
    2   FR_LBL_2022004T134407     LBL          0     0.216     0.784     1
    3  FR_GARF_2022004T134407    GARF          0     0.982     0.018     0
    4  FR_GNEF_2022004T134407    GNEF          0     0.914     0.086     0
    
              nature
    0  Anthropogenic
    1        Natural
    2  Anthropogenic
    3        Natural
    4        Natural