fix

01fe156c · mathieu Lagrange · 75a59fb8 · 01fe156c
--- a/2. Audio Content Analysis/tp/analysis of musical tones.ipynb
+++ b/2. Audio Content Analysis/tp/analysis of musical tones.ipynb
@@ -2,6 +2,7 @@
 "cells": [
  {
   "cell_type": "markdown",
+   "id": "b513b4fa",
   "metadata": {},
   "source": [
    "## Import tools"
@@ -10,6 +11,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "6ca01fe4",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -39,6 +41,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "e75f9acd",
   "metadata": {},
   "source": [
    "## Retrieve the tinysol dataset \n",
@@ -49,6 +52,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "e04e4326",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -61,6 +65,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "76448ed6",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -71,6 +76,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "0a254d5d",
   "metadata": {},
   "source": [
    "## Tones of 4 instruments in the C4-C5 pitch range"
@@ -79,6 +85,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "162d8c5f",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -100,6 +107,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "f14792ce",
   "metadata": {},
   "source": [
    "**Question:** listen to a B4 Flute tone"
@@ -108,6 +116,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "fd8ddc05",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -118,6 +127,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "4cf6149e",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -127,6 +137,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "8d587d16",
   "metadata": {},
   "source": [
    "**Question:** define a function selectRms(y, sr) that computes the amplitude envelope of this tone using the librosa.feature.rms method and select the audio corresponding to the time frame of 8 hop_length before the maximum value of the envelope and 16 hop_length after."
@@ -135,6 +146,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "6637c013",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -152,6 +164,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "faa4a89b",
   "metadata": {},
   "source": [
    "## Feature set (MFCCs and Chromas)\n",
@@ -162,6 +175,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "b5063953",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -184,6 +198,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "98036f27",
   "metadata": {},
   "source": [
    "## Display of the features sets\n",
@@ -202,6 +217,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "22340a35",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -236,6 +252,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "1164aab6",
   "metadata": {},
   "source": [
    "**Answer:** here"
@@ -243,6 +260,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "e4856c4f",
   "metadata": {},
   "source": [
    "**Question:** from the data in track.avg_mfcc, build a matrix X of shape [len(my_tracks), 12], and standardize it row-wise.\n",
@@ -259,6 +277,7 @@
  {
   "cell_type": "code",
   "execution_count": null,
+   "id": "efa62ea2",
   "metadata": {},
   "outputs": [],
   "source": [
@@ -290,6 +309,7 @@
  },
  {
   "cell_type": "markdown",
+   "id": "ee0406ae",
   "metadata": {},
   "source": [
    "**Answer:** here"
@@ -312,7 +332,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.8.5"
+   "version": "3.8.8"
  }
 },
 "nbformat": 4,

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:b513b4fa tags:

 ## Import tools

-%% Cell type:code id: tags:
+%% Cell type:code id:6ca01fe4 tags:

 ``` python
 # deal with matrices
 import numpy as np
 # progress bar
 import tqdm

 # principal component analysis
 from sklearn.decomposition import PCA

 # handle musical datasets
 import mirdata

 # deal with audio data
 import librosa
 from librosa.display import specshow

 # play audio
 import IPython.display as ipd

 # handle display
 %matplotlib inline
 from matplotlib import pyplot as plt
 from matplotlib import cm
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:e75f9acd tags:

 ## Retrieve the tinysol dataset

 https://forum.ircam.fr/projects/detail/tinysol

-%% Cell type:code id: tags:
+%% Cell type:code id:e04e4326 tags:

 ``` python
 tinysol = mirdata.initialize('tinysol')
 tinysol.download()
 # run this line in case of inconsistant results that may be due to database corruption
 # tinysol.validate()
 ```

-%% Cell type:code id: tags:
+%% Cell type:code id:76448ed6 tags:

 ``` python
 # listen a random example track
 example_track = tinysol.choice_track()
 ipd.Audio(example_track.audio_path)
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:0a254d5d tags:

 ## Tones of 4 instruments in the C4-C5 pitch range

-%% Cell type:code id: tags:
+%% Cell type:code id:162d8c5f tags:

 ``` python
 # select pitch range
 low_pitch = librosa.note_to_midi("C4")
 high_pitch = librosa.note_to_midi("C5")
 # select instruments
 my_instruments = [
    "Clarinet in Bb", "Flute",  "Violin", "Cello"
 ]
 # build selector
 my_tracks = {
    track_id: track
    for track_id, track in tinysol.load_tracks().items()
    if low_pitch<=track.pitch_id<high_pitch
    and track.instrument_full in my_instruments
 }
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:f14792ce tags:

 **Question:** listen to a B4 Flute tone

-%% Cell type:code id: tags:
+%% Cell type:code id:fd8ddc05 tags:

 ``` python
 # answer here
 ####
 ```

-%% Cell type:code id: tags:
+%% Cell type:code id:4cf6149e tags:

 ``` python
 # frame rate of representations in samples
 hop_length = 512
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:8d587d16 tags:

 **Question:** define a function selectRms(y, sr) that computes the amplitude envelope of this tone using the librosa.feature.rms method and select the audio corresponding to the time frame of 8 hop_length before the maximum value of the envelope and 16 hop_length after.

-%% Cell type:code id: tags:
+%% Cell type:code id:6637c013 tags:

 ``` python
 def selectRms(y, # audio signal
              sr # sampling rate
             ):
 # answer here
 ####

 ####

 ys = selectRms(*fluteB4[0].audio)
 print(f'Original size: {y.shape[0]}. New size: {ys.shape[0]}')
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:faa4a89b tags:

 ## Feature set (MFCCs and Chromas)

 **Question:** compute the MFCCs and the Chroma representations  for each track. Once computed

-%% Cell type:code id: tags:
+%% Cell type:code id:b5063953 tags:

 ``` python


 for track in tqdm.tqdm(my_tracks.values()):
    # select audio
    y = selectRms(*track.audio)

    # compute MFCC
    mfcc = librosa.feature.mfcc(y, sr, n_mels=40, n_mfcc=12)
    # average over time
    track.avg_mfcc = np.mean(mfcc, axis=-1)

    # compute chroma
    chroma = librosa.feature.chroma_cqt(y, sr)
    # average over time
    track.avg_chroma = np.mean(chroma, axis=-1)
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:98036f27 tags:

 ## Display of the features sets

 **Question:** from the data in track.avg_chroma, build a matrix X of shape [len(my_tracks), 12], and standardize it row-wise.

 Display the dataset by considering the first and second features.

 Apply a Principal Component Analysis (PCA) and display the dataset considering the coordinates of each track along the first and second eigenvectors.

 Is the display improved ?

 Why ?

-%% Cell type:code id: tags:
+%% Cell type:code id:22340a35 tags:

 ``` python
 # CHROMA

 # answer here
 Y = np.random.randn(len(my_tracks), 2) # replace this line
 ####

 ####

 # pitch labels
 pitch_ids = np.array([track.pitch_id for track in my_tracks.values()])
 # prepare figure
 plt.figure(figsize=(5, 5))
 plt.xlim(-2.75, 2.75)
 plt.ylim(-2.75, 2.75)
 # plot each pitch class
 for pitch_class in range(low_pitch, high_pitch):
    sub_Y = Y[pitch_ids==pitch_class, :]
    hue = (pitch_class-low_pitch) / 12
    color = cm.hsv(hue)
    plt.scatter(
        sub_Y[:, 0], sub_Y[:, 1],
        color=color, label=librosa.midi_to_note(pitch_class)[:-1])
 plt.grid(linestyle="--")
 plt.legend()
 plt.tight_layout()
 #plt.gca().set_facecolor("black")
 #plt.legend()
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:1164aab6 tags:

 **Answer:** here

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:e4856c4f tags:

 **Question:** from the data in track.avg_mfcc, build a matrix X of shape [len(my_tracks), 12], and standardize it row-wise.

 Display the dataset by considering the first and second features.

 Apply a Principal Component Analysis (PCA) and display the dataset considering the coordinates of each track along the first and second eigenvectors.

 Is the display improved ?

 Why ?

-%% Cell type:code id: tags:
+%% Cell type:code id:efa62ea2 tags:

 ``` python
 # MFCC

 # answer here
 Y = np.random.randn(len(my_tracks), 2) # replace this line
 ####

 ####

 # instruments labels and markers
 instruments = np.array([track.instrument_full for track in my_tracks.values()])
 markers = ["o", "s", "v", "x", "^"]
 # prepare figure
 plt.figure(figsize=(6, 6))
 plt.xlim(-6, 6)
 plt.ylim(-6, 6)
 # plot each instrument
 for instrument_id, my_instrument in enumerate(my_instruments):
    sub_Y = Y[instruments==my_instrument, :]
    plt.plot(
        sub_Y[:, 0], sub_Y[:, 1],
        markers[instrument_id],
        label=my_instrument)
 plt.legend()
 plt.grid(linestyle="--")
 ```

-%% Cell type:markdown id: tags:
+%% Cell type:markdown id:ee0406ae tags:

 **Answer:** here