Setup Libraries

These instructions install miniconda and the requisite libraries for the tutorial in a conda environment, which is an independent python installation.

Jupyter has the similar concept of kernels, which are independent execution environments. They don’t even have to be Python, kernels for other languages exist as well.

By loading a separate kernel for each project, we avoid the complication of different components/projects having weird interactions, ultimately helping reproducibility.

We first produce a new conda environment with the libraries we require, then we tell Jupyter about this new environment with the ipython executable

NOTE: This step may take a little while

%%bash
#!/bin/bash

KERNEL_NAME="machine-learning-hats"

# Download mambaforge install script
if [[ ! -f $HOME/mambaforge.sh ]]; then
    wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh -O $HOME/mambaforge.sh || { echo "Failed to download the mambaforge installation script."; exit $?; }
fi

# Install miniconda3
if [[ ! -d $HOME/mambaforge ]]; then
    chmod u+x $HOME/mambaforge.sh
    bash $HOME/mambaforge.sh -b -f -u -p $HOME/mambaforge
fi

echo "Sourcing the conda and mamba setup scripts..."
source $HOME/mambaforge/etc/profile.d/conda.sh
source $HOME/mambaforge/etc/profile.d/mamba.sh

# # Not updating miniconda since that results in an issues with mamba 
# # https://github.com/mamba-org/mamba/issues/1775
# echo "Updating the miniconda3 release..."
# conda update -y -n base -c defaults conda 2>&1 || { echo "Failed to properly update miniconda3."; exit $?; }

# Create conda environment
if [[ ! -d $HOME/mambaforge/envs/$KERNEL_NAME ]]; then
    echo "Creating the miniconda3 environment \"$KERNEL_NAME\"..."
    mamba env create -n $KERNEL_NAME -f environment.yml || { echo "Failed to properly setup the environment \"$KERNEL_NAME\" using mamba."; exit $?; }
fi
echo "Activating the kernel \"$KERNEL_NAME\"..."
conda activate $KERNEL_NAME || { echo "Failed to activate the kernel \"$KERNEL_NAME\"."; exit $?; }

python -m ipykernel install --name $KERNEL_NAME --user
echo "Installed \"$KERNEL_NAME\"!"

Sourcing the conda and mamba setup scripts...
Activating the kernel "machine-learning-hats"...
Installed kernelspec machine-learning-hats in /home/cms.rkansal/.local/share/jupyter/kernels/machine-learning-hats
Installed "machine-learning-hats"!

Results

If successful, you should see something similar to the following:

Installed "machine-learning-hats"!

The new kernel you just made will then show up in the various Jupyter notebooks after refreshing the page.