Koa - Rstudio & Jupyter Customization

Owned by davidls
Last updated: Dec 19, 2024
3 min read

How do I create a custom Rstudio Environment?

A custom Rstudio environment requires some manipulation from the command-line so that you can create what is know as a custom module collection named "ood_rstudio".   The basic ingredients for your custom Rstudio environment include:

  1. (Required) A version of R you wish to use

  2. Additional modules that fulfill the dependencies for the R libraries you wish to install

  3. Install the R library. Steps on how to perform an install.packages on Mana can be found here.

  4. Creating a collection with the specific name "ood_rstudio"

For example, if one wanted to use rgdal within rstudio, they would need to also add the gdal module to their custom collection and install the R library.  The steps would be as follows

Custom Rstudio environment
[user@login002 ~]$ srun -p sandbox -c 4 --mem=6g -t 60 --pty /bin/bash [user@node-0005 ~]$ module purge [user@node-0005 ~]$ module load llang/RStudio-Server/2023.12.1+402-gfbf-2023b-Java-11-R-4.4.1 [user@node-0005 ~]$ module load data/GDAL/3.9.0-foss-2023b [user@node-0005 ~]$ R [user@node-0005 ~]$ # Perform install.packages in R of rgdal and exit R [user@node-0005 ~]$ module save ood_rstudio [user@node-0005 ~]$ exit

Once the custom module collection has been made, it will be automatically loaded each time you start an Rstudio session using Mana's Open OnDemand portal.  While this method allows for customization of the Rstudio environment, it does not allow you to have more than one custom environment.  This means, if you wish to switch between version of R or go back to the default Rstudio environment, you need to overwrite the existing ood_rstudio collection or delete the collection.

How do I add custom environments to Jupyter Lab and Notebook?

While Rstudio is setup where it can only have a single customized environments, Jupyter * can have multiple custom environments.  This is possible because each custom environment is a self contained conda environment which is loaded as a unique Kernel.  

Custom Jupyter Kernel
[user@node-0005 ~]$ module purge [user@node-0005 ~]$ module load lang/Anaconda3 [user@node-0005 ~]$ mamba create --name test_env python=3 [user@node-0005 ~]$ source activate test_env [user@node-0005 ~]$ ### Install your required packages ### [user@node-0005 ~]$ # ... [user@node-0005 ~]$ ### Finalize setup with this last package ### [(test_env) user@node-0005 ~]$ mamba install ipykernel [(test_env) user@node-0005 ~]$ python -m ipykernel install --user --name test_env --display-name "test_env" [(test_env) user@node-0005 ~]$ exit

The above will create a  kernel named "test_env" which when you next start a jupyter * instance in Mana's Open OnDemand, will now be listed as a kernel you can utilize.

How can I customize what Jupyter Lab and Notebook version is loaded?

While users can customize the execution environment in Jupyter with many different kernels, customization of the version of Jupyter and supporting environment for Jupyter, can only have a single custom environment.  

A custom Jupyter environment requires some manipulation from the command-line so that you can create a custom conda environment named "ood_jupyterlab" that contains the custom version of jupyter you require. 

Customized Jupyter Environment
[user@node-0005 ~]$ module purge [user@node-0005 ~]$ module load lang/Anaconda3 [user@node-0005 ~]$ mamba create -n ood_jupyterlab -c conda-forge jupyterlab=3 xeus-python [user@node-0005 ~]$ exit

Once a kernel with this named is created, upon loading jupyter lab or notebook in Mana's Open OnDemand, this custom environment is used to launch  jupyter.

Custom Configuration Conflicts

In the case you run into problems and you previously were using jupyter environments outside of Mana's Open OnDemand, please make sure if you have any jupyter configuration files in your home directory, they are not named or located at the default location, or else your custom configuration settings will interfere with the jupyter instance that Open OnDemand is attempting to launch.

The reason for this is because Jupyter applies all parameters from a global config file and those specified at runtime https://jupyterlab.readthedocs.io/en/latest/user/directories.html#labconfig-directories.
For example:, if you have  ".jupyter/jupyter_server_config.py", you will want to rename it to something  like ".jupyter/jupyter_server_config_custom.py" and call it specifically in the cases you wish to utilize it in any Jupyter instances you launch outside of Open OnDemand when using Mana.