New Profiles

The following page documents the required steps for integrating new lectures or laboratories into the RWTHjupyter cluster.

We recommend that each course which plans to use Jupyter, creates a new profile in the cluster. After login, users will be greeted with a profile selection page from which they can choose their desired environment. We also will add a perma-link feature which allows you to generate a link which directly opens a particular profile & notebook. These links are the an adequate form to open Jupyter Notebooks from your Moodle course. A profile needs to be prepared and installed to the cluster before it can be used.

It consists of:

A name identifying purpose of the profile (or the lecture title)
A Git repository containing Jupyter notebooks which should be imported into the workspace by default
A Dockerfile sepcifying the runtime environment and optional third-party Python packages

We provide a list of generic profiles for the most common programming languages but offer the possibility to add new profiles for lectures including customizations like custom third-party dependencies, Jupyter kernels, etc.

Separation of runtime environment and Jupyter notebooks

A key point during the preparation of a new profile is the separation of the runtime enviroment from the Notebook content.

We use nbgitpuller so reguarily synchronize Jupyter notebooks from a Git repositry with the users workspace. This allows the instructors to gradually release the Notebooks alongside the timeline of the lecture. Each user maintains his own private clone of this Git repository and is able to make changes to the Notebooks which will be persisted between sessions.

On the contrary, the runtime enviroment is supposed to me more or less static and should not require frequent updates. All users of a profile will share the same runtime environment. Changes outside the home directory are not presistent and will be lost after logout.

Warning for Git repos

Students will get full read-only access to the Git repository your provide. In case you provide us with a token to synchronize private/internal repos, the students will also see this token. This means: Only share read-only tokens to repositories which dont contain sensitive information (also in other branches) as students will be able to access the repo.

Procedure

In general, the following steps are required for adding a new profile to the cluster:

Setup a local test environment for preparing your Jupyter Environment
Choose a Jupyter Kernel
Create a new Git repository for Jupyter Notebooks and runtime environment
Prepare Jupyter Notebooks
Collect Python/Third-Party requirements
Adapt Dockerfile
Submit request for inclusion in profile list
Wait until request is reviewed and the new profile is included in the list

Hereby, steps 4.-6. are optional and only required if the course requires a non-standard Jupyter kernel or special Python dependencies.

1. Setup a local test environment for preparing your Jupyter Environment

Please follow the instructions from the official Jupyter website: https://jupyter.org/install

For Windows users, we recommend to use the Anaconda Python distribution. Linux and macOS users can use Anaconda as well or simply rely on pip installed via the systems package manager.

2. Choose a Jupyter Kernel

Jupyter supports a variety of different programming languages via different kernels. The original and most commonly used kernel using the IPython interpreter. However also others exist. An updated lists of kernels can be found at the Jupyter wiki.

In principle, almost all of the existing kernels can be used on jupyter.rwth-aachen.de by creating custom runtime environments. Please consult the documentation for the installation of other kernels and add these instructions to the Dockerfile (step 6).

However, in most cases the standard IPython kernel is used and no further steps are necessary.

3. Create a new Git repository for Jupyter Notebooks and runtime environment

We recommend to use the RTWH Software Engineering Services / GitLab to manage, track, distribute Jupyter Notebooks and the definition of the required runtime environment.

This allows for a continous delivery of updates to Jupyter as well as welcomes contributions by Students to the Jupyter notebooks.

You can either use an existing Git repository, which might be already existing for a lecture or create a new one by forking our example profile.

Please using the following link to create a new fork of the example profile: create fork of example profile.

After the fork has been completed, you can clone the fresh repo to your local machine:

git clone git@git.rwth-aachen.de:/stvogel/my-new-course.git
cd my-new-course

4. Prepare Jupyter Notebooks

A new profile can include a set of Jupyter notebooks which come along with it. These Notebooks are will be synchronized from a Git repository every time a user enters the Jupyter environment.

As such, the collection of Notebooks for a course can be expandend during the course of the semester.

To start, please launch Jupyter on your local machine:

jupyter lab

A new browser window will open and present you the Jupyter web interface and you should already see the list of existing files in the current Git repository. Add new Notebooks and fill them with content to your wishes.

5. Collect Python requirements

Depending on the contents of the Jupyter Notebook, additional third-party Python packages might be required. Usually these external dependencies are collected in a requirements.txt or environment.yml file:

Please have a look at the following links for further information:

6. Adapt Dockerfile

Some profiles (e.g. when using special Jupyter Kernels) may require additional modifications to the runtime environment beyond the installation of new Python requirements.

To accomodate these the user can modify the included Dockerfile to run arbitrary commands during the preperation of the environment. Note that these commands are only executed during the Docker build phase.

7. Publish Jupyter Notebooks and Dockerfile in Git repository

After completing steps 2.-4., the resulting changes need to be commited to the Git repo and published on a Git hoster (e.g. RWTH GitLab or GitHub):

git add .
git commit -m "first version of new profile"
git push

8. Open request for creation of new RWTHjupyter profile

Please open a new request for the inclusion of your profile into the RWTHjupyter cluster using the following link: Create new profile

The link above will open a form for submitting a profile request via the RWTH GitLab system. Please follow the provided template and give us a few days to review your request before we make it available in the cluster.

RWTHjupyter User Documentation