Jupyter notebooks for remote science
Computational notebooks have massively gained in popularity in the past years. They allow to share rich documents with text, code, mathematical formulas, images, videos etc. both in static and interactive ways. They are heavily used both in research to document projects and in teaching to provide content support. Notebooks exist in different flavors, but currently the most popular form is the Jupyter (Julia, Python, R) notebook. Here, we briefly describe 1. What type of content they can be used for, 2. Solutions to write/run notebooks online (no installation required), 3. How to share the documents, interactively or not and 4. A few examples of courses given as notebooks.
1. What is a notebook
Notebooks are interactive documents with rich content that run directly in the internet browser. They are composed of series of blocks, called cells that can be executed individually. Notebooks are typically used when computer coding is involved but they provide a rich environment to create interactive documentation in general. So even if you don't need the coding part, you can still use notebooks to create documents and use the many existing resources to share them.
Notebooks are composed of mainly two types of cells: text cells and code cells.
Text cells
Markdown
The content of text cells is formatted using the Markdown language, a very simple markup language allowing to create formatted text. For example italic and bold are generated by surrounding text with one *italic*
or two stars **bold**
. Titles and subtitles are generated by preceding text by a series of hash signs # My Title
, ## My subtitle
.
Images
Images can be embedded using either a simple markdown syntax ![](myimage)
or directly html language which can be customized (e.g. for image size) <img src="myiamge" style="width: 100px;"/>
where myimage
is a path or a weblink to an image (e.g. https://img.myswitzerland.com/671846/407):
Equations
Fortunately, Jupyter also allows to embed $\LaTeX$ text. There a multiple solutions for this: surround your equation $\vec F = m*\vec{a}$ with dollar signs $\vec F = m*\vec{a}$
, use double dollar $$\vec F = m*\vec{a}$$
to create a block:
$$\vec F = m*\vec{a}$$
or in a separate cell use the standard \begin{equation} ... \end{equation}
:
def my_fun(x):
y = x**2 + 3*x + 2
return y
my_fun(5)
Not only numerical calculations are possible but also symbolic ones, e.g. using sympy:
import sympy
import warnings
warnings.filterwarnings("ignore")
from ipywidgets import interact, FloatSlider
import numpy as np
import matplotlib.pyplot as plt
t, k, c, y_0, ydot_0 = sympy.symbols('t k c y_0 ydot_0')
y = sympy.Function('y')
damped_oscillator = sympy.Eq(y(t).diff(t, t) + c* y(t).diff(t) + k*y(t), 0)
fun_solved = sympy.dsolve(damped_oscillator, y(t), ics={y(0):y_0, y(t).diff(t,1).subs(t, 0): ydot_0})
damped_oscillator
fun_solved
plt.plot(np.arange(-10,11),my_fun(np.arange(-10,11)));
Results can also be shown as interactive plots which the user can play with to get an intuition:
def myfun(var):
solved = fun_solved.subs({y_0:1, ydot_0:0, k:var, c : 1})
sympy.plot(solved.rhs,ylim = (-1,1),xlim = (0,10));
interact(myfun, var = FloatSlider(min=1, max = 10,step = .5));
from IPython.display import YouTubeVideo
YouTubeVideo("6FZ5k9_6Vlw",560,rel=0)
2. Using Jupyter notebooks
Of course Jupyter can be installed on any computer, the simplest route being to use Anaconda or Miniconda as a base for installation. Even though this works flawlessly in a large majority of cases, it is often desirable to avoid any installation procedure, especially in the current period where in-person meetings should be avoided. As Jupyter notebooks have become very popular and as they are entirely rendered in the browser, there are many solutions to run the notebook on a remote computer and simply display it in the browser. The different solutions vary in the resources available, price etc.
Renku from the Swiss Data Science Center
The SDSC provides for all academic institutions free access to a Jupyter environment on a platform called Renku. It allows to create entire projects including code and data and provides a full fledged access to a Jupyter environment entirely identical to one running on a laptop. One can login with a switch.edu account (your regular @xxx.unibe.ch account) or a Github account and create both private and public projects which are in fact repositories on Gitlab. This service is free.
Google colab
Google has developed it's own flavor of Jupyter called Colab. It works exactly in the same way as a regular notebook, only the interface looks slightly different. Here notebooks are saved on the GoogleDrive of the account you are using to connect to Colab. One of the main advantages of this solution is the access to GPUs. This service is free and you can purchase a subscription to enjoy longer sessions and more computing power.
Mybinder
For those already familiar with Python, Github and Jupyter, an interesting solution is the mybinder service. This allows you to turn any repository on e.g. on GitHub into an interactive Jupyter session. For that, you only have to include in your repository a file specifying the necessary packages to run your code. One negative side is that sessions are temporary, and any changes are lost on session end. This service is free.
Private JupyterHub
If you need e.g. a highly customized Jupyter environment or the possibility to share large amounts of data you might want to create your own Jupyter service called JupyterHub. Users of such a service are simply provided a link and again connect to the service via their browser, getting access to a standard Jupyter session. ScITS has experience in setting up such services for example using SWITCHengines computing resources.
3. Sharing documents
File exchange
Jupyter notebooks are simple text files. So you can simply share them as any other file type. For example you can upload all of them on ilias. This means however that to open them, people have to locally install Jupyter. Again in the perspective of avoiding local installations, it is easier to give access to them on some type of server.
Code repositories and notebook renderer
The most common solution for this is to use a code repository such as GitHub or Gitlab. If you are worried about publicly releasing your work, you can leave repositories private and share them with your students. This however requires all of them to have e.g. a GitHub account. Notebooks can be visualized as static objects by browsing through the repository. Note that even though notebooks are rendered as static objects, the links and videos will still work. A more efficient way of displaying notebooks is however to use the nbviewer service, which efficiently renders any notebook. Notebooks created on Renku (see above) can be visualized on Gitlab directly.
Interactive sharing
As mentioned above, if you want to go beyond rendering static notebooks, you can use mybinder or Google Colab to provide a quick access to interactive versions of your notebooks that collaborators and students can test and modify. You can embed directly in the notebooks badges to further simplify this procedure. Here for Colab: and here for mybinder:
Create a course website with GitHub pages
Imagine now that you have a series of notebooks for a course and you would like to assemble them into a simple website. If you are not familiar with web technologies, it can be a daunting task to create such a simple website. GitHub pages is a service integrated in GitHub which takes a lot of the pain away from that process i.e. no database, no domain to setup, no HTML to write etc. However it still involves setting up the appropriate environment to create content. This is where Fastpages a solution built on top of GitHub pages comes to the rescue: it completely automates the process of creating and publishing the web page. The only requirement is to be minimally familiar with GitHub (forks, pull requests etc.). Faspages offers a very detailed step by step guide on how to proceed and turns your collection of notebooks into a blog. See for example this notebook on my blog.
4. Examples of courses
We finish here by giving a few examples of courses of different nature.
Computational courses
At ScITS we heavily use notebooks to teach programming in various areas, e.g. image processing. For that course, a Github repository contains all the course material. One can visualize it directly in the repository or using nbviewer. It can even be run interactively using the mybinder service.
Scientific courses
Virtually all disciplines can benefit from being presented at least partially in interactive forms. To see for example how notebooks are used to teach chemistry, you can have a look at this specific notebook from this course. It contains videos, text, equations, symbolic calculations and is directly executable on Google Colab.
Here's another great example of a course on mechanics and numerical approximations. Again it uses all the resources available in notebooks. Note that if you visit the Github page, you will see this sort of badge telling you, you can run the material in a mybinder session.