IPython Notebook on Windows Azure

Whether you're absolutely new to Python and want to learn it in a fun, interactive environment or do some serious parallel/technical computing, the IPython Notebook is a great choice. As an illustration of its capabilities, the following screenshot shows the IPython Notebook being used, in combination with the SciPy and matplotlib packages, to analyze the structure of a sound recording:

This document will show you how to deploy the IPython Notebook on Windows Azure, using Linux or Windows virtual machines (VMs). By using the IPython Notebook on Windows Azure, you can easily provide a web-accessible interface to scalable computational resources with all the power of Python and its many libraries. Since all installation is done in the cloud, users can access these resources without the need for any local configuration beyond a modern web browser.

Create and Configure a VM on Windows Azure

The first step is to create a Virtual Machine (VM) running on Windows Azure. This VM is a complete operating system in the cloud and will be used to run the IPython Notebook. Windows Azure is capable of running both Linux and Windows virtual machines, and we will cover the setup of IPython on both types of virtual machines.

Linux VM

Follow the instructions given here to create a virtual machine of the OpenSUSE distribution.

Windows VM

Follow the instructions given here to create a virtual machine of the Windows Server 2008 R2 distribution.

Create an Endpoint for the IPython Notebook

This step applies to both the Linux and Windows VM. Later on we will configure IPython to run its notebook server on port 9999. To make this port publicly available, we must create an endpoint in the Windows Azure Management Portal. This endpoint opens up a port in the Windows Azure firewall and maps the public port (HTTPS, 443) to the private port on the VM (9999).

To create an endpoint, go to the VM dashboard, click "Endpoints", then "Add Endpoint" and create a new endpoint (called ipython_nb in this example). Pick TCP for the protocol, 443 for the public port and 9999 for the private port:

After this step, the "Endpoints" Dashboard tab will look like this:

Install Required Software on the VM

To run the IPython Notebook on our VM, we must first install IPython and its dependencies.

Linux

To install IPython and its dependencies, SSH into the Linux VM and carry out the following steps.

1. Install NumPy, Matplotlib, Tornado and other IPython's dependencies by doing:

   sudo zypper install python-matplotlib
   sudo zypper install python-tornado
   sudo zypper install ipython

2. Download and install the latest version of IPython by doing:

   sudo easy_install http://github.com/ipython/ipython/tarball/master

Windows

To install IPython and its dependencies on the Windows VM, Remote Desktop to connect to the VM. Then carry out the following steps, using the Windows PowerShell to run all command line actions.

1. Install Python 2.7.2 (32 bit) from http://python.org. You will also need to add C:\Python27 and C:\Python27\Scripts to your PATH environment variable.

2. Install distribute by downloading the file http://python-distribute.org/distribute_setup.py and then running the command:

   python distribute_setup.py

3. Install Tornado and PyZMQ by running the commands:

   easy_install tornado
   easy_install pyzmq

4. Download and install NumPy and Matplotlib using the .exe binary installers available on their respective web sites.

5. Download and install OpenSSL. You will need to install both the "Win32 OpenSSL v1.0.1c Light" and "Visual C++ 2008 Redistributable" from http://slproweb.com/products/Win32OpenSSL.html. You will also need to add C:\OpenSSL-Win32\bin to your PATH environment variable.

6. Download and install the latest version of IPython by doing:

   easy_install http://github.com/ipython/ipython/tarball/master

Configure the IPython Notebook

Next, we configure the IPython Notebook. The first step is to create a custom IPython configuration profile to encapsulate the configuration information:

ipython profile create nbserver

Next we cd to the profile directory to create our SSL certificate and edit the profiles configuration file.

On Linux:

cd ~/.config/ipython/profile_nbserver/

On Windows:

cd ~\.ipython\profile_nbserver

On both platforms create the SSL certificate as follows:

openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem

Note that since we are creating a self-signed SSL certificate, when connecting to the notebook your browser will give you a security warning. For long-term production use, you will want to use a properly signed certificate associated with your organization. Since certificate management is beyond the scope of this demo, we will stick to a self-signed certificate for now.

In addition to using a certificate, you must also provide a password to protect your notebook from unauthorized use. For security reasons IPython uses encrypted passwords in its configuration file, so you'll need to encrypt your password first. IPython provides a utility to do so; at a command prompt run:

python -c "import IPython;print IPython.lib.passwd()"

This will prompt you for a password and confirmation, and will then print the password as follows:

Enter password: 
Verify password: 
sha1:b86e933199ad:a02e9592e59723da722.. (elided the rest for security)

Next, we will edit the profile's configuration file, which is the ipython_notebook_config.py file in the profile directory you are in. This file has a number of fields and by default all are commented out. You can open this file with any text editor of your liking, and you should ensure that it has at least the following content:

c = get_config()

 # This starts plotting support always with matplotlib
 c.IPKernelApp.pylab = 'inline'

 # You must give the path to the certificate file.

 # If using a Linux VM:
 c.NotebookApp.certfile = u'/home/ipadmin/.config/ipython/profile_nbserver/mycert.pem'

 # And if using a Windows VM:
 c.NotebookApp.certfile = r'C:\Users\Administrator\.ipython\profile_nbserver\mycert.pem'

 # Create your own password as indicated above
 c.NotebookApp.password = u'sha1:b86e933199ad:a02e9592e5 etc... '

 # Network and browser details. We use a fixed port (9999) so it matches
 # our Windows Azure setup, where we've allowed traffic on that port

 c.NotebookApp.ip = '*'
 c.NotebookApp.port = 9999
 c.NotebookApp.open_browser = False

Run the IPython Notebook

At this point we are ready to start the IPython Notebook. To do this, navigate to the directory you want to store notebooks in and start the IPython Notebook server:

ipython notebook --profile=nbserver

You should now be able to access your IPython Notebook at the address https://[Your Chosen Name Here].cloudapp.net.

When you first access your notebook, the login page asks for your password:

And once you log in, you will see the "IPython Notebook Dashboard", which is the control center for all notebook operations. From this page you can create new notebooks, open existing ones, etc:

If you click on the "New Notebook" button, you will see an opening page as follows:

The area marked with an In []: prompt is the input area, and here you can type any valid Python code and it will execute when you hit Shift-Enter or click on the "Play" icon (the right-pointing triangle in the toolbar).

Sinc we have configured the notebook to start with NumPy and matplotlib support automatically, you can even produce figures, for example:

A powerful paradigm: live computational documents with rich media

The notebook itself should feel very natural to anyone who has used Python and a word processor, because it is in some ways a mix of both: you can execute blocks of Python code, but you can also keep notes and other text by changing the style of a cell from "Code" to "Markdown" using the drop-down menu in the toolbar:

But this is much more than a word processor, as the IPython notebook allows the mixing of computation and rich media (text, graphics, video and virtually anything a modern web browser can display). For example, you can mix explanatory videos with computation for educational purposes:

or embed external web sites that remain live and usable, inside of a notebook file:

And with the power of Python's many excellent libraries for scientific and technical computing, a simple calculation can be performed with the same ease than a complex network analysis, all in one environment:

This paradigm of mixing the power of the modern web with live computation offers many possibilities, and is ideally suited for the cloud; the Notebook can be used:

• as a computational scratchpad to record exploratory work on a problem,

• to share results with colleagues, either in 'live' computational form or in hardcopy formats (HTML, PDF),

• to distribute and present live teaching materials that involve computation, so students can immediately experiment with the real code, modify it and re-execute it interactively,

• to provide "executable papers" that present the results of research in a way that can be immediately reproduced, validated and extended by others,

• as a platform for collaborative computing: multiple users can log into the same notebook server to share a live computational session,

• and more...

If you go to the IPython source code repository, you will find an entire directory with notebook examples which you can download and then experiment with on your own Windows Azure IPython VM. Simply download the .ipynb files from the site and upload them onto the dashboard of your notebook Windows Azure VM (or download them directly into the VM).

Conclusion

The IPython Notebook provides a powerful interface for accessing interactively the power of the Python ecosystem on Windows Azure. It covers a wide range of usage cases including simple exploration and learning Python, data analysis and visualization, simulation and parallel computing. The resulting Notebook documents contain a complete record of the computations that are performed and can be shared with other IPython users. The IPython Notebook can be used as a local application, but it is ideally suited for cloud deployments on Windows Azure

The core features of IPython are also available inside Visual Studio via the Python Tools for Visual Studio (PTVS). PTVS is a free and open-source plug-in from Microsoft that turns Visual Studio into an advanced Python development environment that includes an advanced editor with IntelliSense, debugging, profiling and parallel computing integration.