Tags: floss, python, programming
One of the key tenants of open source development is to scratch your own itch, that is, to build something of use and value to their authors (compare that with commercial development --building something for a customer-- or research --new solutions to challenging problems--). However, for a project to survive and to attract that elusive second contributor, it is important to make the project useful to others. Which bring us to the problem of having some sort of user interface. Being a back-end person, I usually struggle creating them. Some of my projects (for example, see http://poissonmagique.net) remain in a state of "working back-end good enough for me". This is of course useful enough for me but a far cry from useful to anybody else. I have found recently some relatively new technology and workflow that might enable to get out of this local equilibrium: widgets for Jupyter Notebook.
As technology progresses, there is always the question "should I spend the time to learn this new technology or should I wait to see if it is just a fad?". ipython notebooks has been a project that has enjoyed a massive popularity in my environment and it has taken time for me to embrace it. First it was the question of whether it was coming here to stay, which with The Big Split™ it is clear it came here to stay and it is embracing its language agnostic roots much clearly. Second, a problem with having been in technology for a while it is that it is easy to miss out things that are radically different by simply mapping them to things that you know already. In the case of Notebooks, I usually wrote them off as a fancy terminal for an interactive session. I already do a lot of work in the console and write session log files in ASCII. I saw no need to have some fancy HTML output, but after studying the project better I can see I was just seeing the project through the selective vision of my existing knowledge and wasn't able to appreciate it in its totality.
So, what it is? Of course this is much better explained in their documentation but my take is that it is a captive interpreter session being accessed through messages queues from variety of front-ends. Commands can be sent to these interpreter sessions (called "kernels") and their output is sent back to the front-ends. A decoupled back-end/front-end for REPLs. And the front-end of choice is of course the Web. The sessions can be written to disk and re-executed at a later time. Also shared with others. The whole framework is language agnostic (all its needed is for the back-end REPL speaks a well-documented wire protocol) and there are "kernels" for python, Perl, Scala, Haskell (to name a few languages I'm interested on).
This is of course a boon for scientific work and data science. In this post I want to explore how its custom widgets functionality made it possible to quick-prototype easily and efficiently (in terms of coding, not execution) Use Cases for a functional back-end. Moreover, these Use Cases can then be shared with others that might be interested in developing production code for them!
Compared to a regular REPL, the key difference that is easy
to miss on a cursory examination of Jupyter is its ability to
return complex data types in the form of Backbone.js widgets. Through
the ipywidgets
project it is possible to return from the REPL input rich HTML
widgets that are live, meaning that they
generate messages processed by the running kernel as you interact
with them. The use of the widgets for building
interactive dashboards is what makes Jupyter such a boon
for Data Science. A less obvious possibility explored in this post
is to use them to quick-prototype a Use Case. You start by loading
the back-end with the necessary data (by connecting to a DB or
loading a data file, for example). Do you need to choose an
element from a list? Use
a widgets.Select. Need to initiate some
process? Use a widgets.Button. Different events
can be bound to the widgets and they are executed by callbacks all
in python code. This of course only useful for quick-prototyping,
I am not advocating building a whole site this way, but each of
the notebooks can then be shared with other people. And in the mid
time for local use they actually provide a working UI to interact
with the system.
The ipywidgets tutorial (you need Jupyter Notebook installed to render the link) is all you need to get you started. Here is a small example I wrote of how to use them to make local file selection widget, also embedded here:
To conclude, I would like to say that interacting with a computer this way, by programmatically building widgets and modifying data in the back-end incrementally to accomplish my own user needs feels like a completely different paradigm of computer interaction. I have read about Smalltalk and it feels like that. There are no programs, just interaction with the computer to achieve a goal. I think I have gone a long way from my original dimissing of Jupyter as "just the same as the interpreter but on the Web" to this new realization. Give it a try, you might be surprised as well!