Blaze: A lightweight literate programing preprocessor

September 12, 2017


 I’ve been playing with many literate programming tools since this technique of document-first programming came into my life two years ago.

Literate programming (LP), a concept that has been around since at least the 80s, is back in the spotlight since the Eve language (released by the Eve team headed by Chris Granger of Light Table fame) was released to the public in 2015.

While LP’s original concept aimed to create a document-first system whereby the building blocks of the actual code could be read (by the human or compiler) in any order (as it is in Eve), the contemporary state of the art is focused on creating the document-first model with existing procedural languages.

There are two main kinds of tool in the modern LP category:

  1. 1) Tools that write beautiful documentation based on comments in source code, such as Docco and (a personal favourite) Marginalia
  2. 2) Tools that allow you to execute code-fenced source code within some markup, such as Literate, litpro, and tiny lit.

The first is an evolution of the documentation processors you will be familiar with, no real innovation has happened in this space since Javadoc. (Much though I love Sphynx et al)

The second category I think is worth exploring.

It is into this ecosystem I present Blaze.

Blaze is Tiny:

#!/usr/bin/env sh
args=$1
script=$2

file_extension=$(echo $script |awk -F . '{if (NF>1) {print $NF}}')

if [ "$file_extension" = "md" ]
then
    cat $script | awk '{ if (/^```/) { i++; next } if ( i % 2 == 1) { print } }' > $script.out
    $args $script.out
    exit_code=$?
    rm $script.out
else
    $args $script
    exit_code=$?
fi
exit $exit_code

But what it gives you is the ability to execute your markdown files as though they were scripts: It is a drop-in replacement for /usr/bin/env:

#!blaze python
print("hi")

It then allows as many paramaters to be passed to your interpreter as you like (unlike normal shebangs), which means you can use tools like pexNote that we are able to use pex’s ephemeral venv trick to run python with any requirements pre-installed :

#!blaze pex arrow --
import arrow
print("run", arrow.now().humanize())  # blaze only processes .md files, plain scripts can be run as-normal

Blaze’s REAL trick, is that if it is called with a .md file, it only executes code inside triple-backtick codefences, as in this all-encompasing example of a literate program with built-in requirements:

#!blaze pex flask flask_restful --

# Imports
First the imports, this demo requires the `flask_restful` package.
Then we set up the Flask wsgi application object, `app` and the api wrapper, `api`.

``python
from flask import Flask
from flask_restful import Resource, Api

app = Flask(__name__)
api = Api(app)
``

# Flask Restful resources
We define a single `HelloWorld` resource, that responds with a simple json
object on a `GET` request.

``python
class HelloWorld(Resource):
    def get(self):
        return {'hello': 'world'}
``

# Routing
`api.add_resource()` wires the `Resource` class `HelloWorld` into the flask
router at `/`.

``
api.add_resource(HelloWorld, '/')
``

# Run Server
After we have created everything, we run the flask werkzeug server.

``
if __name__ == '__main__':
    app.run()
``

In the previous codeblock, double backticks should be triple, but that messes with markdown highlighting - sorry!

Magic, right?

Overhead

Blaze introduces minimal startup overhead, somewhere between 5-20ms, an almost zero runtime overhead (sh is running, I suppose).

Python Shebang

λ ./py-test.py
hi
./py-test.py  0.02s user 0.00s system 94% cpu 0.025 total

Blaze Shebang

λ ./blaze-test.py
hi
./blaze-test.py  0.02s user 0.00s system 67% cpu 0.030 total

Prior Art / Acknowledgements

Blaze is currently a hacked-together LP tool that is only suitable for one-off scripts. I borrowed the awk code-fence-stripping code from @trauber.

And the name?

Blaze is lit, bro 😎

Blaze: A lightweight literate programing preprocessor - September 12, 2017 - Tristram Oaten