Interactive Living Documents

Modern web software rarely lives up to its potential and allow users to do what they really want to do. Web users are locked into single purpose systems with little integration besides hyperlinks or primitive and inflexible embedding. Let’s discuss a number of applications and discuss the similarities.

  • Wikis
  • Content Management Systems
  • Web Forums
  • Online code editors
  • Customer Relationship Management
  • Document Management Systems
  • Issue Trackers
  • Workflow systems
  • CRUD applications
  • Feeds
  • Social networks
  • Image galleries

Wikis manage interlinked formatted pages and the history of these pages over time. Content Management Systems manage the composition of a documents such as media items and links. Web forums record ordered messages in a hierarchy.  Online code editors manage multiple files in different formats and provide edit facility. Issue trackers enforce state changes of an item over time. A real-time feed displays new items as they occur. A social network manages relationships between people and enables sharing of items. These systems essentially:

  1. implement a user interface to edit things
  2. store different objects each containing different fields: issues, status updates, gallery, photograph, user, post, thread, forum, menu, menu item
  3. keep relationships between different things
  4. display the data in a particular way

Taking a step back from the tools available, what might people use standard word processed documents for?

  1. A job advertisement written in a word processor has a set of fields such as qualifications and a responsible individual.
  2. A purchasing department study of offerings may have a repeating structure of offerings with fields for each. Each offering may go through a workflow perhaps involving separate evaluation.
  3. A user organising an activity may list tasks and requirements which each have a status and a cost.
  4. A user conducting research may wish to record and organise notes in a specific structure.

Users that create a document structure in a word processor are required to manually update documents or pick a system that at best approximates a solution. These examples share the following in common:

  1. They each collect multiple disparate pieces of information in a certain format.
  2. The same information can be displayed in one or more ways.
  3. There is a workflow for the information that is collected and displayed and how its state changes.
  4. Information collection takes place while the document is still being written.
  5. All of the above may be configured specifically for this page only.
  6. All of the above can be referred or included and used elsewhere.

Now imagine a system that has these attributes.

Each page is its own editor and viewer

An interactive living document takes the best features of CRUD applications, blogs, word processing, real-time chat, dashboards, forums and wikis and Apache Wave and puts them together. How you can interact with a page is determined by what is on it.

Divergent Writing

A divergent writing system acknowledges that when people write, they do not write or think linearly. A task they start with can lead to further separate tasks. A writer should not be constrained by the software that they use. They should not have to save what they are doing, create a new document, go back to the original document and include it by copy and pasting or overtly complex transclusion mechanism. They should be able to create a document within the context of another and decide how it should be displayed later.

A divergent writing interface can be visualised as a thin but tall graph where each node represents a split in the writing process. Divergences can be opened and closed and individually edited. A divergence may not necessarily be included in the page – the user interface that facilitates divergent writing does so because it becomes easier to start things while halfway through another process. This operates in the same way as outliner software.

Always collecting data

Living documents are meant to support multiple collaborators and allow multiple users to contribute to the construction of documents. A document is not only for consumption.

  • A user should be able to treat a document as an application. A user may like how the living document has been structured: they may wish to use the same document for their own reviews. They may wish to supply their own data for the generated tables and graphs.
  • A blogger reviewing products can accept product names for suggested reviews.
  • If the user opts to share the data with the living document author, the author can verify the information is valid and not spam and allow it to appear on the site in the same format as the other articles.
  • I am writing a post about different keyboard shortcut formats. I include some examples but want users to provide their own examples. They may wish to link to a man page, a Git Hub to refer to existing keyboard shortcut encodings. Alternatively they could just use a freestyle comment box.

Embed anything

A living document system should not restrict what you can embed. Many different ecosystems should be accessible from a living document as if they were first class citizens. For example:

  • Other rendering languages should be embeddable into the page. I should be able to embed whatever will be understood by the viewer. This includes more than just HTML, SVG, video and flash. I should be able to embed fragments of PostScript, Latex and any other language that generates graphics.
  • If the language is not natively renderable by the viewer, at the source should be viewable.

Natural Habitat

The natural habitat for data is where the data belongs and where it is updated. Living documents seek to use the natural habitat rather than copy data. Pasting data into a living document from another source without having a mechanism in place to synchronise the two items is not recommended.

  • Paste events are automatically marked as paste contexts by living documents. This is so that pasted data can be tracked and updated. A paste event may even trigger a search of existed pasted data to see if it has already been pasted, in which case, the two will be connected.
  • A user who includes a graphic in a document is able to change the graphic externally and have it automatically appear in the living document whenever it is updated.
  • I am comparing web libraries and want to provide small code samples that demonstrate the performance of each. I could write the code or paste it directly into the blog but this disconnects the information from its Natural Habitat. Instead there should be some kind of linkage between the code in source control and its displaying on the article.

Example: A Reviewer

I’m writing a post about multiple products in the same market segment and want to write an editorial piece for each. I evaluate products on multiple criteria where each gets a numeric rating. I want to record attributes about the product such as the vendor, its price at the time of review and attach a number of photographs. I open the interactive document system and select Add. I type ‘Laptop Review 2013‘. This creates a fresh document in the context of this name. I write a basic introduction. I can now continue in one of two ways:

  • I figure that my reviews should be a template. I select Add and type a particular product’s name. I figure that this review should use a template, so I click Add again and type Review. I want a product header to appear at the top so I select Add and type ‘Product Name Heading‘. I do the same for every other field and jump back to the LaptopMaker Model 500 context. Here I see that the document below wants data including a Product Name Heading. I either type the data and select it and select the available field or select Add and type the field.
  • I can begin writing my first review. I write the product’s name as a heading, an introduction to the product and my ratings for each criteria. I attach a number of images of the product. Now I have a complete review and it contains all the data for a single product but I have not indicated what is what. I highlight my finished review for a single product, select Template and type ‘Review‘ and ‘LaptopMaker Model 500′. This creates two new contexts: a template context called ‘Review‘ and a document context for the product called ‘LaptopMaker Model 500′ that makes a reference to the Review context. This newly created Review document is not that useful right now because it is always contains a review about a specific product and its contents never changes, each of the fields I used need to be extracted so I can reuse the template for other reviews. I highlight the product name heading and select Extract and type ‘Product Name Heading‘. This will replace the existing heading with empty slot that wants to accept a ‘Product Name Heading‘ and move the actual product heading to the product context. We can now create further reviews selecting Create and Review.

Now that I have my completed reviews, I can decide how they should be displayed in different ways. I want my reviews to be visible on a single page but with a summary at the end. Since my document is a living document, I can create a summary view by referring to elements of the individual reviews. I create graphs and comparison tables using the data from my individual reviews. Since this data shares the same origin, when I update any data relating to a review it will automatically change everywhere, even across contexts.

A living document interface

The components of a living document interface and its behaviours are as follows:

  • Editor: The editable region functions as a normal WYIWYG interface.
  • Semantic Autocomplete Allows the user to define and include things into the page.
  • Relationship Graph The current connections between the pieces of information available

While you’re writing in your living document, you have primitives beyond the standard words and lines. Instead you create arbitrary relationships between things, include them on the page and decide how they should be rendered later. A living document interface works on the principle of describing what you want and building it up over time. So rather than inserting an image like you would in traditional word processor, markup or WYSIWYG editor you instead specify you intent behind what the image is and later apply an image to that. This sounds more complicated than it is. The way an intent might be supplied is via semantic autocompletion, a tool similar in functionality to the following:

  • Intellisense in Visual Studio
  • Command palette in Sublime
  • Navigate to Action in IntelliJ
  • Insert Macro in Confluence

The contents of semantic autocompletion is not completely pre-determined by the developer of the living document software. Instead this widget contains various arbitrarily determined pieces of data and operations that you have used or created before. These commands may be similar to:

You might use a semantic living document for any of the following:

  • Writing about code? You could have code extracts, code output, log outputs.
  • Mocking stuff up in code? You could include a live REPLs, a test viewer and editor.

Now take the primitives above and imagine you can:

  • Use semantic  line to create any of the above things such as ‘create review’, ‘include program + output’, ‘include REPL’
  • Explore or search any of the above arbitrarily and see where it is used in context
  • Transclusion comes for free, you can refer to a portion of any document and have it update everywhere whenever you change it
  • Everything is versioned and reversible.

Interface Defined at Authorship

A document should not be forgotten once posted. The document should adapt and grow over time. An article and its components have states that can change over time. An article is not just in view or edit mode. For example, multiple people can be responsible for an document, each handling aspects such as functionality, data, design and copywriting. Users can leave behind contexts for other users to handle..

  • Any user may wish to indicate that some behaviour or logic should take place so that when a developer opens the document, they are required to implement it.
  • A content writer may indicate than an image should be displayed at a certain point. What image should be displayed is up to the designer.

Creating an article should also involve creating a workflow to determine the order that things happen and various state transitions.

Zoomable Interfaces

The interface can be zoomed out and detail removed to show the divergent tree in context with everything else. A timeline view could flip this graph 90 degrees to imitate a timeline.

The Friend Pipe

When you want to send a file to someone, you have multiple choices. You can send emails, you can use file sharing services. You have to make the decision each time. We can use the friend pipe. The friend pipe:
filename | sam

This pipes the file filename to the friend sam. At this point we do not care how the file should be transferred. Only that it is transferred in the quickest way at this point. We can broadcast status updates with the fp (friendpipe) command.

friends "Anyone free for lunch? 1.30"

To use the friendpipe, we must learn about another user in order to know how to transfer data to them. The friend manager allows us to create relationships with other people. There are many approaches to adding users:

friend add
friend add git://
friend add
friend add ssh://user@
friend add xmpp://

This will pull information about how to connect to the user.


  • friendpipe should encrypt content. This can be accomplished exchanging public keys as part of the bootstraping process.
  • The basic unit of an interaction is a file transfer. Further abstractions such as social networking can be built on-top.
  • Plugins are used to handle the actual transfer between users. (For example: email, file services like Dropbox, SSH, FTP etc)
  • Content negotiation picks the best transfer mechanism between two users.
  • Broadcasting files to multiple users requires more intelligent picking of algorithms.