A paean on structured editing

My friends may have picked up that I am big on this. There are some abstract ideas I have come up with to think about the ideas, like uniformly managing the variable levels of abstraction of the data we work with, but I hope to use a concrete and pleasant worked example to show what I am thinking of.

Editing: one of the most basic ways of generating data on a computer. Indeed, barring very graphics-intensive workflows, editing seems to capture all the work we do with generated rather than captured or observed data, so it makes sense to get this right. I think a lot of the ways people are going at the moment are against what I feel is best practice, so I may as well get some thoughts out.

Consider the humble text editor. You could type, and control the exact format and layout with retro ASCII art, and it was all very nice. That is, unless you wanted italics, or pretty underlining, or any of the other things which are natural in printed language.

The next step was clear: insert some control commands you could type, add a formatter, and you get LaTeX, which is beautiful to output but horrible to input. Should you really have to type \section{ when you want to add a new section? It seems wrong. So, you write some key bindings so you only have to enter CTL-J for a new section. Your editor still displays the clunky LaTeX markup to you though, which seems silly. I typed CTL-J, so why should my editor display \section{ to me before the heading?

We are drawing out the key concern here: what the computer stores is perhaps different from what the computer works with, which is different from what we want to interact with. Some data we deal with in a highly abstracted way, like vector graphics, where the primatives are miles beneath the stars shapes we draw and rotations we drag. Some data is pretty much as we work with it, like simple text. The limitation of the LaTeX syntax is that it is totally useless at using context to determine with how much abstraction to the present the data to us. A visual editor only gets you so far, as you find out when you make lovely graphs using TikZ or PSTricks and drop back down to the low-level interface.

The purpose of structured editing then is to present to the user editing capabilities for each document element at the user’s level of abstraction, and as a corollary to handle the implications of what storage format is to be used internally.

One area where progress is not happening is on the web, in in-browser interfaces. The regression to Textile, Markdown, and other text-based formats is sad. Mozilla, my favourite foundation, has in the past been involved with a lot of research and experimentation forwards in this sort of area. Daniel Glazman worked through a lot of concepts in editing, announcing, releasing, and then obsoleting some of his ideas here. There is a lot to think through when working with highly-structured documents and users of different abilities. His NVu editor, now dead and replaced by BlueGriffon, was an HTML-centric attempt in this direction. A lot of other dead ends in this direction, like Verbosio (though resurrected, and interesting work still going on), show the difficulties of working this out.

I am not as especially attached to XML as you might think, though I know and love the tools and workflow. Word in fact is a substantial structured editing project in some ways. Though traditionally it was terrible, mixing highly abstract styling commands with quick little low-level commands to make text italic and violating all the principles of separation of abstraction, I gather that recent versions can enforce some style constraints and give more of the rigid idea which captures information at the user’s level of thought.

My ideal at the moment is XXE. The backend storage is DocBook5, a mature and well-designed extensible solution that I am comfortable working with and supporting in the future. It can take images, equations, SVG, and more. Whatever interface of the future we work with to edit will not be far in broad conceptual terms from what I have in front of me, at least for the case of technical composite documents where simpler flat interfaces with a uniform level of abstaction will not do.

As a final aside, a WP link sends me on further trains of thought…