Proposal: document preparation improvements

(See also general notes on proposals.)

Isabelle/Isar is ultimately about beatiful proof documents, not profane “proof scripts”. This explains why high-quality rendering of theory files in PDF-LaTeX has been part of the game from early on. In the past 15 years, the Isabelle document preparation system has been applied to produce numerous articles, books, theses etc. based on content that is formally checked in the logic (usually Isabelle/HOL). Here is an arbitrary example from AFP.
The Isabelle manuals are usually produced as Isabelle documents as well, e.g. see the sources in the directory src/Doc of the Isabelle distribution.

Traditional Isabelle document preparation is a side-effect of a session build process that is run in batch-mode; see also the Isabelle System manual, chapters 2 and 3. The main command-line tools for session management with document preparation are isabelle mkroot and isabelle build. This edit-typeset-preview cycle works, but feels a bit awkward today, where interactive theory and proof development with continuous checking in the Prover IDE (PIDE) is used routinely. So it his high time for various renovations and reforms of Isabelle document preparation as sketched below.

Immediate build process of documents within the Prover IDE.
There is no particular reason why LaTeX should be run via batch-mode tools on the command-line. The Prover IDE could take care of that interactively, with much shorter turnaround cycles of the edit-typeset-preview cycle. To achieve that, parts of the document preparation in Isabelle/ML need to be moved to Isabelle/Scala, and slightly generalized to become stateless and applicable to partial/unfinished document sources.
GUI panel for document preparation in Isabelle/jEdit.
This should allow to control document preparation in the IDE, e.g. to specify which parts of the document are presently interesting, or to change options for LaTeX and add-on tools.
Improved error reporting from LaTeX

LaTeX error output should be parsed and presented within the Prover IDE adequately, with proper error positions over the original source text. The notoriously obscure LaTeX log files need to be overcome.
Light-weight document markup as in Markdown / Commonmark with approximative preview in the source.
Instead of old-fashioned LaTeX markup like \begin{itemize} \item ... \end{itemize} there could be more direct indication of itemization in the source (with actual bullets from the Isabelle symbol repertoire). Using conventions from Markdown, it would approximate a preview of the final typesetting already in the source text. Enumerations and description lists could be done similarly. Note that there is no need for sections (which are already supported via explicit Isar commands), nor for hyperlinks and other formal inserts (which are already supported via document antiquotations).
Improved HTML output
A subset of document markup that fits into the limited Markdown / Commonmark format sketched above could be taken as starting point for better HTML presentation, with relatively modest ambitions and technical requirements. It is important to recall that “HTML” as such is a huge and vaguely defined collection of standards: producing portable output that looks nice on most browsers requires significant expertise or special tools, or both.

A few improvements have already made it into the repository after Isabelle2014. For example, the repository version Isabelle/872f330a0f8a provides support for BibTeX files in Isabelle/jEdit. In Isabelle/9986fb541c87 there is also support for @{cite} antiquotations, with formal links etc.