Category Archives: PIDE

Visual Studio Code as Prover IDE for Isabelle

Microsoft is more and more becoming an Open Source company, e.g. it has joined the Linux foundation as Platinum member in November 2016. A notable Open Source project by Microsoft is Visual Studio Code: under the slogans “Code editing. Redefined. Free. Open Source. Runs everywhere.” it provides a very interesting editor framework, as a desktop application based on Node.js and TypeScript.

The prover community has already started to support this emerging successor of vi and Emacs, e.g. see Coq Support for Visual Studio Code and the Lean for VSCode.

Isabelle/VSCode is now following this trend: I have spent some weeks in December 2016 / January 2017 with VSCode, using a little bit of TypeScript and implementing the new Language Server Protocol in Isabelle/Scala. The subsequent screenshot shows formal annotations produced by Isabelle/PIDE in the usual manner, while the editor rendering is all done by VSCode. For more information, see the report Isabelle/VSCode in January 2017.

The Isabelle/VSCode project was funded by Aesthetic Integration (AI). See also my article about a talk at ITP 2016 by Grant Passmore (co-founder of AI).

Flattr this!

Isabelle/PIDE as IDE for ML

On Friday 18-Nov-2016 10:00, I will give a presentation about PIDE at Laboratoire de Recherche en Informatique, Orsay (Paris Sud).

Abstract:

Isabelle is usually positioned as environment for interactive and automated theorem proving, but its Prover IDE (PIDE) may be used for regular program development as well. Standard ML is particularly important here, since it is the bootstrap language of Isabelle/ML (i.e. SML with many add-ons) and Isabelle/Pure (i.e. the logical framework).

The ML IDE functionality of Isabelle + Poly/ML is manifold:

  • Continuous feedback from static analysis and semantic evaluation is already available for years, e.g. Isabelle2014 (August 2014). It is a corollary of how PIDE interaction works, and of the integration of the Poly/ML compiler into that framework. Source files are statically checked and semantically evaluated while the user is editing. The annotated sources contain markup about inferred types, references to defining positions of items etc.
  • Source-level debugging within the IDE is new in Poly/ML 5.6, which is bundled with Isabelle2016 (February 2016). The Prover IDE provides the Debugger dockable to connect to running ML threads, inspect the stack frame with local ML bindings, and evaluate ML expressions in a particular run-time context. See also here.
  • IDE support for the Isabelle/Pure bootstrap process is new technology for the coming release of Isabelle2016-1 (December 2016). The ROOT.ML file acts like a quasi-theory in the context of theory ML_Bootstrap: this allows continuous checking of all loaded ML files. The theory file is presented with a modified header to import Pure from the running Isabelle instance.
  • It is also possible to modify standalone SML projects, to edit the sources freely in the ML IDE. For example, MetiTarski can participate after some trivial changes of its ROOT.ML file.

Overall, we move more and more to an integrated framework for development of formal-reasoning tools, but other applications are admissible as well.

The Slides are available, together with their sources (which are required for the live system demo).

Flattr this!

Isabelle/PIDE — from Interactive Theorem Proving to Integrated Theorem Proving

On Tuesday 15-Nov-2016 14:00, I will give a presentation about PIDE at Laboratoire Spécification et Vérification, Cachan (Paris). See also the official announcement.

Abstract:

Interactive theorem proving was historically tied to the read-eval-print loop, with sequential and synchronous evaluation of prover commands given on the command-line. This user-interface technology was adequate when Robin Milner introduced his LCF proof assistant in the 1970s, but today it severely restricts the potential of multicore hardware and advanced IDE front-ends.

The Isabelle Prover IDE breaks this loop and retrofits the read-eval-print phases into an asynchronous model of document-oriented proof processing. Instead of feeding a sequence of commands into the prover process, the primary interface works via edits over immutable document versions. Execution is implicit and managed by the prover in a timeless and stateless manner, making adequate use of parallel hardware.

PIDE document content consists of the theory sources (with dependencies via theory imports), and auxiliary source files of arbitrary user-defined format: this allows to integrate other languages than Isabelle/Isar into the IDE. A notable application is the Isabelle/ML IDE, which can be also applied to the system itself, to support interactive bootstrapping of the Isabelle/Pure implementation.

Further tool integration works via “asynchronous print functions” that operate on already checked theory sources. Thus long-running or potentially non-terminating processes may provide spontaneous feedback while the user is editing. Applications range from traditional proof state output (which often consumes substantial run-time) to automated provers and dis-provers that report on existing proof document content (e.g. Sledgehammer, Nitpick, Quickcheck in Isabelle/HOL). It is also possible to integrate “query operations” via additional GUI panels with separate input and output (e.g. for manual Sledgehammer invocation or find-theorems).

Thus the Prover IDE orchestrates a suite of tools that help the user to write proofs. In particular, the classic distinction of ATP and ITP is overcome in this emerging paradigm of Integrated Theorem Proving.

The Slides are available.

Flattr this!

PIDE as Standard ML IDE for bootstrapping Isabelle

Isabelle is usually advertized as environment for interactive and automated theorem proving, but its Prover IDE (PIDE) may be used for regular program development as well. Standard ML is particularly important here, since it is the bootstrap language of Isabelle/ML (i.e. SML with many add-ons) and Isabelle/Pure (i.e. the logical framework).

Using Isabelle/PIDE for bootstrapping Isabelle itself is now possible in recent repository versions, e.g. Isabelle/1c1f8531ca37 – see also README_REPOSITORY for general explanations how to build and run that. Here is the relevant NEWS entry from that version:

IDE support for the Isabelle/Pure bootstrap process. The initial files src/Pure/ROOT0.ML or src/Pure/ROOT.ML may be opened with Isabelle/jEdit: they act like independent quasi-theories in the context of theory ML_Bootstrap. This allows continuous checking of ML files as usual, but results are isolated from the actual Isabelle/Pure that runs the IDE
itself.

The ML project consists of a sequence of ML_file commands in ROOT.ML. Projects other than Isabelle can do the same with SML_file for official Standard ML. Afterwards, the following Poly/ML command line is able to build the project without the IDE: poly --eval "val SML_file = PolyML.use" --use ROOT.ML

What is also notable in the Isabelle/Pure bootstrap environment is the structure Thread_Data for global state variables within the current thread. There are two implementations: (1) physical and (2) virtual. The virtual version is used when Isabelle/Pure is loaded into itself: it allows to manage many versions of the load process with different intermediate states in a value-oriented manner.

Flattr this!

ML debugging within the Prover IDE

Isabelle/ML is based on Poly/ML and thus benefits from the source-level debugger of that implementation of Standard ML. The Prover IDE provides the Debugger dockable to connect to running ML threads, inspect the stack frame with local ML bindings, and evaluate ML expressions in a particular run-time context. A typical debugger session is shown in the screenshot below.

[ML debugger]

ML debugging depends on the following pre-requisites.

  1. ML source needs to be compiled with debugging enabled. This may be controlled for particular chunks of ML sources using any of the subsequent facilities.

    1. The system option ML_debugger as implicit state of the Isabelle process. It may be changed in the menu Plugins / Plugin Options / Isabelle / General. ML modules need to be reloaded and recompiled to pick up that option as intended.
    2. The configuration option ML_debugger, with an attribute of the same name, to update a global or local context (e.g. with the declare command).
    3. Commands that modify ML_debugger state for individual files: ML_file_debug, ML_file_no_debug, SML_file_debug, SML_file_no_debug.

    The instrumentation of ML code for debugging causes minor run-time overhead. ML modules that implement critical system infrastructure may lead to deadlocks or other undefined behaviour, when put under debugger control!

  2. The Debugger panel needs to be active, otherwise the program ignores debugger instrumentation of the compiler and runs unmanaged. It is also possible to start debugging with the panel open, and later undock it, to let the program continue unhindered.

  3. The ML program needs to be stopped at a suitable breakpoint, which may be activated individually or globally as follows.

    For ML sources that have been compiled with debugger support, the IDE visualizes possible breakpoints in the text. A breakpoint may be toggled by pointing accurately with the mouse, with a right-click to activate jEdit’s context menu and its Toggle Breakpoint item. Alternatively, the Break checkbox in the Debugger panel may be enabled to stop ML threads always at the next possible breakpoint.

Note that the state of individual breakpoints gets lost when the coresponding ML source is re-compiled! This may happen unintentionally, e.g. when following hyperlinks into ML modules that have not been loaded into the IDE before.

The debugger panel (see screenshot) shows a list of all threads that are presently stopped. Each thread shows a stack of all function invocations that lead to the current breakpoint at the top.

It is possible to jump between stack positions freely, by clicking on this list. The current situation is displayed in the big output window, as a local ML environment with names and printed values.

ML expressions may be evaluated in the current context by entering snippets of source into the text fields labeled Context and ML, and pushing the Eval button. By default, the source is interpreted as Isabelle/ML with the usual support for antiquotations (like ML, ML_file). Alternatively, strict Standard ML may be enforced via the SML checkbox (like SML_file).

The context for Isabelle/ML is optional, it may evaluate to a value of type theory, Proof.context, Context.generic. Thus the given ML expression (with its antiquotations) may be subject to the intended dynamic run-time context, instead of the static compile-time context.

The buttons labeled Continue, Step, Step over, Step out recommence execution of the program, with different policies concerning nested function invocations. The debugger always moves the cursor within the ML source to the next breakpoint position, and offers new stack frames as before.

Flattr this!

Release of Coq 8.5

On 21-Jan-2016 Coq 8.5 has been released, with an unusually long distance of about 3.5 years to Coq 8.4.

Coq 8.5 is notable for having a little bit of PIDE (Prover IDE) support (by Enrico Tassi), without using that name nor the Scala-based PIDE library of Isabelle. It is all based on OCaml, and connects to the regular CoqIde frontend.

Flattr this!

HOL4 workshop at CADE-25

The HOL4 workshop will happen on Sunday 02-Aug-2015 and Monday 03-Aug-2015 in Berlin, as an associated event of the 25th International Conference on Automated Deduction (CADE-25). The main theme are future directions and visions on HOL4 development: both users and developers are invited to participate in the discussion.

Even though I am myself not an HOL4 person, I will give a presentation about Isabelle/PIDE/jEdit as integrated development environment for Standard ML. The abstract is as follows:

After more than 7 years of development, Isabelle/PIDE/jEdit is today the standard way to interact with that particular proof assistant. In Isabelle2015 (May 2015) the TTY-based REPL and its wrapper for Proof General / Emacs have already been dismantled. This radical move might be taken as an opportunity of the HOL4 community to attract former Isabelle users who really do want to use plain TTY interaction. Or as an opportunity to discuss possibilities for HOL4 users and developers to make their own moves towards full-scale IDE support.

As a very modest start, I would like to present various possibilities of Isabelle/PIDE to operate as IDE for Standard ML, which happens to be the underlying language platform of HOL4 as well. This touches various facilities of Poly/ML that David Matthews provides specifically to tool builders: run-time compiler invocation with IDE feedback, toplevel environment management, structured toplevel printing (with markup and hyperlinks), and potentially also run-time debugging of SML (still unused in Isabelle2015).

Beyond that it is also possible to integrate any other languages that are related or unrelated to the prover platform, using PIDE libraries either on the ML or Scala side of that IDE framework.

Flattr this!

Proposal: document preparation improvements

(See also general notes on proposals.)

Isabelle/Isar is ultimately about beatiful proof documents, not profane “proof scripts”. This explains why high-quality rendering of theory files in PDF-LaTeX has been part of the game from early on. In the past 15 years, the Isabelle document preparation system has been applied to produce numerous articles, books, theses etc. based on content that is formally checked in the logic (usually Isabelle/HOL). Here is an arbitrary example from AFP.
The Isabelle manuals are usually produced as Isabelle documents as well, e.g. see the sources in the directory src/Doc of the Isabelle distribution.

Traditional Isabelle document preparation is a side-effect of a session build process that is run in batch-mode; see also the Isabelle System manual, chapters 2 and 3. The main command-line tools for session management with document preparation are isabelle mkroot and isabelle build. This edit-typeset-preview cycle works, but feels a bit awkward today, where interactive theory and proof development with continuous checking in the Prover IDE (PIDE) is used routinely. So it his high time for various renovations and reforms of Isabelle document preparation as sketched below.

Immediate build process of documents within the Prover IDE.
There is no particular reason why LaTeX should be run via batch-mode tools on the command-line. The Prover IDE could take care of that interactively, with much shorter turnaround cycles of the edit-typeset-preview cycle. To achieve that, parts of the document preparation in Isabelle/ML need to be moved to Isabelle/Scala, and slightly generalized to become stateless and applicable to partial/unfinished document sources.
GUI panel for document preparation in Isabelle/jEdit.
This should allow to control document preparation in the IDE, e.g. to specify which parts of the document are presently interesting, or to change options for LaTeX and add-on tools.
Improved error reporting from LaTeX

LaTeX error output should be parsed and presented within the Prover IDE adequately, with proper error positions over the original source text. The notoriously obscure LaTeX log files need to be overcome.
Light-weight document markup as in Markdown / Commonmark with approximative preview in the source.
Instead of old-fashioned LaTeX markup like \begin{itemize} \item ... \end{itemize} there could be more direct indication of itemization in the source (with actual bullets from the Isabelle symbol repertoire). Using conventions from Markdown, it would approximate a preview of the final typesetting already in the source text. Enumerations and description lists could be done similarly. Note that there is no need for sections (which are already supported via explicit Isar commands), nor for hyperlinks and other formal inserts (which are already supported via document antiquotations).
Improved HTML output
A subset of document markup that fits into the limited Markdown / Commonmark format sketched above could be taken as starting point for better HTML presentation, with relatively modest ambitions and technical requirements. It is important to recall that “HTML” as such is a huge and vaguely defined collection of standards: producing portable output that looks nice on most browsers requires significant expertise or special tools, or both.

A few improvements have already made it into the repository after Isabelle2014. For example, the repository version Isabelle/872f330a0f8a provides support for BibTeX files in Isabelle/jEdit. In Isabelle/9986fb541c87 there is also support for @{cite} antiquotations, with formal links etc.

Flattr this!

Proposal: remote prover connectivity for Isabelle/PIDE

(See also general notes on proposals.)

“Cloud computing” is one of these buzzwords without any particular meaning, but the idea to run heavy-duty computations remotely is rather old: some “big-iron” in the background provides the CPU and memory resources for substantial applications, while the user interacts with the system via some small local terminal. Already in the classic days of Proof General (around 1999) it was common-place to run Emacs locally on a workstation and the prover process remotely on a server (via rsh). Alternatively it was possible to run both the editor and the prover remotely and use X11 as display protocol, which was especially important for the rather heavy XEmacs of that time.

This normal mode of distributed computing was almost forgotten, when the performance of local laptops and remote servers were approaching the same order of magnitude (due to the demands of the gaming industry). This was only an episode over a single decade, though, and we are already back to the traditional situation where local and remote machines can differ significantly. In 2014, typical mobile devices were limited to 2–8 CPU cores and 2–8 GB RAM. This is very little compared to low-end workstations or high-end servers, with something like 8–36 CPU cores and 32–512 GB RAM, or more.

Note that some big Isabelle applications already go beyond the possibilities of small machines with only 4–8 GB RAM, but for more memory Poly/ML process needs to be switched from 32-bit to 64-bit mode, which also doubles the memory demands. Thus there is a discontinuity here: stepping out of the “small device” category means to go for 16–32 GB RAM minimum.

This motivates the demand for remote prover connectivity for Isabelle and its Prover IDE (PIDE). The most basic approach is to run the internal socket connection for the PIDE protocol between ML and Scala over ssh. This should be sufficient for fast and reliable local networks. For non-local networks, there are the usual questions about bandwidth, latency, and reliability of the connection. The PIDE protocol requires relatively high bandwidth (which is easily provided by common DSL connections), but can afford high latency due to its asynchronous nature. Lack of reliability might turn out a real problem, though: resetting a lost TCP/IP connection naively means to restart the prover process and recheck the whole session from start, which could take minutes or hours.

Thus a more advanced approach would keep both the ML and Scala side of PIDE together on the server. Remote access then works via a separate PIDE display protocol, which is postulated here and still needs to be defined and implemented. Depending on active buffers and open text areas in the editor, the remote side would provide continuous access to incoming PIDE document markup, without demanding persistent management of the whole PIDE state locally. Loosing the connection would merely mean to reconnect the IDE to the remote Isabelle/Scala/ML component, which keeps running indefinitely.

Thus the mode of operation becomes more like the re-connection facility of VNC or RDP (but not X11). Of course it is already possible today with Isabelle2014 to use VNC or RDP for a completely remote ML/Scala/IDE process, but remote ML/Scala and local IDE would make this more comfortable for the user, with better graphics performance and reactivity.

Taking this perspective of remote PIDE sessions one step further could mean to support low-bandwidth, high-latency, unreliable connections of mobile networks: sitting on a train with a laptop and local IDE, while re-connecting to a remote PIDE session on a big server, would really count as cloud computing. We should think here of editing whole libraries like AFP on the spot, with immediate feedback. A bit more efforts will be required to get there, though.

In summary, the following stages are possible, depending on the amount of resources spent on this subject:

  1. Simple remote PIDE socket connection via ssh, usable for fast and reliable local networks. (The jEdit text editor already provides some means to manage ssh, so this merely requires the usual study of sources with subsequent tinkering and polishing to make it work smoothly.)
  2. Separate PIDE display protocol where the editor is local and the Isabelle/Scala/ML session is remote. This should be usable for fast DSL network connections.
  3. Support for smooth disconnection and re-connection for mobile networks.
  4. Development of a completely different PIDE front-end that works on tablets or smart-phones (Android or iOS).

The last point is speculative: it merely sketches to horizon of what could eventually be targeted, if there were lots of resources and several enthusiastic people working on it.

Flattr this!