(archive 'newLISPer)

April 11, 2006

The road to (new)LISP

Filed under: newLISP — newlisper @ 15:35


Someone asked me why I’m here, learning newLISP. If you want to know, read on.

In the 1990s I worked for a large company, in their technical publications division. The software we used was a document publishing system called Interleaf, running on Sun workstations.

Interleaf was a very powerful, if quirky and idiosyncratic, DTP system. I don’t think there’s anything on the market today which is as powerful as Interleaf was 10 years ago. Even FrameMaker, the long document processing application that Adobe purchased (and has neglected ever since), couldn’t match Interleaf in its prime.

I’ve often wondered how on earth Interleaf managed to screw it up so quickly – they filed for bankruptcy not long after having installed their system in most of the really big companies. Perhaps it was the reliance on their proprietary ‘desktop GUI’ system – they had developed a complete Mac-like desktop interface during the 1980s, possibly inspired by the Xerox Star rather than by the Mac itself, and they didn’t switch quickly enough to the more mainstream GUI systems such as Windows.

Still, at the time, Interleaf was the best choice for producing large documents. And Interleaf Inc. must have thought that their killer feature would be the choice of Lisp as the customization and scripting language for the release of Interleaf version 5. The entire top layer of the product was a ‘Lisp Zone’ – you used Lisp files to configure the graphical user interface, the keyboard layouts, the menu entries, and so on. And you used Lisp functions to access the underlying ‘engine’, and these functions called the routines that did the real text and image processing work. There was a built-in Lisp terminal (the Listener), a little Lisp Editor with Emacs-friendly keyboard commands, and special icons to represent Lisp scripts.

I found Interleaf Lisp easy and useful. It struck me as a simple and elegant language, expressive enough to handle different types of job yet small and consistent enough so that I didn’t have to think too much about the tools, and could concentrate on the task in hand. They do say that ‘ignorance is bliss’!

The Lisp scripts I wrote would probably not be considered ‘real’ Lisp at all, by the standards of the professional Lispers who frequent the Lisp cyberworld these days. The only requirement at the time was that the scripts were quick, reasonably reliable, and well commented. I hadn’t learnt about closures, s-expressions, lambda calculus, pointers, object orientation, garbage collection, dynamic and lexical scoping (Interleaf Lisp had dynamic scoping, but I didn’t know what that was), type declarations, and all that stuff. At the time I had no idea that Lisp was considered to be the preferred language of the programming world’s intelligentsia, the playground of academics rather than a pragmatic workhorse of industry. The only other place I’d heard of Lisp was in AutoCAD.

Looking at these scripts now is interesting. Here’s a function from a very early Interleaf to HTML translator I wrote in about 1993:

(defun tokens-to-markup (cmpn)
  (let (data m1 m2 toks tok (charct 0) key)
     ;; get all the tokens in the component
     (setq m1 (tell cmpn mid:get-marker nil))
     (setq m2 (tell cmpn mid:get-marker t))
     (setq toks (tell m1 mid:get-substring m2))
     ;; loop through all the tokens
     (while (setq tok (pop toks))
        (setq charct (1+ charct))
        (setq key (car tok)) (setq data (cdr tok))
         ((eql :char key)        (html-write-char data))
         ((eql :glue key)        (html-glue-fcn data ))
         ((eql :doc-font key)    (html-font-fcn data))
         ((eql :hyph-char key)   (html-write-char data))
         ((eql :frame key)       (html-frame-fcn data))
         ((eql :autonumber key)  (html-autonumb-fcn data tok))
         ((eql :hard-return key) (html-write-hr data))
         ((eql :tab key)         (html-write-tab data))))))

The tell functions send messages to Interleaf objects, such as a paragraph containing some text.

And here’s a simple function that creates an index marker from selected text, or, if there’s no selection, opens up the Index marker dialog box:

(defun create-an-index-token ()
"put selected text into new index token, or open new index token"
 (if (tell *text-editor* mid:get-selection)
  (setq text-markers (tell *text-editor* mid:get-selection))
  (setq text (tell (car text-markers) mid:get-substring (cdr text-markers) t t))
  (tell *text-editor* mid:create :index (format nil "~A" text))
  (wn-message nil (concat "Put \"" text "\" in Level 1 of index token.")))
 (tell *text-editor* mid:create :index)))

While Interleaf was filing for Chapter 11 bankruptcy, the desktop publishing action moved elsewhere, to FrameMaker, Mac OS, Windows, Quark Xpress, PageMaker, and so on. On Macs, DTP applications were scripted by AppleScript, and on Windows, by Visual Basic. And I forgot about Lisp for a decade.

AppleScript is an interesting language. It was a brave attempt by Apple to design an easy-to-use programming language with a natural English-like syntax, rather than the algebraic form preferred by languages such as BASIC and C. Matt Neuburg observes that the developers adopted some ideas from Lisp and Scheme languages, and combined them with HyperTalk and other fashionable trends of the day. In his Definitive Guide to AppleScript, he writes:

A number of features of AppleScript seem to suggest that someone on the original AppleScript team was fond of LISP (or some LISP dialect such as Scheme).

AppleScript is really the only way you can easily talk to the applications running on your Mac, and it’s the way that applications communicate with each other and with the system, so you can hardly avoid using it. But if you have to process text files or interact with the Unix engine, there are plenty of alternatives. So, because my AppleScripts were taking ages to run and often hanging, or quitting with ‘stack full’ errors, I decided to investigate some of the other languages available. I liked Python and Ruby, but was less keen on Perl. But after using Ruby for a while, and reading about some of its Lisp-like features, I thought it would be cool to find another simple and easy to use version of Lisp. After 10 years there must have been a lot of progress. Or so I thought.

In fact, I spent nearly three days trying to find and install a Lisp system! (Well, off and on.) I followed the various links in the CLiki – Common Lisp Wiki, to try and find a version of Lisp that offered some of the simplicity that I thought was the hallmark of Lisp, at least in my own experience.

At first I was unsuccessful, but I learned many things. I learned:

  • that there are lots of versions of Common Lisp
  • that Lisp wasn’t always as simple as I thought it was
  • that I didn’t know what a makefile was
  • that not many people know how to write a proper installer for Mac OS X
  • that documentation isn’t a priority for most developers
  • that excessive use of Common Lisp can make you take the world too seriously

Eventually I discovered newLISP: it had a proper Mac OS X installer, it was fast, compact, full-featured (most of the stuff you need is built-in) and well-documented. It works, the way I want it to work. (And the way Mac users have expected their systems to work for years.) Most of all, it offers some of that simplicity that I had first glimpsed in Interleaf Lisp. There aren’t any hang-ups about having to do things the academically pure way, and no sense that efficiency of execution or ease of programming have been sacrificed for the sake of some theoretical principle.

There aren’t too many substantial differences between Interleaf Lisp and newLISP, the way I use them. newLISP generally has easier syntax. The rough edges have been smoothed out and the terminology improved, so we can forget about ‘car’ and ‘cdr’, ‘rplaca’ and ‘nconc’, ‘defun’ and ‘mapcar’, ‘progn’ and ‘terpri’, and replace them with friendlier names. Equality predicates have been simplified, and you can refer to functions in a tidier way (without having to use “#'” all the time). Best of all, newLISP can usually do most of the things you ask it to with a built-in function.

However, the big thing I miss is that Interleaf Lisp was integrated with the application. With newLISP I can’t talk to my Mac applications as easily. For example, to get the name of the currently playing track in iTunes I would love to be able to write something like this:

(tell iTunes get-current-track-name)

whereas at present I have to do this:

(exec [text]osascript -e 'tell application "iTunes"
    get the name of the current track
    end tell
' [/text] )

which isn’t so Lisp-y, and you have to be able to write the script in AppleScript first, then quote it to within an inch of its life so that it makes it through the shell unharmed.

I can’t resist finishing with this comment about newLISP on CLiki:

This is NOT Common Lisp, but if you’re migrating from Python, Ruby, Perl, etc, this may be just what you’re looking for. Likewise, if you need/want one consistent Lisp-like system to run on various platforms, consider using this for a while… then come back to Common Lisp.

Well, I might come back to Common Lisp one day. When Common Lisp is as useful as newLISP, perhaps!



  1. >Im very new to Lisp but would it be a lot of work to hook newLisp up to applescript likt the pythion guys have done with appscript.An Objc bridge would also be brilliant. I have been looking at OpenMCLs Objc bridge and the code you write with it is much nicer than the code you write with something like gtk-server and newLisp.

    Comment by Anonymous — April 12, 2006 @ 08:51 | Reply

  2. >Well, I always think that it depends on whether you’re doing something for the first time (which takes ages) or doing something that you’ve done before. Someone who had experience in ObjC and MacOS development would find it much easier and faster than someone who hadn’t.The author of appscript is an AppleScript guru as well as being a Python guru. Which would help!I personally wouldn’t know where to start! ;-)

    Comment by newlisper — April 12, 2006 @ 11:24 | Reply

  3. >Thanks for this well working iTunes hack. Others readers should note, that there is a tick ‘ delimiter after: end tell, which the formatting slipped on to the next line.

    Comment by don Lucio — April 14, 2006 @ 21:23 | Reply

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Blog at WordPress.com.

%d bloggers like this: