(archive 'newLISPer)

October 31, 2007

Virtually Free Speech

Filed under: newLISP — newlisper @ 14:55

Not relevant for the archive version

It’s been a month since I created unbalanced-parentheses, so I thought it would be good to write a few things about this site and the service provided by my hosts NearlyFreeSpeech.net.

The site is powered by newLISP, one of the many CGI languages that NearlyFreeSpeech.NET support – there’s Haskell and OCamel there too – see the full list here. They’re running version 9.1.1 of newLISP, which isn’t too old, although version 9.2 has been out since August this year, so I’ve got to avoid using functions such as dostring, which was introduced in 9.2.1.

I gather that the hosting is on a cluster of FreeBSD servers. I use a Mac locally, but I haven’t had any problems uploading newLISP code that runs OK on my system onto their system. Well – Macs, BSD – it’s all from California, right? :-)

NearlyFreeSpeech.NET operate a policy of charging you according to how much storage space you use and how much bandwith your visitors use.

My site uses up nearly 3 MB of storage space, but most of that is the access.log file, because the content and all the newLISP code required (apart from the binary which isn’t mine anyway) takes up less than 750 KB.

As far as bandwidth goes, I don’t use up much. There are about 30-50 visitors a day, and the daily bandwidth is a couple of megabytes.

So far, then, a month of hosting has cost me $0.18. That’s 8 pence in traditional British money. Or 0.12 cents in newfangled Euros. That definitely qualifies as nearly Free Speech – in fact it’s Almost As Close As You Can Get To Free Yet Still Cost Money Speech! I’m just hoping that it doesn’t go horribly wrong and something I write gets really popular…

I’ve not had much occasion to ask for support, but I gather it’s efficient. I like the plain-speaking web site too.

The detail provided by the access log is intriguing – you can see what people did when they visited your site, in considerable and sometimes baffling detail, and sometimes you find yourself asking – ‘why are you searching for that?’ or, in extreme cases, ‘who are you anyway?’. You often don’t get access logs for free or restricted hosting, so as it’s a first for me it’s been fascinating. Although reading access logs soon gets tiresome, and it’s much better to use the stats package that NearlyFreeSpeech.NET provide. Here’s a snippet:

The software that runs this site is available in the Downloads area. It consists of the file index.cgi, a SQLite database, and a set-up file which define some global variables and contains some HTML template information. Everything else – with one exception – is provided with a standard newLISP download which you can get from the official site.

Thanks to the NearlyFreeSpeech.NET folks for doing a good job in hosting this site.


October 29, 2007

Rabbit season

Filed under: newLISP — newlisper @ 09:30

When you’ve used the little newLISP debugger to hunt down the bugs inside the functions, you might want to take a step back and observe the flow of your program, watching the data hop from one function to another. Here’s a useful technique I learnt from newLISP guru Fanda.

Using newLISP macros, you can – to some extent – change the way the language operates. (Probably many other programming languages let you do this too, not just those in the Lisp family, although I’m not familiar with any other language, so I wouldn’t know!) The following macro modifies the action of the define function which you normally use to build your code.

Put it at the beginning of your script to have all your function definitions modified:

(define-macro (my-define:my-define @farg) 
  (set (@farg 0) 
    (letex (@fn (@farg 0) 
            @arg (rest @farg) 
            @arg-p (cons 'list 
                (fn   (@x) (if (list? @x) (first @x) @x)) 
                (rest @farg))) 
            @body (cons 'begin (args))) 
       (lambda @arg 
          "\n[" '@fn   ; function name
          "]\n\t params: " 
          @arg-p     ; function's parameters
          "\n\t args: " 
          (args)     ; function's arguments 
          "\n\t[" '@fn  "] result: " 
          @body      ; result of function
  (constant (global 'newLISP-define) define)
  (constant (global 'define) my-define)

Starting at the last line: we can use constant to make the word define point to any function or macro we like, so after defining a replacement for define, called my-define, we swap out the standard version. Before doing that, it’s prudent to keeping a link to the standard version!

After defining this macro, this code runs when newLISP next encounters a define. The argument – referred to here as @farg – contains the function name, the function’s arguments, and the expressions in the function body. In newLISP, data can be code, and code can be data.

The @farg argument is the body of the function definition that you’ve written that’s passed – unevaluated – to the macro.

The body of the macro consists of a single statement, and it’s effectively (set 'function-name arguments body), equivalent to (define function-name arguments body). The letex function combines both let and expand, and expands the symbols and assigns them to local symbols.

The rest of the macro is mainly concerned with organizing the contents of the original function definition and building a new function from them. But there’s one addition: there’s a println function inserted into the function definition, for printing out the details of the function’s parameters and results.

In action, a couple of functions defined like this:

(define (average a b c)
  (div (add a b c) 3))
(define (action n1 n2 n3)
  (average (mul n1 2) (mul n2 3) (mul n3 4)))
(action 2.1 4.4 6.8)

produce this trace:

   params: (2.1 4.4 6.8)
   args: ()
  [action] result: 
   params: (4.2 13.2 27.2)
   args: ()
  [average] result: 14.86666667

The output lets you watch the data being passed to and returned by each function. I’ve found it very useful, because you can find the moment when the data gets mangled.

The @ signs are used here simply to produce symbol names that are unlikely to clash with symbol names used elsewhere. You can’t define functions that use some of the symbol names used in this macro:

(define (f @arg-p)
    (println @arg-p))

(f "cool")

symbol is protected : (list @arg-p)
called from user defined function f

It’s at this point that my brain begins to hurt, so that’s enough macros for now.

Comment from cormullion

Lutz says we don’t need the duplicated my-define:my-define in the macro name…

October 25, 2007

Duck season: part 1

Filed under: newLISP — newlisper @ 17:50

In these days of 80 GB iPods and 500 GB hard drives it’s always refreshing to see some elegant minimalism in action. newLISP is surprisingly modest in size. At just 243 KB, the newLISP binary is easily an order of magnitude smaller than just one of the JPEG images output by my digital camera. Of course, with the addition of modules, utilities, documentation, and Java graphics libraries, the size of a practical newLISP installation is a download of over 1 MB. Ouch.

Tucked away in a corner, somehow, somewhere, there’s a little debugger. It’s simple, but useful. If, like me, you haven’t used this very much, join me in this quick tour.

The key function is trace. To start and stop the debugger, use trace with either true or false:

(trace true)  ; start debugging
(trace nil)   ; stop debugging

Used on its own, trace returns true if the debugger is active.

One more thing and we’re ready to go bug hunting.The trace-highlight command lets you control the display of the expression that’s currently being evaluated, and some of the prompts. I’m using a VT100-compatible terminal, so I can use weird escape sequences to set the colours. Type this in at the newLISP prompt:

(trace-highlight "27[0;37m" "27[0;0m")

But since I can never remember this, it’s in my .init.lsp file, which runs if /usr/share/newlisp/init.lsp does when you start newLISP. If you can’t use these sequences, you can use plain strings instead. (Unicode symbols also work in this context, but I find the colour change more effective than funny little pictures.)

The other debugging function is debug. This is really just a short-cut for switching tracing on, running a function in the debugger, and then switching tracing off again.

So, suppose we want to run a file called old-file-scanner.lsp, which contains the following code:

#!/usr/bin/env newlisp

(define (walk-tree folder)
  (dolist (item (directory folder))
   (set 'item-name (string folder "/" item))
   (if (and (not (starts-with item ".")) (directory? item-name))
       ; folder 
       (walk-tree item-name)
       ; file
        (not (starts-with item "."))
        (set 'f-name (real-path item-name)) 
        (set 'mtime (file-info f-name 6))
          (> (- (date-value) mtime) (* 5 365 24 60 60)) ; non-leap years :)
            (push (list mtime item-name) results)))   

(set 'results '())
(walk-tree {/usr/share})
(map (fn (i) (println (date (first i)) { } (last i))) (sort results))

This scans a directory and subdirectories for files previously modfied 5 or more years ago. (Notice that the file ends with expressions that will be evaluated immediately the file is loaded.) First, switch tracing on:

(trace true)

Then load and start debugging:

(load {old-file-scanner.lsp})

Or, instead of those two lines, type this one:

(debug (load {old-file-scanner.lsp}))

Either way, you’ll see the first expression in the walk-tree function highlighted, awaiting evaluation:

Now you can press the s, n, or c keys (‘step’, ‘next’, and ‘continue’) to proceed through your functions: ‘step’ evaluates every expression, and steps into other functions as they are called; ‘next’ evaluates everything until it reaches the next expression at the same level; and ‘continue’ runs through without stopping again.

If you’re clever, you can put a (trace true) expression just before the place where you want to start debugging. If possible, newLISP will stop just before that expression and show you the function it’s just about to evaluate. In this case, you can start executing the script with a simple (load…) expression – don’t use debug if you want to skip over the preliminary parts of the script. I think newLISP usually prefers to enter the debugger at the start of a function or function call – you probably won’t be able to drop in to a function part way through. But you might be able to organize things so that you can do something similar. Here, for example, is a script that drops into the debugger halfway through a loop:

(set 'i 0)

(define (f1)
  (inc 'i))

(define (f2)
  (dotimes (x 100)
    (if (= i 50) (trace true))))
> (load {simplelooop.lsp})
(define (f1 )
  (inc 'i))
[-> 5 ] s|tep n|ext c|ont q|uit > i
[-> 5 ] s|tep n|ext c|ont q|uit > 

Notice how the function f1 appeared – you don’t see anything in f2 this way.

At the debugger prompt, you can type in any newLISP expression, and evaluate any functions. newLISP seems to be happy to let you change the values of some symbols, too. But don’t redefine any functions… if you try to pull the rug from underneath newLISP’s feet you’ll probably succeed in making it fall over.

The source code displayed by the debugger doesn’t include comments, so if you want to leave yourself helpful remarks – or inspiration – when looking at your code, use text strings rather than comments:

  (define (f1)
    [text]This sentence will appear in the debugger.[/text]
    ; But this sentence won't.
     (inc 'i))

Sometimes, though, it’s not the detailed workings of individual functions that you’ll want to examine. It can be more useful to stand further away and observe the data as it flows from from function to function.

Next time – after Duck Season, Rabbit Season!

Comment from cormullion

still testing cookies…

October 22, 2007


Filed under: newLISP — newlisper @ 09:15

I quite liked the tooltip on this:


October 21, 2007

Syntax this

Filed under: newLISP — newlisper @ 17:24

I made a small change in the (list-downloads) function – that’s the one that runs when you click the Downloads link below. I thought it would be nice to have Lutz’s syntax program running when you click on a link to show the file in a browser window.


} {} f { modified } mod-date { size } file-size { bytes } {


You can find syntax.cgi on the main newLISP web site. It’s a cool script.

Are all those tags still considered to be acceptable, in these days of CSS and XHTML? I think they’re deprecated (which used to mean ‘prayed against (as being evil)’). Perhaps there’s a better method.

Another thing I’ve got to work out is how to colourize code when it’s displayed in a post. That will have to wait for another day.

October 5, 2007

Jack the Glypher

Filed under: newLISP — newlisper @ 17:41

After looking into captchas and FIGlets a few weeks go, I spent a little time wondering about doing something similar in newLISP. I didn’t feel able to rewrite the FIGlet code completely, so I did some investigation of easier alternatives. In a burst of wanton glyphomania, I ended up with code that produced this. I’ve put them vertically because this layout won’t like them horizontally:

    $$$$$$$$      s$$$$$$$$$$$$$      
    $$$$$$$$$$$$$$$$$$$+  t$$$$$$$$7  
    $$$$$$$$$$$$            #$$$$$$$  
    $$$$$$$$&               $$$$$$$$  
   #$$$$$$$6                #$$$$$$$+ 
   #$$$$$$$-                 $$$$$$$$8
    $$$$$$$                   $$$$$$$$

      %$$$$$$$$$$$o     6@$$$$$$$$$$.    
   $$$$$$$$            i$$$$$$$$$$$$     
    $$$$$$$$$$$$            0$$$$$$$$$$$7

  @$$$$$               i$$$$s               $$$$$$
 t$$$$$$$$          i$$$$$$$$$$           $$$$$$$$
  #$$$$$$$+       0$$$$$$$$$$$$$        t$$$$$$$s 
   o$$$$$$$6    @$$$$$$#  #$$$$$$6     $$$$$$$@   
    &$$$$$$$  $$$$$$$      0$$$$$$-  6$$$$$$$     
      $$$$$$$$$$$$7         -$$$$$$$$$$$$$$       
        $$$$$$$$               $$$$$$$$$          

  $$$$$$$$               6$$$$$$$$$$$$$$$


      $$$                                $$$$$$$$$
    $$$$$$$$$$$$+               @$$$$$$$$$$$$$    

    $$$$$$$+        0$$$$$$$$$$$$$    
    $$$$$$$                   $$$$$$$@
   o$$$$$$$                   $$$$$$$%
   $$$$$$$+             7$$$$$$$$$$.  

For the typophiles everywhere, this is a nightmare world of tortured type and screaming serifs. Those elegant Bezier curves have just been reduced to jagged outlines. On the bright side, this was Comic Sans MS, the most hated font on the internet, so perhaps the typophiles won’t mind too much. (Anyway, it didn’t work well with Bodoni.)

The task of generating these letters involves a couple of stages. The first is to create the fonts; the second is to feed in the string that you want to display and produce the ASCII output. Of the two, the first is much more interesting.

The hard work is done by a clever little program I found on Apple’s web site, the ASCIIMoviePlayer. This is a small command-line application that interfaces with QuickTime (MacOS X and Windows only, I think) and outputs any QuickTime compatible file in ASCII format. It does movies – I’ll watch Crouching Tiger, Hidden Dragon in ASCII format next week – but any compatible image is OK too.

After making a few changes, I was ready to write the newLISP wrapper that attacks those innocent fonts and leaves the mangled letters behind.

So, here’s the script – Jack the Glypher:

(load (append (env "NEWLISPDIR") "/guiserver.lsp"))

(set 'font {Comic Sans MS} 'font-size 200  )

(define (trim-glyph glyph)
  ; strip off the white space fore and aft
  (set 'height (length glyph) 'width (length (first glyph)))
  ; find the line with the fewest spaces at the end
  (set 'longest 0)
  (dolist (row glyph)
    (set 'longest (max longest (length (trim row "" " ")))))
  ; trim all lines to the longest
  (dolist (row glyph)
    (nth-set (glyph $idx) (chop row (- (length row) longest))))

(set 'window-height 400)
(gs:window 'Glypher 200 200 200 window-height)
(gs:set-background 'Glypher 1 0 1)
(gs:panel 'GlyphPanel)

(gs:set-grid-layout 'GlyphPanel 1 1 0 0)
(gs:canvas 'glyph)
(gs:set-background 'glyph 0 0 0)

(gs:add-to 'GlyphPanel 'glyph)
(gs:add-to 'Glypher 'GlyphPanel)

(gs:set-visible 'Glypher true)
(gs:set-font 'glyph font font-size)
(gs:set-paint '(1 1 1))

(set 'start-char 32)
(map set '(x y)  (gs:get-font-metrics 'glyph {g}))

(set 'context-name (string (replace { } font "") font-size))

(while (< start-char 127)
  (gs:draw-text 'glyphchar (char start-char) 0 200)
  (set 'temp-file (string "/tmp/" {glyph} (string start-char) {.png}))
  (gs:export temp-file)
  (set 'ascii-glyph (exec (string {/usr/local/bin/ASCIIMoviePlayer } temp-file)))
  (set 'ascii-glyph (trim-glyph ascii-glyph))
  (context (sym context-name) (string {glyph} start-char) ascii-glyph)
  (inc 'start-char)
  (gs:delete-tag 'glyphchar))

; save and disappear into the night
(context (sym context-name) {height} (length ascii-glyph)) ; height, last one will do
(save (string (env {HOME}) {/Desktop/ascii-} context-name {.lsp}) (sym context-name))

This is a newLISP-GS script that produces a series of PNG files of individual letters, runs then through a hacked version of ASCIIMoviePlayer, and then saves the resulting text files in a newLISP context.

The context file starts out like this:

(context 'ComicSansMS200)

(set 'glyph100 '(
  "                                            " 
  "                                            " 
  "                                            " 
  "                                    $$$$$$$s" 
  "                                    $$$$$$$6" 
  "                                   #$$$$$$$ " 
  "                 8$$$$$$$$$$$$$    $$$$$$$  " 
  "        ,$$$$$$$$$$$$$s7it%$$$$$$$$$$$$$$$  " 
  "     7$$$$$$$                     $$$$$$$@  " 
  "    $$$$$$$@                      $$$$$$$8  " 
  "    ;$$$$$$$.                     $$$$$$$$  " 
  "       &$$$$$$$$$$$7       =$$$$$$$$$$$$$$  " 
  "               %$$$$$$$$$$$$$$$;    =$$$$#  " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            " 
  "                                            "))

So, having created a few fonts, we can write another script to load the context and build the output strings. This doesn’t have to be a newLISP-GS script:

(set 'font {ComicSansMS} 'font-size 200)
(load (string (env {HOME}) 
        (string {ascii-} font font-size) {.lsp}) 
      (sym (string font font-size)))

(set 'ctxt ComicSansMS200)

(define (ctxt:get-glyph n)
  (if (= n 32)
    (dup (dup { } 12) ctxt:height true)
    (eval (sym (string {glyph} n) ctxt))))

(define (message s)
  (let (big-buffer '())
  (dostring (i s)
    (set 'g (ctxt:get-glyph i))    
    (push g big-buffer -1))
  (dotimes (r (length (first big-buffer)))
    (dolist (i big-buffer)
      (print (i r)))

(dostring (c "newLISP")
  (message (char c))

There’s small infelicity with this script, to do with the definition of ctxt. Can you fix it?

Comment from m i c h a e l

Man, newlisper, this is definitely the work of a true bricoleur.

Somehow, with the letters composed of the characters like this, even Comic Sans can be tolerated ;-)

m i c h a e l

October 2, 2007

Hello nu (Lisp)

Filed under: newLISP — newlisper @ 11:48

In other nu-s: say hello to the nu programming language. Nu™ is an open-source programming language descended from Lisp. Read all about it here. It’s based around Objective C, which means that, in practical terms at least, it’s MacOS X only at the moment. Tim, nu’s creator, says:

In a lot of ways it’s like Ruby, but in one important way, it’s Lisp. You might say that it’s “Ruby with Parentheses”, but it’s called Nu, and it’s not just for the desktop.

And here’s a sample of nu code:

    (class ConverterController is NSObject
      (ivar (id) window (id) form)
      (imethod (id) init is
       (super init)
       (set @window 
          ((NSWindow alloc)
               initWithContentRect:'(125 513 383 175)
               styleMask:(+ NSTitledWindowMask
       (@window setTitle:"Currency Converter")
       (let (form ((NSForm alloc) initWithFrame:'(15 70 348 85)))
        ('("Exchange Rate per $1" 
           "Dollars to Convert" 
           "Amount in Other Currency") each:
          (do (text) (form addEntry:text)))
        (form set: (interlineSpacing:9 
        ((@window contentView) addSubview:form)
        (set @form form))
       ((@window contentView) addSubview:((NSBox alloc) initWithFrame:'(15 59 353 2)))
       (let (button ((NSButton alloc) initWithFrame:'(247 15 90 30)))
            (button set: 
            ((@window contentView) addSubview:button))
       (@window orderFront:nil)
      (imethod (void) convert: (id) sender is
         ((@form cellAtIndex:2) 
            ((* ((@form cellAtIndex:0) floatValue)
                ((@form cellAtIndex:1) floatValue))

As you can see, you’ll be better off if you already know your way around Objective C (notice all those NS objects?). Unfortunately I’m already using up most of my brain cells working with newLISP, and I didn’t see any tutorials floating around to make learning Nu any easier than it looks. But it’s great to see people coming up with new languages.

Comment from cormullion

Sorry about the formatting. It’s hard to retain the original’s formatting given the limited width of the text area…

October 1, 2007

ThisService reaches version 2

Filed under: newLISP — newlisper @ 17:04

The cool utility ThisService from the excellently-named WaffleSoftware has been updated to version 2. It’s a clever way of running scripts written in scripting languages – such as newLISP – from the Services menu (the menu that’s available in every application, with a few dishonourable exceptions). It’s MacOS X specific – there’s probably an equivalent on other platforms.

One good addition is that you can now make services that link to scripts stored somewhere sensible on your disk. Before, your script was copied into the service, but now you can keep scripts in their rightful place and yet still use them in the services you create.

To complement this change, there’s also a packaging option, so that scripts and services can be merged ready for distribution.

The example scripts provided with this donation-ware application (written in Ruby, Perl, Python, and AppleScript – the four horsemen of Macintosh scripting) all seem to avoid reading line by line from STDIN, instead preferring to read any text input into a variable first. The word ‘slurp’ is used…

In newLISP, a typical service might look like this. This puts a comment before every line of the selection:

 #!/usr/bin/env newlisp

 (while (read-line)
   (push (current-line) the-text -1))

 (dolist (line the-text)
   (println {; } line))


As before, there will be the inevitable problems with applications that use “\r” rather than “\n” as line breaks. I think I’ve bored myself with that before!

Comment from Jesper

As the writer of the Ruby, Perl and AppleScript versions (and, also, of ThisService itself – hello everyone) I just want to defend my usage of slurping. It’s really easy – most of the time you want to run the whole input through a filter, and that might not work well with line-by-line or word-by-word or 512-summat-byte-block-by-block.

It will confuse the least amount of people who just want to get stuff done. Peter Hosey wrote the Python scripts completely independently (I said to comment well and keep STDIN/OUT UTF-8 and that was basically it), and he too chose slurping. I’d like to believe that this confirms my suspicions. ;)

Comment from cormullion

Hi! Slurping is cool. And it’s what I used too. I just quite like the word, which is much more descriptive than the usual way of describing it!

And ThisService is cool too, Jesper! If I was currently employed, I’d send you some money :-)

Blog at WordPress.com.