(archive 'newLISPer)

January 11, 2006

Script analysis

Filed under: newLISP — newlisper @ 16:28

newLISP can be a very concise language. If you’re a novice scripter, like me, you might sometimes be puzzled at the brevity of the scripts written by advanced newLISP gurus. So I thought it might be interesting to look at one of my simple newLISP scripts in detail, to see whether brevity arises from the language or from the skill and experience of the newLISP ninja. Experienced newLISPers and/or MacOS X users can tell me whether I’m getting into bad habits, if they can be bothered to read all this.

Here’s a short newLISP script that ties together three components of the MacOS X Tiger operating system. It runs on selected graphics files, and I usually start it using Big Cat (which I mentioned earlier). It uses an image processing tool called sips, which is a useful utility for processing graphics files, and displays confirmation using a notification tool called Growl. Its job is simply to scale all selected images by 50%.

 1 #!/usr/bin/newlisp
 3 (set 'file-list (rest (rest (main-args))))
 5 (define (reduce-file file-name)
 6   (letn
 7    ((image-data
 8       (exec (format "sips -g pixelHeight -g pixelWidth '%s' " file-name)))
 9     (old-pixel-width
10      (integer (nth 1 (parse (nth 2 image-data)))))
11    (new-pixel-width
12      (/ old-pixel-width 2))
13    (sips-command
14      (format "sips --resampleWidth %d '%s' " new-pixel-width file-name)))
15    (and
16      (copy-file (string file-name) (string (replace "." (string file-name) "-1.")))
17      (exec sips-command)
18      (exec (format "/usr/local/bin/growlnotify %s -m \"processing file  '%s'\"" (date) file-name)))))
20 (dolist (f file-list)
21  (reduce-file f))
23 (exit)

Line 3 sets the symbol file-list so that it contains the names of the files. The main-args method knows all the arguments that were handed to this invocation of the newLISP program, but we don’t want the first one (“/usr/bin/newlisp”) or the second one (the name of this newLISP script, whatever it might be), so we take the rest of the rest of the arguments. In theory I prefer to use longer variable names such as file-list rather than f or l, but in practice I don’t always remember to!

Line 5 sees the start of a function that will do all the hard work. It will process the file in file-name (which is going to be passed to it in line 21 – you have to define the functions first, before you call them).

Line 6 starts a block (not sure if that’s the right word?) with letn. This block continues all the way to line 18, the end of the definition of the reduce-file function. letn has two parts: in the first part you declare local variables, and in the second part you evaluate expressions that can access those local variables. I usually define variables in this more expansive style:

(set 'x 1)
(set 'y 2)
(set 'z 3)

but I’m trying to learn the more advanced techniques of letn and let:

   ( (symbol value)
     (symbol value) )
   (body) )

These symbols are local rather than global, which I gather is considered good practice (but probably not important for these small scripts).

Why letn rather than let, though? The reason is that line 10 uses the value of the symbol image-data, which was defined in lines 7 and 8. If I’d used let, line 10 wouldn’t work, because the value of image-data wouldn’t have been available until the end of the symbol declaration section (line 14). letn does the right thing – it’s a ‘nested let’, allowing you to access symbol values as soon as they’re defined.

 8      (exec (format "sips -g pixelHeight -g pixelWidth '%s' " file-name)))

Line 8 runs the sips command, with the sole aim of getting the file’s current width and height. I couldn’t find a way to get and set image properties simultaneously, so I’m calling sips twice, first to get the value, then, in line 14, to divide it by two. The exec command runs the sips command, and I’m using format to create the command that gets sent to the shell. Fortunately format uses the more familiar printf() style formatting conventions, rather than Common Lisp’s language-within-a-language version, so you can probably tell what’s happening. It’s a useful way to be able to add or change command options, too. Notice the single quotes that enclose the file name placeholder %s: this is because a lot of my files have spaces in their names…

The result of the command, in image-data, is a list of three strings:

("/Users/me/untitled folder/image045.jpg" "  pixelHeight: 437" "  pixelWidth: 525")

so lines 9 and 10 do a quick bit of parsing to get the width value and convert it to an integer (eg 525):

 9     (old-pixel-width
10      (integer (nth 1 (parse (nth 2 image-data)))))
  • this isn’t very elegant, really, but I couldn’t think of a better or easier way. When decoding these, remember that all indexing starts at zero, so parse works on the third element of image-data, and integer works on the second element of the resulting list returned by parse.

Lines 11 and 12 calculate the new width and save it in new-pixel-width, using integer division (/) rather than floating point division (div):

11   (new-pixel-width
12      (/ old-pixel-width 2))

Now I can build the second invocation of the sips command (lines 13 and 14) and assign it to the symbol sips-command. The reason I do this in two stages is that I’ve found it useful to be able to check what command actually got sent, and it’s easier to insert debugging or logging statements if required. In a way, this has made the script longer, however…

13   (sips-command
14      (format "sips --resampleWidth %d '%s' " new-pixel-width file-name)))

I’m using a good text editor, so balancing those parentheses is very easy. If not, it would have been hard to remember to put the extra parenthesis at the end of line 14, which is needed because this is the end of the symbol definition section of letn.

The second, or body, part of the letn block is for the expressions that will actually do stuff with these local symbols.

15  (and
16    (copy-file (string file-name) (string (replace "." (string file-name) "-1.")))
17    (exec sips-command)

I’m using and here to evaluate a series of expressions. I don’t need to, because let accepts one or more expressions as part of its body section, but if it didn’t I could have used begin. When you’re using newLISP, you need to know when you can supply a series of expressions, and when you can’t. However, and is useful here because it evaluates each expression in turn but stops as soon as one expression fails (ie evaluates to nil or an empty list). This saves you having to test the results returned by each expression. Of course, sometimes things fail without returning nil or the empty list, but that’s another angle which I’ve forgotten to cover.

18    (exec (format "/usr/local/bin/growlnotify %s -m \"processing file  '%s'\"" (date) file-name)))))

In line 18, I’m calling Growl. Growl is a free utility for MacOS X that provides a notification service – basically pop-up windows that appear unobtrusively on the screen for a while and then fade away (or go when you click them). Growl has been around for a while now, and applications are starting to support it – there’s no system service that does anything similar. I’ve got quite a lot of Growl notifications already set up: new incoming email messages, song titles being played by iTunes, and so on, and I’ve become accustomed to this way of receiving gentle reminders and transient information. Here I’m simply calling growlnotify, the shell version, because newLISP doesn’t have a built-in interface to it. The date function provides a date stamp.

Finally, the dolist function runs my reduce-file function for every file in the list of files.

20 (dolist (f file-list)
21  (reduce-file f))

Do I need exit to finish? I’m not sure whether I do or not. It might be a habit that carries over from using newLISP in a terminal.

23 (exit)

So, is conciseness a consequence of the language or a habit of the writer? I suspect that you can get satisfaction from producing an extremely concise script, but also from producing an easy to read and well-commented script. I do know that occasionally I would really like a program that analyses other people’s newLISP programs and produces an explanation as detailed as this one!



  1. >For line 10 you might look into implicit indexing:instead of:(integer (nth 1 (parse (nth 2 image-data))))do implicit indexing:(integer (parse (image-data 2) 1))it is easier to read because the index comes after the object to be indexed. There is a chapter about this in the manual. Sometimes implicit indexing may obscure things, but this is a case where it makes the code easier to read.In line 16 you don’t need 2 of the ‘string’ wraps, instead of:(copy-file (string file-name) (string (replace “.” (string file-name) “-1.”)))Do a shorter:(copy-file file-name (replace “.” (string file-name) “-1.”)))Only the last time ‘string’ is necessary to protect ‘file-name’ from beeing changed by the destructive ‘replace’. The other two times ‘string’ is only converting to a string what is a string already.

    Comment by Lutz — January 12, 2006 @ 13:03 | Reply

  2. >Thanks, Lutz! As ever your help is welcomed, and your improvements welcome!

    Comment by newlisper — January 12, 2006 @ 14:39 | Reply

  3. >Some short optimizations:Use implicit slicing instead of rests.(set ‘file-list (2 (main-args)))I also prefer to use implicit lets with sets rather than letn. This allows the reader to see the local variables in the definition of the function.(define (reduce-file file-name , image-data old-pixel-width new-pixel-width sips-command) (set ‘image-data (exec (format “sips -g pixelHeight -g pixelWidth ‘%s’ ” file-name)) ‘old-pixel-width (integer (parse (image-data 2) 1)) ‘new-pixel-width (/ old-pixel-width 2) ‘sips-command (format “sips –resampleWidth %d ‘%s’ ” new-pixel-width file-name) )And if you want to really shorten your code instead of the last dolist you can do the following(map reduce-file file-list)This last one is more a matter of preference but I normally use (string) instead of (format) — especially when I’m not worrying about decimal formatting.(exec (string “sips -g pixelHeight -g pixelWidth ‘” file-name “‘”))If you can guarantee that your variable is a string – use append – it’s faster.One thing to note using (main-args) — all main-args are strings. This can be important when you pass integers by the command-line.

    Comment by Gordon — January 18, 2006 @ 22:31 | Reply

  4. >Good comments by Gordon, although I disagree with one of them. Using let is much clearer to me than using “implicit lets with sets,” for two reasons. One is > (set ‘x 42)42> (define (f y) (set ‘x (+ y 2)))(lambda (y) (set ‘x (+ y 2)))> (f 55)57> x57Oops! What happened to ‘x’? The use of ‘let’ will protect the top level ‘x’ (if you didn’t really mean to change the top-level ‘x’), and the uses of functions ‘g’ and ‘h’ below demonstrate this:> (set ‘x 42)42> (define (g y) (let (x) (set ‘x (+ y 2))))(lambda (y) (let (x) (set ‘x (+ y 2))))> (g 55)57> x42> (define (h y) (let ((x (+ y 2))) x))(lambda (y) (let ((x (+ y 2))) x))> (h 55)57> x42Another reason to use ‘let’ is that it tells the reader of your code: “these are my local variables.” If you ‘set’ any variable in your function which is not on the ‘let’ list, you might mean to change the state of a top-level variable. However this may be sloppy — it’s better to shadow the top-level variable with a local ‘let’ binding and the new value will obtain through to subsequent function calls, because of dynamic binding.> (define *my-parm* 42)42> (define (f x) (let ((*my-parm* 24)) (g x)))(lambda (x) (let ((*my-parm* 24)) (g x)))> (define (g y) (+ *my-parm* y))(lambda (y) (+ *my-parm* y))> (f 1)25

    Comment by Rick Hanson — April 30, 2006 @ 03:11 | Reply

  5. >Have you seen this (from SICP)?:”The general form of a let expression is(let ((<var1> <exp1>) (<var2> <exp2>) … (<varn> <expn>)) <body>)The first part of the let expression is a list of name-expression pairs. When the let is evaluated, each name is associated with the value of the corresponding expression. The body of the let is evaluated with these names bound as local variables. The way this happens is that the let expression is interpreted as an alternate syntax for((lambda (<var1> … <varn>) <body>) <exp1> … <expn>)No new mechanism is required in the interpreter in order to provide local variables. A let expression is simply syntactic sugar for the underlying lambda application.”I always used to wonder why there was a ‘let’ versus ‘let*’ (or ‘letn’ in newLISP), and SICP answered my question beautifully in this passage.

    Comment by Rick Hanson — April 30, 2006 @ 03:33 | Reply

  6. >Thanks, Rick. I’ve never heard of SICP, so that’s something for me to look at.Gordon was using the “,” technique for defining local variables in the function definition. Does this do the same thing as the (let …) syntax?

    Comment by newlisper — April 30, 2006 @ 08:40 | Reply

  7. >Yes, you are right — Gordon is scoping his local variables using extra function parameters. Lutz says this in the newLISP manual, BTW:”Some of the functions in the example programs use a comma to separate the parameter list in two groups. This is not a special syntax of newLISP but rather a visual trick. The comma is just a symbol like any other symbol. The parameters after the comma are not required when calling the function; they declare local variables in a convenient way. This technique is based on the fact that parameter variables in a lambda expression are local and that in newLISP parameters in lambda expressions are optional and not required to be filled in by the caller.”So I guess it’s OK, but it “feels” sloppy. (I am a programmer, but I still feel! :-) This trick reminds me of the very same one we use in awk functions to declare local variables (as there is no ‘let’ in awk). I hated it then and I still hate it. :-)

    Comment by Rick Hanson — April 30, 2006 @ 21:03 | Reply

  8. >Oh sorry I forgot to answer your question: “Gordon was using the “,” technique for defining local variables in the function definition. Does this do the same thing as the (let …) syntax?”Effectively, yes. In the first case, you can have local variables with “the comma trick” by ‘set’ting (non-mandatory) extra function arguments — they being your “local” variables. In the ‘let’ case, you have an anonymous function inside your original function, and your “local” variables are the parameters of the (inside) anonymous function. I like the separation afforded the second case, and it also let’s me avoid, in most cases, using the ‘set’, which I try to avoid of possible. Sometimes, though, it’s use can’t be helped.

    Comment by Rick Hanson — April 30, 2006 @ 21:16 | Reply

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: