(archive 'newLISPer)

December 29, 2005

Easy peasy

Filed under: newLISP — newlisper @ 21:37

One of the great things about being a Dad is that you get the chance to share the excitement and enthusiasm that young children have for the world around them – at least, before they grow cynical and jaded like us adults. Kids are full of wonder, and some of them are even interested in numbers and sums – for a while! Computers can be great tools for exploring.

For fun I decided to use newLISP to show some simple primary school mathematics to a receptive audience (of one). newLISP is actually quite suitable for this purpose, with its fast, simple, interactive, and non-mathematical syntax, and it was challenging for me to come up with the answers on the spot.

Perhaps only younger children would watch with wonder as the following expression scrolls inexorably through to the end:

(sequence 1 1000000)

But it is a real million! “Wow” was the appreciative response. It’s the printing and scrolling that takes the time, rather than the counting, on my machine, because:

(time (sequence 1 1000000)) ;- 279 milliseconds

although I think the computer’s cheating if it’s not printing the numbers out.

The sequence function also lets you specify a step other than 1 as an optional third argument.

(sequence 1 1000 7)

This is good for making sure that the computer knows its tables.

sequence has a big brother, series, which is good for showing what happens when you multiply rather than add. Sensibly, you don’t specify the upper limit this time, just the multiplying factor and the number of repeats:

(series 2 2 30)
;-> (2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384
32768 65536 131072 262144 524288 1048576 2097152
4194304 8388608 16777216 33554432 67108864 134217728
268435456 536870912 1073741824)

The factor function is extremely quick, but didn’t supply the information that the audience was hoping to see:

(println (factor 100))
;-> (2 2 5 5)

with cries of “What about 10, 25, and 50?”! Perhaps schools teach factors differently. But this version met with approval:

(define (factors n , factor-list)
    "return a list of all the factors of n"
    ((< n 4) nil ) ; don't bother if n is 1, 2, or 3
        (let (limit (/ n 2)) ; only do up to n/2
        (for (x 2 limit)
            (if (= (% n x) 0) ; add x to list of factors if no remainder
                (push x factor-list -1))))
            factor-list))) ; return list
(for (x 1 1000) (println x " " (factors x)))
1 nil
2 nil
3 nil
4 (2)
5 nil
6 (2 3)
7 nil
8 (2 4)
9 (3)
10 (2 5)
11 nil
12 (2 3 4 6)

There’s been some discussions here, about whether 1 is a factor of everything, whether 1 is prime, and whether zero is even, to name just a few. You can’t always convince young people that these questions have been resolved by respected authorities hundreds of years ago. Notice the fudging in the first cond term, which allowed me to produce the required results.

But we can use the official factor function to write a function that tests whether a number is prime (another source of fascination for the budding young mathematician).

(These test functions are called predicates. newLISP provides a set of predicate functions whose names end with question marks. We can list them by using the filter function, looking through a list of symbols for names ending with “?”:

(println (filter (lambda (s) (ends-with (name s) "?" )) (symbols 'MAIN)))
; ->
(? NaN? array? atom? context? directory? empty? file? float? integer?
lambda? legal? list? macro? nil? primitive? quote? string? symbol?

I seem to remember that predicates in classic Lisp end with the letter “p”. newLISP’s idea of using a question mark seems a more natural solution.)

But back to the prime numbers. A prime number predicate can be just a simple test of how many numbers are returned by factor. All the prime numbers return themselves when tested with factor, so we see if length is equal to 1:

(define (prime? n)
    (=      (length (factor n))

And then we can easily find all the prime numbers up to a million:

(for (n 1 1000000)
    (println n " " (prime? n)))

which also takes the computer a minute or two! An internet search revealed that some people spend a lot of effort trying to calculate prime numbers quickly, so I think we should stop now.


December 21, 2005

Christmas quiz: four by four

Filed under: newLISP — newlisper @ 10:53

Here’s a version of a traditional puzzle to exercise your mental maths muscles before Christmas. In newLISP, define symbols zero through nine that evaluate to their integer equivalents. In other words, define symbols so that you can type this into a newLISP console:

> zero one two three four five six seven eight nine

and get the answer:

0 1 2 3 4 5 6 7 8 9

However, the restrictions are that you cannot use any digits other than 4, and you must use four 4s, no fewer and no more. You can also use all the arithmetic operators (and parentheses, obviously).

To get you started, here’s the easiest one – zero:

(set 'zero (- 44 44))

December 18, 2005

ISO dates

Filed under: newLISP — newlisper @ 21:35

For another web site (not this one), I occasionally have to generate an RSS feed ‘manually’, by running a short script that extracts headlines and paragraphs and puts them into an XML file. Thanks to newLISP’s regular expressions, this is not a difficult (or interesting) task.

However, one small problem I had was that, for some strange reason, the RSS XML standard doesn’t seem to allow the use of dates written using the ISO 8601 standard. So I had to convert these into the RFC822 standard. Again, newLISP offered a convenient solution.

The parse function is great because it allows you to specify more than one delimiter, using a regular expression. So, this ISO format date:

2005-10-16 12:12:12

is easily split into its constituent parts if you use three delimiters, ” “, “-“, and “:”. (The 0 says to use regular expressions.)

(parse "2005-10-16 12:12:12" { |-|:} 0)

which returns (“2005” “10” “16” “12” “12” “12”). These strings are then converted to integers by mapping the int function over the list:

(map int (parse "2005-10-16 12:12:12" { |-|:} 0))

which returns (2005 10 16 12 12 12). These numbers happen to be in the right order for use with date-value, which gives them temporal significance. Since date-value wants six integers, we can use apply to apply the date-value function directly to the six list elements.

(apply date-value (map int (parse "2005-10-16 12:12:12" { |-|:} 0)))

The integer result, 1129464732, is the time in seconds of our ISO date since 1970-1-1 00:00:00. Finally, date is now able to output a suitable string given this number. date uses the standard strftime formatting techniques, so it’s easy to find the right mystic runes for RFC822:

(date (apply date-value (map int (parse "2005-10-16 12:12:12" { |-|:} 0)))  0 "%a, %d %b %Y %H:%M %Z" )

giving us “Sun, 16 Oct 2005 13:12 BST”.

December 16, 2005

Is this the right room for an argument?

Filed under: newLISP — newlisper @ 15:13

One of the pleasant things about Lisp is the arguments. No, not the sort of argument that you’ll find on Planet Lisp, I mean the way you supply information to functions. For those of us more familiar with other languages, Lisp seems very casual about the whole business.

For example, you’d expect to be able to do this:

(+ 2 2) ;-> 4

but being able to do this seems odd at first:

(* 2) ;-> 2

but then it’s nice to be able to do this:

(div 1 2 3 4 5 6) ;-> 0.001388888889

I think you can supply any number of arguments to a Lisp function.

You can also define functions using this very casual approach to specifying how many arguments are required. Say I want to write a simple mean command. This would be a good start:

(define (mean num-list)
    (div    (apply add num-list)
            (length num-list)))
(println (mean '(1 2 3 4 5 6 7 8 9 10))) ;-> 5.5

The apply function adds the list argument, then the sum is divided by the length of the list.

Another way to do this is to use the args function. This accesses the arguments passed to the function.

(define (mean)
    (if (args)
        (div    (apply add (args))
                (length (args)))))

which lets us type any of the following:

(mean)                             ;-> ( )
(mean 1)                           ;-> 1
(mean 1 2)                         ;-> 1.5
(mean 1 2 3 4 5 6 7 8 9 10)        ;-> 5.5

which is more flexible. And newLISP offers even more ways of specifying arguments to functions, and things can get more complicated when you start using macros and lambda expressions.

But at this level, the hardest thing is to remember and make use of the flexibility offered.

Finding duplicate files on MacOS X

Filed under: newLISP — newlisper @ 09:56

One of the problems that apparently only I encounter is that of duplicate files. It’s probably due to the disorganized way I manage my projects. But there’s no obvious way to find duplicate files. There are some applications that can find them, but I haven’t found a way using the command line. There are some interesting problems:

– similar files don’t always have the same names!

Surprisingly, identical files don’t always have similar names. When you duplicate a file using the Finder, the name of the copy is changed. When you download files using Safari, numbers are appended. And so on. And then you make a safe backup copy of a folder while working on a new project.

– resource-fork files look empty

There are some files that, when you list them at the command line, appear to be 0 bytes. For example:

$ ls -l /System/Library/Fonts/H*
-rw-r--r--   1 root  wheel  7501763 Mar 20  2005 /System/Library/Fonts/Hei.dfont
-rw-r--r--   1 root  wheel        0 Mar 28  2005 /System/Library/Fonts/HelveLTMM
-rw-r--r--   1 root  wheel        0 Mar 28  2005 /System/Library/Fonts/Helvetica LT MM
-rw-r--r--   1 root  wheel   991360 Mar 20  2005 /System/Library/Fonts/Helvetica.dfont

These are resource-fork files, a clever technology inherited from the classic (ie old) MacOS which occasionally causes a few headaches in the brave new world of classic (ie old) Unix. (The technique for finding them involves adding /rsrc to the path.)

A newLISP tool for finding duplicates

I don’t recommend running this script on an entire disk. You specify one or two folders to examine. The script collects the filenames and their sizes in a simple Lisp list, then sorts the list. (This alone warns you not to tackle more than 20000 files!) Then, if two files appear to have the same size, the MD5 checksums are compared. I’m making the assumption that if two files have the same size and the same MD5 checksum, they’re probably duplicates. Finally, I’m not doing anything with the results except listing them – I’m never entirely happy with scripts that delete things for me.

I’ve found newLISP to be an excellent tool for this sort of job. I’m not interested in performance – it’s just the sort of task to run while you do something more interesting!


;; finds duplicate files
;; ignores filenames but tests sizes and MacOS resource forks
;; usage: find-duplicate-files.lsp [folder1 folder2 ...]
;; needs version containing (real-path)

(define (set-spotlight-comment file comment)
    "set Spotlight comment of file to comment"
        (exec (format [text]osascript -e 'set pf to POSIX file "%s" ' -e 'tell application "Finder" to set comment of pf to "%s" ' [/text] file comment)))

(define (walk-tree folder , item-name )
" build a list of all files in the folder, with sizes:
    ((size1 file1) (size2 file2) ... )"
    (dolist (item (directory folder))
        (set 'item-name (string folder "/" item))
        (if (and
                    (directory? item-name)
                    (!= item ".")
                    (!= item ".."))
            (walk-tree item-name) ; recurse
            ; else process the item
                (not (starts-with item ".")) ; skip hidden files
                (set 'path-name (real-path item-name))
                (file-info path-name) ; skip symlinks...
                (set 'dataforksize (first (file-info path-name)))
                (if (file? (format {"%s"/..namedfork/rsrc} path-name ))
                    ; add resource fork size if one exists at /..namedfork/rsrc
                    (set 'resourceforksize
                        (first (file-info (format {"%s"/..namedfork/rsrc} path-name ))))
                    (set 'resourceforksize 0))
                ; put composite file size and file name into dupe-list
                (push (cons (+ dataforksize resourceforksize ) path-name ) dupe-list -1 )))))

; start
(if (> (length (main-args)) 2)
    (dolist (folder (2 (main-args)))
        (println "... gathering files in folder " (real-path folder) "\n")
        (walk-tree folder))
        (println "... gathering files in folder " (real-path) "\n")
        (walk-tree (real-path))))

(println "... sorting " (length dupe-list) " items\n")
(set 'dupe-list (sort dupe-list )) ; sort by size - very important!
(println "... duplicates are: \n")

; see if two adjacent items have the same size

; this is a kludge to avoid an error
; If we start with item 1, we have no 'previous' pair for comparison
; I'd really like to start at item 2...
(set 'previous (last dupe-list))

    (current dupe-list)
    (if (= (first current) (first previous)) ; current same size as previous?
            ; same size, compare md5 checksums

            (set 'current-dataforkmd5
                (exec (format {md5 -q "%s"} (last current)))) ; fails if file contains double quote?

            (set 'current-resourceforkmd5
                (exec (format {md5 -q "%s/..namedfork/rsrc"} (string (last current)))))

            (set 'previous-dataforkmd5
                (exec (format {md5 -q "%s"} (last previous))))

            (set 'previous-resourceforkmd5
                (exec (format {md5 -q "%s/..namedfork/rsrc"} (last previous))))

                    (> (+ (first current ) (first previous) 0)) ; not 0
                    (= current-dataforkmd5 previous-dataforkmd5 )
                    (= current-resourceforkmd5 previous-resourceforkmd5)
                    (println (format "     %12d %s" (first previous) (last previous)))
                    (println (format "   = %12d %s" (first current) (last current)))
                    (set-spotlight-comment (last current) (string "duplicate " (last previous)))
    ; remember this one for the next comparison
    (set 'previous current)))

(println "... finished")

Update 2006-01-06

Every time I run this file, I find things that need improving slightly. I noticed recently that the /rsrc technique for looking at resource forks is now deprecated. I should be looking for ‘namedresource’ or something. I will change it one day.

The script also seems to not like a few files. I will investigate this one day. Luckily the script doesn’t do anything to the file system…

Update 2006-01-13

I’ve changed the resource-fork handling, and changed the way the filenames are quoted. The script was having problems with filenames that have single quotes in them. I found a whole bunch of font files that used the possessive apostrophe in their names. By using format with curly brackets, I hope that the quoted filenames can pass through the shell without damage.

Update 2006-04-20

More minor corrections, including calls to (real-path), and a call to SpotLight via osascript. This significantly slows the script down, so comment out the call to (set-spotlight-comment) if speed is important.

December 14, 2005

Running newLISP scripts in BBEdit/TextWrangler

Filed under: newLISP — newlisper @ 09:52

If you’re using BBEdit or TextWrangler on MacOS X, it’s easy to write newLISP scripts that process text windows. Store your scripts in the ~/Library/Application Support/BBEdit/Unix Support/Unix Filters/ folder.

A basic script looks like this:

(dolist (file-name (2 (main-args)))
    (set 'file (open file-name "read"))
    (while (read-line file)
        (println (current-line)))
    (close file))

Notice the 2 in (2 (main-args)): this extracts the name of the file from the list of arguments that BBEdit passes to newLISP. The arguments are as follows:

  1. argument 0: “/usr/bin/newlisp”
  2. argument 1: path to the Unix filter
  3. argument 2: path to BBEdit’s temporary copy of the document

So this script processes argument 2, the temporary version of your current document that BBEdit creates.

As it stands, this script simply replaces each line of the current window with itself. To make it do something more useful, just insert more code:

(dolist (file-name (2 (main-args))) ; 2
    (set 'file (open file-name "read"))
    (while (read-line file)
            (replace {(\d+)}
                (string (add (int $1) 1)) 0 )))
    (close file))

This adds 1 to every number in the text.

By the way, BBEdit/TextWrangler can run Unix filters on the current selection only, not just the whole file.

December 12, 2005

Getting newLISP onto a new Mac

Filed under: newLISP — newlisper @ 21:39

When I got my new Mac PowerBook I was looking forward to do a bit of LISP programming. MacOS X comes with a lot of open-source software pre-installed, including a number of programming and scripting languages, not to mention the Darwin operating system itself. You can write Perl, Ruby, Python and Tcl scripts immediately, but there’s no Lisp available ‘out of the box’. (Emacs is installed, but that’s another story.)

There are four or five versions of Lisp that you can currently run on MacOS X, including CLISP and OpenMCL, as well as some of the commercial Lisp products, such as LispWorks. But I think it would also be a great idea to add newLISP to the default MacOS installation. For one thing, it would allow people to try out some Lisp-style scripting and programming using a fast and well-documented implementation, without being frightened away by the bloated monster that is Common Lisp. It would also show that there’s no reason why scripting and web application building can’t be done just as easily and efficiently using Lisp as with Python or Ruby.

I’m going to write to the head honcho over at Apple’s Darwin division and ask for newLISP to be installed on all new Macs!

December 11, 2005

Posting from newLISP

Filed under: newLISP — newlisper @ 21:49

Not relevant in the archive version

I’m using the post-url function in newLISP to post to this blog. Here’s the function:

(define (make-new-post blog-id blog-username
     blog-password post-title post-body)
    (write-buffer buffer  (string  (xml-parse
    (post-url  "http://plant.blogger.com/api/RPC2"
    (format "
<?xml version=\"1.0\"?>
" blog-username blog-password blog-id
  post-body post-title post-title) "" 10000 ) (+ 1 2 8)))))

and I call it like this:

(make-new-post blog-id blog-username
              blog-password post-title post-body )

Nearly all of this is the XML request code. The xml-parse function reads the answer from Blogger and formats it for my benefit. If you use this code, reformat the XML to remove stray returns and tabs!

December 9, 2005

Choosing a file on MacOS X

Filed under: newLISP — newlisper @ 12:59

For one task, I want to choose a file interactively, using the Mac’s
Open File dialog (which contains SpotLight searching powers too).

The only way to do this at the moment that I can find involves
calling AppleScript, via the osascript system command. This is
slightly harder than easy, because you’ve got to quote the files
correctly if they’re to survive their journey through the Finder, the
shell, and your newLISP code. Luckily newLISP provides us with some
useful tools for protecting both the AppleScript code and the file
names (which can also contain spaces).

(define (choose-file-with-prompt prompt)
     (exec (format {osascript -e 'tell application "Finder"
     set af to choose file with prompt "%s"
     end tell
     set pf to POSIX path of af ' } prompt )))
(set 'f (choose-file-with-prompt "Choose an XML file"))
(println (xml-parse (read-file (first f))))

The basic rules of LISP

Filed under: newLISP — newlisper @ 09:49

LISP is surprisingly simple. There are only a few things you should know to get started.

LISP works with lists of items. Each list is enclosed in parentheses. LISP tries to use the first item of the list as a function or method, unless you tell it not to.

Here’s some LISP code

(+ 2 2)

LISP sees the first item, the + sign, and tries to apply it to the other items in the list. This works out nicely, and gives the answer 4.

The next thing you have to know is how to stop LISP trying to evaluate lists like this. Put a single quote before the list:

'(+ 2 2)

LISP sees the quote and doesn’t apply it to the list, so the result is (+ 2 2), which is a list. This use of quotation marks is similar to written English, where we use quotation marks to indicate words and phrases that we don’t want interpreted in the usual way. LISP doesn’t mind you putting lists inside other lists:

(+ 2 (* 3 2))

The inside list tells LISP to multiply 3 and 2, which simplifies the list to:

(+ 2 6)

and this then evaluates to 8.
So LISP is easy!

Next Page »

Blog at WordPress.com.