Syntax matters

My fellow newLISP blogger Kazimir observes in his poll analysis that one of the main reasons why people don’t use Lisp is the syntax:

Lisp syntax is the most important reason for majority of people who do not use Lisp.

The thing I don’t understand, though, is what exactly is wrong with Lisp syntax that makes people avoid the language. Perhaps that’s another research opportunity.

Here’s a simple scripting problem. Given a text file containing a series of random paragraphs, separated by percent signs, sort them so that they are ordered according to the first significant word in each paragraph. That is, words such as “a” and “the” shouldn’t affect the sort order.

Here’s one solution:

#!/usr/bin/perl -w

my %sort_buckets;
my %exclusions;

my $file_to_sort = '/path/to/file_to_sort';
my $sorted_file = '/path/to/sorted_file';

while () {
    chomp;
    $exclusions{$_}++;
}

open $in, "<", $file_to_sort
    or die "Can't open file: $!";

$/ = "%\n";

while () {
    my $line = $_;
    my @words = split " " => $line;
    my $sort_key = '';
    for (0..$#words) {
        if ($exclusions{lc($words[$_])}) {
            next;
        } else {
            $sort_key = join "" => map { lc($_) } @words[$_..$#words];
            last;
        }
    }
    $sort_buckets{$sort_key} = $line;
}

close $in;

open $out, ">", $sorted_file
    or die "Can't open file: $!";

foreach (sort keys %sort_buckets) {
    print $out $sort_buckets{$_};
}

close $out;

__END__
a
the
this
that
you
when
is
may
be
if
and

I’m not sure I know what’s going on there, but it’s a typical practical (and probably quick) solution from a Perler. For comparison, here is a newLISP version:

#!/usr/bin/env newlisp

(set 'common-words
  '("a" "the" "this" "that" "you" "when" "is" "may" "be" "if" "and"))

(define (remove-common text)
    (difference
       (find-all {[a-zA-Z]+} (lower-case text))
        common-words))

(define (compare text1 text2)
    (< (remove-common text1)
       (remove-common text2)))

(write-file {/path/to/sorted-file.txt}
    (join (sort
             (parse (read-file {/path/to/file_to_sort.txt}) "%")
             compare)
     "%"))

It’s not clear to me why this syntax is considered unappealing, or indeed how it could be improved. I’ve written about this before, so obviously I haven’t sought hard enough for an answer.

Can we measure some aspect of these two files? How about looking at the use of alphabetic characters and parentheses. For the Perl example:

Parentheses: (7 7)
Braces: (11 11)
Brackets: (2 2)
Alphabetic: 403
Non-alphabetic: 642
ratio 0.6277258567

And the newLISP example:

Parentheses: (17 17)
Braces: (3 3)
Brackets: (1 1)
Alphabetic: 256
Non-alphabetic: 168
ratio 1.523809524

The preponderance of the Lisp parentheses is apparent. But otherwise, the Lisp example is plainly much more alphabetic, more readable, more civilized. It’s just possible that the non-mathematical syntax, and the lack of familiar and friendly “x = x + 3” forms (as learned by schoolchildren from 10 years and upwards) are enough to disarm and perplex the non-Lisper.