Large Lisp Modeling
I'm back on my CL kick on again, and since I've been playing with LLMs, I thought I'd see how they fare.
I decided I'd write a program to interface with the firmware on the Beepy. I haven't written any CL in a while, and even longer since I wrote production code.
My first pass at reading in a firmware interface file as a string was this:
(defun read-interface-as-string (interface)
(let ((path (firmware-file interface)))
(if (uiop:file-exists-p path)
(string-right-trim (uiop:read-file-string path))
(error (make-condition 'bad-path :interface interface)))))
I like how this turned out. Consider the definition of
read-interface-as-string. What are some ways to make this
function more robust?
It spit out a long, detailed, and iterated series of options, ulimately ending on the following:
(defun read-interface-as-string (interface)
"Read the contents of a sysfs interface file as a trimmed string.
INTERFACE is a string identifier (e.g., 'sensor') mapped to a file
path. Returns a non-empty string with whitespace trimmed. Signals
BAD-PATH if the file doesn't exist, is a directory, is empty, or
cannot be read."
(unless (stringp interface)
(error 'bad-path :interface interface))
(let* ((path (handler-case (firmware-file interface)
(error (e)
(error 'bad-path :interface interface))))
(truename (or (uiop:truename* path)
(error 'bad-path :interface interface))))
(when (uiop:directory-pathname-p truename)
(error 'bad-path :interface interface))
(handler-case
(with-open-file (stream truename
:direction :input
:element-type 'character
:external-format :utf-8)
(let ((content (string-trim
'(#\Space #\Tab #\Newline
#\Return #\Linefeed #\Page)
(read-line stream nil ""))))
(when (string= content "")
(error 'bad-path :interface interface))
content))
(file-error (e)
(error 'bad-path :interface interface)))))
Additionally, it gave me a bunch of great ideas/considerations, including:
- Mocked testing code to validate the functionality.
- Enhanced filesystem checks (validating it's under /sys/ and has the correct permissions.
- It mentioned caching---not relevant here, but definitely a useful thing to think about.
- Adding some logging. This is something I ran into when I was looking at resurrecting Sinclair (old Lisp blogging software) last night.
As a tool for finding functions (it helped me find a few UIOP and CLHS functions), provide some recommendations for robustness that I can use going forward so I don't need to consult an AI (or wade through a bunch of docs,though there is certainly a time and place for that).
It also rewrote my dumb function to convert a number to a string; I
wrote the dumb version of read-interface-as-string first, then write
a much simplified version of this. I found the advice to disable
*read-eval* useful, which is what led to the above interaction.
(defun read-interface-numeric (path)
"Read a numeric form from the file at PATH, signaling non-numeric-interface if invalid.
Assumes read-interface-as-string returns trimmed file contents."
(let* ((string (read-interface-as-string path))
(form (handler-case
(let ((*read-eval* nil)) ; Disable evaluation
(read-from-string string))
(reader-error ()
(error 'non-numeric-interface
:interface path
:value string)))))
(unless (numberp form)
(error 'non-numeric-interface
:interface path
:value form))
form))
It for sure writes some sketchy code (things that are off, but only subtly so), and in other cases it just made up functions. Still, fun times.