This is a tech post and the intended audience is the few people who will bash their heads against a piece of software until it gets installed. I’ve just gone through one of these “sessions”, so I thought I’d document it here for the next person.

Zach Beane created quicklisp, which is a great way to load up common lisp libraries without a headache.

To give an example of this, to load my favorite web programming framework, all I need to do is type the following: (ql:quickload 'weblocks)

It loads everything I need and I can instantly get started working. Very cool! Now after a bit of programming I find that I’d like to customise the storage layer to use a mysql database, but that’s when it gets hairy…

First of all, weblocks’ clsql interface depends on clsql-fluid, which as a project doesn’t package anything… so this doesn’t give quicklisp an opportunity to distribute it. I guess someone could make a package, but I assume Zach would like to have the project maintainers do this instead of himself.

Secondly quicklisp already had installed the clsql library for me and I couldn’t find a command to remove it. So after looking through the documentation and the source I went and deleted things under my quicklisp directory (and ~/.cache) to remove quicklisp’s version of clsql. A remove command in quicklisp would be a nice touch. It would also be nice to get some like functionality for quicklisp in managing software beyond installing & searching. Maybe Zach already has plans for these for when the code goes out of beta…

Thirdly, I fetched clsql-fluid as described on the website and linked it in and loaded it up… but got errors:

"The name NIL does not designate any package."

What is happening here is that clsql-uffi tests that it’s properly loaded by looking for a defined symbol (atol64) to have an associated function. If that function doesn’t exist then we havn’t loaded the foreign libraries and therefore haven’t successfully loaded. The problem with this is that the clsql-uffi package doesn’t exist yet, so we get the error when the test code tries to (find-package #:clsql-uffi).

Since this test is a predicate of sorts I decided that instead of erroring it should just return nil, so I wrapped those probing statements in an: (ignore-errors ...) statement. Once I’d done this it allowed me to get to the next level of errors :)

Fourthly, the syntax used in the uffi calls seems different than what the uffi installed via quicklisp expects. After getting beyond the “nil package” problem above, I encountered this error:

caught ERROR:
; (in macroexpansion of (UFFI:CONVERT-FROM-FOREIGN-STRING CHAR-PTR
:LOCALE ...))
; (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
; error while parsing arguments to DEFMACRO CONVERT-FROM-FOREIGN-STRING:
; unknown keyword: :LOCALE; expected one of :NULL-TERMINATED-P,
:LENGTH


After looking around a bit I got the tip that cffi-uffi-compat might be more compatible than quicklisp’s version of uffi, so I edited clsq.asd, clsql-mysql.asd, clsql-uffi.asd, and uffi/clsql-uffi-package.lisp and edited them to use the uffi compatibility layer from cffi. After all that was done I did another (ql:quickload 'clsql-mysql) and the clsql-fluid version loaded. whew

I hope this helps others stuck in a similar situation and here is the result of a git diff in the root of the clsql-fluid source:

diff --git a/clsql-mysql.asd b/clsql-mysql.asd
index a176f85..1263457 100644
--- a/clsql-mysql.asd
+++ b/clsql-mysql.asd
@@ -20,8 +20,8 @@
 (in-package #:clsql-mysql-system)
 
 (eval-when (:compile-toplevel :load-toplevel :execute)
-  (unless (find-package 'uffi)
-    (asdf:operate 'asdf:load-op 'uffi)))
+  (unless (find-package 'cffi-uffi-compat)
+    (asdf:operate 'asdf:load-op 'cffi-uffi-compat)))
 
 (defvar *library-file-dir* (append (pathname-directory *load-truename*)
                                   (list "db-mysql")))
@@ -48,8 +48,8 @@
   t)
 
 (defmethod operation-done-p ((o load-op) (c clsql-mysql-source-file))
-  (and (symbol-function (intern (symbol-name '#:mysql-get-client-info)
-                               (find-package '#:mysql)))
+  (and (ignore-errors (symbol-function (intern (symbol-name '#:mysql-get-client-info)
+                                              (find-package '#:mysql))))
        t)) 
 
 (defmethod perform ((o compile-op) (c clsql-mysql-source-file))
@@ -80,7 +80,7 @@
   :description "Common Lisp SQL MySQL Driver"
   :long-description "cl-sql-mysql package provides a database driver to the MySQL database system."
 
-  :depends-on (uffi clsql clsql-uffi)
+  :depends-on (cffi-uffi-compat clsql clsql-uffi)
   :components
diff --git a/clsql-mysql.asd b/clsql-mysql.asd
index a176f85..1263457 100644
--- a/clsql-mysql.asd
+++ b/clsql-mysql.asd
@@ -20,8 +20,8 @@
 (in-package #:clsql-mysql-system)
 
 (eval-when (:compile-toplevel :load-toplevel :execute)
-  (unless (find-package 'uffi)
-    (asdf:operate 'asdf:load-op 'uffi)))
+  (unless (find-package 'cffi-uffi-compat)
+    (asdf:operate 'asdf:load-op 'cffi-uffi-compat)))
 
 (defvar *library-file-dir* (append (pathname-directory *load-truename*)
                                   (list "db-mysql")))
@@ -48,8 +48,8 @@
   t)
 
 (defmethod operation-done-p ((o load-op) (c clsql-mysql-source-file))
-  (and (symbol-function (intern (symbol-name '#:mysql-get-client-info)
-                               (find-package '#:mysql)))
+  (and (ignore-errors (symbol-function (intern (symbol-name '#:mysql-get-client-info)
+                                              (find-package '#:mysql))))
        t)) 
 
 (defmethod perform ((o compile-op) (c clsql-mysql-source-file))
@@ -80,7 +80,7 @@
   :description "Common Lisp SQL MySQL Driver"
   :long-description "cl-sql-mysql package provides a database driver to the MySQL database system."
 
-  :depends-on (uffi clsql clsql-uffi)
+  :depends-on (cffi-uffi-compat clsql clsql-uffi)
   :components
   ((:module :db-mysql
            :components
diff --git a/clsql-uffi.asd b/clsql-uffi.asd
index e5a75f6..e78f09e 100644
--- a/clsql-uffi.asd
+++ b/clsql-uffi.asd
@@ -50,8 +50,8 @@
   nil) ;;; library will be loaded by a loader file
 
 (defmethod operation-done-p ((o load-op) (c clsql-uffi-source-file))
-  (and (symbol-function (intern (symbol-name '#:atol64)
-                               (find-package '#:clsql-uffi)))
+  (and (ignore-errors (symbol-function (intern (symbol-name '#:atol64)
+                                              (find-package '#:clsql-uffi))))
        t))
 
 (defmethod perform ((o compile-op) (c clsql-uffi-source-file))
@@ -79,7 +79,7 @@
   :description "Common UFFI Helper functions for Common Lisp SQL Interface Library"
   :long-description "cl-sql-uffi package provides common helper functions using the UFFI for the CLSQL package."
 
-  :depends-on (uffi clsql)
+  :depends-on (cffi-uffi-compat clsql)
 
   :components
   ((:module :uffi
diff --git a/clsql.asd b/clsql.asd
index 35e172a..51da309 100644
--- a/clsql.asd
+++ b/clsql.asd
@@ -26,8 +26,8 @@
 
 ;; need to load uffi for below perform :after method
 (eval-when (:compile-toplevel :load-toplevel :execute)
-  (unless (find-package 'uffi)
-    (asdf:operate 'asdf:load-op 'uffi)))
+  (unless (find-package 'cffi-uffi-compat)
+    (asdf:operate 'asdf:load-op 'cffi-uffi-compat)))
 
 (defsystem clsql
     :name "CLSQL"
diff --git a/uffi/clsql-uffi-package.lisp b/uffi/clsql-uffi-package.lisp
index be5abf5..153ca1f 100644
--- a/uffi/clsql-uffi-package.lisp
+++ b/uffi/clsql-uffi-package.lisp
@@ -19,7 +19,7 @@
 (in-package #:cl-user)
 
 (defpackage #:clsql-uffi
-  (:use #:cl #:uffi)
+  (:use #:cl #:cffi-uffi-compat)
   (:export
    #:find-and-load-foreign-library
    #:canonicalize-type-list