Looking at the short improvisation from a few weeks back, I decided to use a tool to convert audio to MIDI. Why? Well this would be my first step in starting to orchestrate the basic melody of the improvisation and take what was a brief moment in time and grow it into a fully formed musical concept.
Traditionally this would be done by transcription of the melody as well as defining the chord and song structure. This is still the best and most thorough method because human thought is going into each note as well as the relationships between the notes to the structure and theme.
But this is 2025… we can just let computers do all that hard stuff for us right? As much as I would love to have transcription skills of a 19th century music scribe, alas my abilities are highly underdeveloped.
Turning to our robot overlords, I did a quick google search and one of the first tools to come up was:
This is a free to use, browser client side, conversion of audio to midi using Google Magenta. A few minutes of crunching was all it took to convert the audio file into the following MIDI file:
The results are not pleasant to listen to, but the basics of the notes are mostly there, the general rhythm, and the structure. This is a rough equivalent of if I were to go and hammer on a MIDI controller keyboard for a few minutes, but instead it’s gleaned from a (crappy) recording on a real piano.
The next steps would be to start to map this to a BPM, time signature, key signature, and song structure. Fixing bad notes comes next as well as chord harmonization and playback through MIDI instruments to look at timbre of sounds that fit those voicings.
Looking forward to spending more time on this and happy for now to have the rough transcription knocked out in a few minutes.
A few months back during the holiday break, I had an opportunity to reconnect to some old friends on IRC via Libera.chat and my memories of a Common Lisp library for scoring music called Gsharp were revived. Not bad after 22 years.
I attempted to load up the old software, but the graphics library that it’s built upon, McCLIM, has had some real development done in recent years by Daniel Kochmański and team. They’ve been keeping that direct lineage of the Symbolics early and revolutionary graphics interface going and subsequently the old Gsharp code needed too many changes just to half render.
In stepping through that process I had a chance to discuss the current state of Common Lisp as well as Gsharp’s creator’s vision for a new library for music scoring along with some of the changes in Lisp Library support and some big happenings in the big wide world of musical fonts.
I decided it would be fun to try and crack into this, since it combines a few of my passions into a single coding project. First, I needed a way to load fonts into a Common Lisp graphics system and render them to the screen. Here is where Daniel and team’s hard work of building a TrueType font backend to McCLIM came in handy and so I got to work in making a simple proof of concept.
CLIM is a really interesting design based upon the Common Lisp Object System, which is one of the most pleasing object oriented implementations to use. It’s a big and robust system with a very open design that allows for a ton of customization and flexibility. The spec is rather huge and I remember being kind of overwhelmed by it all many years ago.
I’ve since had two decades of professional and amateur coding experience with Common Lisp and coming back to this spec is still a lot to take in. Talking with Daniel, Robert Strandh, and I few others, I was able to cobble the couple lines necessary render a TrueType font out to the screen. Now I just needed a music font.
The game changer in the world of music fonts is the W3C standard called SMuFL, which defines a set of glyphs (plus relationships, ligatures, sets, ranges, attributes, engraving, and registration details) in a standard way, so that fonts can be used across various scoring software. These fonts typically use OpenType font files, which are an extension of TrueType.
Attempting to load OpenType fonts using the ZPB-TTF library didn’t work, so instead I changed tact to converting the OTF files into TTF files using FontForge, which ZPB-TTF could read, and subsequently McCLIM could render.
The problem with converting OTF fonts to TTF fonts is that OpenType allows for the SMuFL metadata to be stored in the font directly. This important information is used to properly render and position the font in a viewpane, but during the conversion to TrueType all of this information was lost.
As much as I would like to extend the ZPB-TTF library into an EAJ-OTF library, it is a non-trival undertaking for a future-future date. Instead I chose to examine the JSON files that accompany the SMuFL standard as well as each compliant font. These JSON files provide all of the metadata typically stored in the OTF font itself for other font types such as SVG, WOFF, etc.
Using the JSON definitions from the SMuFL spec and Bravura the model font that the spec was built around, I was able to build out a library to parse all of the font metadata and load it in such a way that all of the definitions and associations could be made between glyphs. There 13 main phases to loading a SMuFL font, each with a need to pull in data, cross reference and build out the font metadata for general use.
After doing the proof of concept, I dug into the SMuFL spec and started writing code to parse and relate all of the JSON data. The result is a library called Cloveleaf:
Cloveleaf allows for a TTF metadata font to be defined, loaded, and prepped to use by other software. The ultimate goal is to create a cloveleaf backend for McCLIM to allow the TTF fonts to be rendered using the correct positioning, ornamentation, and clarity in order to make a useful, readable scoring application.
The immediate goal for Cloveleaf is to get unit tests written, correct issues in trying to load lesser compliant SMuFL fonts and then look at OpenType support. If you have any comments or suggestions, please leave a note below.
My triple sloths module, which is an amazing source of chaotic modulation, took a dive.
The module initially worked, but the “apathy” portion of the module recently started outputting a constant -12V instead of the ever undulating voltage that I’ve grown accustomed to using in practically every synth patch.
On President’s Day I had a moment, so I visually inspected the board and all my hand soldered SMD connections looked OK. I verified all of the values of the resistors and the position of the ICs. Using the multi-meter I also confirmed all of the connections in the Apathy circuit to ground and the voltage supplies. Everything looked good.
At the end of all of that I wiped down the module to remove all of the left behind solder flux. Plugging it back in it worked!! Great, all done :)
Of course that wasn’t the end of the story, because after adding a few cables it went back to not working. The fact that it was intermittent was an clear sign that it was a non obvious bad solder joint.
Re-flowing the solder with lots of flux was the next step and greatly improved the soldering around the ICs. Another quick wipe down with flux remover and it fired right back up and ran as it should.
Happy ending, nice smooth modulation and back to making music.
From these definitions, I can see that the above functions take two arguments, the first of which is an array of two numbers and the second is an “environment”. The concept of an Environment is defined in uSEQ/src/lisp/Environment.cpp, appears to me to be similar to the concept of lexical scope, and is basically handled by modulisp. As a user the first argument’s two values are what is provided, one a number and the other a phasor.
Looking at the meat of these functions, they perform the following:
FAST: multiplies by the provided factor SLOW: divides by the provided factor OFFSET: adds the provided offset.
The effect of this is to transform the “phasor” by applying the above rule in a new environment linked as a child to the original. This allows for modifications to the parent to ripple down to the child environments.
This does seem to imply that the global BPM is used throughout the system and defining systems with multiple BPMs is not currently not supported. Something I’ll definitely ask about in the Discord server.
Had a few moments this weekend to put together a library to connect to uSEQ. It uses a standard common-lisp implementation (tested with SBCL) and cl-libserial-port to form a serial connection to the uSEQ device (RP2040 based usb device).
I implemented a basic REPL to have a back and forth with the device. uSEQ ships with a much more full featured library called useqedit that I would recommend most people to use. I wrote this library to fold uSEQ into my native coding environment, which is Emacs and SLIME.
The cl-useq library can be found on my github along with a description and examples.
It’s been a loong while since I last posted and I’ve been getting back into coding and music, the intersection of which got me into EE and CompSci way back in the early 2000s.
Common Lisp was the language of CLM and other software synthesis that attracted me at the time with fantasies of studying at Stanford’s CCRMA lab and the LaBRI Institute.
Nowadays there are new options like Incudine and OpusModus, which continue that direction and give me hope that I can continue to experiment with software based composition.
Lately I’ve been spending my days studying and practicing on an old studio piano as well as building a modest Eurorack DIY modular synth setup.
A kit popped up over at Thonk for a “livecoding” module called “uSeq” by Emute Labs, which is connected to the University of Sussex Experimental Music Lab. It uses a Lisp dialect (uLisp) as a means to program the module using their own DSL called ModuLisp. There is even a published paper written that discusses livecoding with this tool.
The kit as sold by Thonk
The kit comes with everything you need to assemble the module minus a soldering iron, solder, and about 1/2 an hour of time. There are two SMD switches, but other than it’s all through hole components. They have a great build guide here and there is also a prebuilt version if you’d prefer to rack it up and go.
Soldered with Kestler no clean 60/40 solder
Installed and testing the blinkenlights.
The soldering went well and everything fired up on first go. The 10 LEDs lit up in circling sequence and it showed up as a USB device on my Gentoo Linux host running realtime Linux Kernel 6.6.36. Here is the output of dmesg:
[646594.147296] usb 1-3: new full-speed USB device number 6 using xhci_hcd [646594.295794] usb 1-3: New USB device found, idVendor=2e8a, idProduct=f00a, bcdDevice= 1.00 [646594.295812] usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [646594.295820] usb 1-3: Product: RP2040 [646594.295825] usb 1-3: Manufacturer: Generic [646594.295830] usb 1-3: SerialNumber: E463B013DB56402B [646594.332211] cdc_acm 1-3:1.0: ttyACM0: USB ACM device [646594.332284] usbcore: registered new interface driver cdc_acm [646594.332291] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
The bolded text “ttyACM0” is the device name for connecting to this new USB connection via the device file /dev/ttyACM0. The issue is that the permissions on that file did not allow my user to open the device:
elliott@deph ~ $ ls -al /dev/ttyACM0 crw-rw---- 1 root dialout 166, 0 Aug 4 04:22 /dev/ttyACM0
The OS is assigning the Rasberry Pi 2040 chip on uSeq as a communications device (like modems) rather than a USB serial device and so it assigns the “dialout” group and root user. If I’d like to connect to the device, I’ll need to either join that group or change permissions. I decided to change the permissions and the best way to do that is by setting up a udev rule. Here is what I added:
The idVendor and idProduct were from dmesg output above, but also can be found by doing a lsusb -vvv to see all the details of the device. I have the “audio” group setup, my user configured to use it, and I changed the permissions to “0660” to ensure read/write access to that group as well as the root user. The udev rules were reloaded and the cable replugged back in to complete the permissions adjustment.
elliott@deph ~ $ ls -al /dev/ttyACM0 crw-rw---- 1 root audio 166, 0 Aug 4 04:22 /dev/ttyACM0
Now that we have a device and it’s attached, we can connect to it via a serial console emulator app. Luckily Emacs has built in methods and it was quick to load serial-term mode and connect to the device with an “9600 baud 8N1” connection rate using line-mode.
All this went flawlessly and I was soon issuing ModuLisp commands to the uSeq module to get reoccurring patterns on the analog outputs. Looking at the code it’s apparent that there is a python interface (useqedit) and I believe a SuperCollider interface here. The easiest method of connecting with uSeq is the webbased interface and it works well.
Over the next few weeks I’d like to go over the current state of the development environment, commands, and other aspects about using this module towards doing some livecoded audio.
This is a tech post and the intended audience is the few people who will bash their heads against a piece of software until it gets installed. I’ve just gone through one of these “sessions”, so I thought I’d document it here for the next person.
Zach Beane created quicklisp, which is a great way to load up common lisp libraries without a headache.
To give an example of this, to load my favorite web programming framework, all I need to do is type the following: (ql:quickload 'weblocks)
It loads everything I need and I can instantly get started working. Very cool! Now after a bit of programming I find that I’d like to customise the storage layer to use a mysql database, but that’s when it gets hairy…
First of all, weblocks’ clsql interface depends on clsql-fluid, which as a project doesn’t package anything… so this doesn’t give quicklisp an opportunity to distribute it. I guess someone could make a package, but I assume Zach would like to have the project maintainers do this instead of himself.
Secondly quicklisp already had installed the clsql library for me and I couldn’t find a command to remove it. So after looking through the documentation and the source I went and deleted things under my quicklisp directory (and ~/.cache) to remove quicklisp’s version of clsql. A remove command in quicklisp would be a nice touch. It would also be nice to get some like functionality for quicklisp in managing software beyond installing & searching. Maybe Zach already has plans for these for when the code goes out of beta…
Thirdly, I fetched clsql-fluid as described on the website and linked it in and loaded it up… but got errors:
"The name NIL does not designate any package."
What is happening here is that clsql-uffi tests that it’s properly loaded by looking for a defined symbol (atol64) to have an associated function. If that function doesn’t exist then we havn’t loaded the foreign libraries and therefore haven’t successfully loaded. The problem with this is that the clsql-uffi package doesn’t exist yet, so we get the error when the test code tries to (find-package #:clsql-uffi).
Since this test is a predicate of sorts I decided that instead of erroring it should just return nil, so I wrapped those probing statements in an: (ignore-errors ...) statement. Once I’d done this it allowed me to get to the next level of errors :)
Fourthly, the syntax used in the uffi calls seems different than what the uffi installed via quicklisp expects. After getting beyond the “nil package” problem above, I encountered this error:
caught ERROR:
; (in macroexpansion of (UFFI:CONVERT-FROM-FOREIGN-STRING CHAR-PTR
:LOCALE ...))
; (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
; error while parsing arguments to DEFMACRO CONVERT-FROM-FOREIGN-STRING:
; unknown keyword: :LOCALE; expected one of :NULL-TERMINATED-P,
:LENGTH
After looking around a bit I got the tip that cffi-uffi-compat might be more compatible than quicklisp’s version of uffi, so I edited clsq.asd, clsql-mysql.asd, clsql-uffi.asd, and uffi/clsql-uffi-package.lisp and edited them to use the uffi compatibility layer from cffi. After all that was done I did another (ql:quickload 'clsql-mysql) and the clsql-fluid version loaded. whew
I hope this helps others stuck in a similar situation and here is the result of a git diff in the root of the clsql-fluid source:
In working with old VW’s you end up looking at a lot of… old VW stuff. I assume this is true of most things you do, you end up looking at historical info to get an idea where to take the next step.
Sometimes this leads you to the most boring series of endless searches that yield nothing, but sometimes you luck out and find what your looking for and can even be pleasantly surprised along the way:
Interestingly enough I arrived at the reason why I was looking in these old brochures in the first place… the hose layout for my 1963 :)
It’s nice when you can get period correct info from a sources that are so inspiring. You can check out more at thesamba.com
