Raspberry Pi Pico Synth_Dexed?

Advanced, Synth Modules 11 Minutes

For a while I’ve wanted to get Synth_Dexed up and running on a Raspberry Pi Pico. This is the library written by Holger Wirtz for use with the Teensy microcontroller, and the core synth engine used for MiniDexed.

This is how I got everything up and running although it is too early to know if this is a worthwhile activity or not!

Note: this is just the very first set of tests to see if anything is even feasible, so don’t expect a playable, working synth! It is just making some hardcoded sounds for the time being.

And the performance isn’t what anyone would call stellar… I can currently manage 4-note polyphony if the sample rate is dropped. That is about it right now!

  • Part 1: Building Synth_Dexed for the Pico.
  • Part 2: Assessing the performance and analysis of the Pico audio library.
  • Part 3: MIDI and some basic usability (finally).
  • Part 4: More MIDI; Bank and Voice loading; SysEx support; and USB-MIDI.
  • Part 5: Details of how to build the hardware.

This is based on ideas and information found from examining the following:

Reference material:

Warning! I strongly recommend using old or second hand equipment for your experiments.  I am not responsible for any damage to expensive instruments!

If you are new to microcontrollers, see the Getting Started pages.

Intro and Hardware Requirements

The core aim is to be able to run a instance of Synth_Dexed on one of the cores of a Raspberry Pi Pico and feed it information that controls how it generates sound and then play the samples back out over some kind of audio interface.

Eventually it should receive this information from MIDI or some kind of built-in interface. Sound output ideally would support an I2S audio DAC – I’ve started with the Pimoroni Pico Audio Pack – but I’d like to add PWM and possibly others.

It remains to be seen if the performance maps a single instance to a core or allows for several instances to be running concurrently and at what level of polyphony.

Other requirements that I’m chewing over:

  • Polyphonic to some level.
  • MIDI.
  • Should allow for flexible separation of interface and synth engine.
  • Should allow for several Picos to act together to provide multiple synth engines.
  • Supports voice bank loading from the Pico’s onboard flash memory “disk”.
  • Use the Synth_Dexed library with the minimum necessary changes – ideally none!

To get started and start experimenting will eventually requires the following hardware:

  • Raspberry Pi Pico.
  • Means of getting audio out – initially its only doing I2S, but PWM could be added.
  • Eventually some means of getting control signals in (USB or serial MIDI or hard-coded).

The following GPIO pins are being used:

  • I2S: I2S_DATA = GP9; I2S_CLOCK = GP10
  • Debug UART: TX = GP0; RX = GP1
  • MIDI (eventually): TX = GP4; RX = GP5

Note that the Pimoroni Audio Pack also makes use of GP22 as a mute switch, but I’m not using that.

The Environment

I set up the Raspberry Pi C/C++ SDK according to my previous set of instructions from here: Getting Started with the Raspberry Pi Pico C/C++ SDK and TinyUSB MIDI.

Once again I’m doing all this in the Ubuntu virtual machine.

The created a picodexed project with the following structure:

picodexed
    +--- build
    +--- cmsis
    +--- src
    +--- synth_dexed 

I’m not going through the whole discovery process of how I got to this point, but here are a few notes of things I learned on the way in case I need to refer back to them!

CMSIS is required to support some of the ARM DSP functions used by Synth_Dexed, but not all of it is necessary. After hunting around for how to build this into the Raspberry Pi Pico (the SDK only has some very basic interface definitions, not the whole proper thing) I eventually stumbled across Chris Hockuba’s Raspberry Pi Pico CMSIS – cmsis-pi-pico – on Gitlab so I cloned that into my cmsis area and used that.

A really useful clue as to what is required to get Synth_Dexed running on a different architecture to the original Teensy can be found from examining the following files created as part of the MiniDexed discussion:

In terms of approach, I basically created the most basic main.cpp I could and created CMakeLists.txt files for the main application and the Synth_Dexed library and then just kept trying to build it to see what was still broken.

I had a frustrating diversion for a while, not noticing that I’d created a main.c and not a main.cpp and that when it included dexed.h from Synth_Dexed it couldn’t find <cstdlib> – the C++ standard library… I spent ages trying to work out why my compiler couldn’t find its own libraries…

One thing that appeared to be missing totally though, was a definition of boolean. Now I could have gone through Synth_Dexed changing boolean to bool, but in the end implemented a “filler” or “wrapper” header file with everything extra that Synth_Dexed needed but was missing – dexed_if_common.h – and stuck a typedef bool boolean; in there.

So far, the only change I’ve had to make to the library itself is to add the following line into Synth_Dexed/src/dexed.h and then make sure the path for include files can find it during compilation:

#include "dexed_if_common.h"

I still haven’t quite got to grips with the whole cmake infrastructure required for building Raspberry Pi Pico projects, so the CMakeLists.txt files I’ve created are almost certainly sub-optimal at present. I still need to get my head around PRIVATE, PUBLIC, INTERFACE qualifiers for example and probably have more directories defined for INCLUDE paths than is strictly necessary.

But it builds and everything I have so far can be found on GitHub. This includes a simple script to take the basic repository and add in CMSIS and Synth_Dexed and hack in that single change to dexed.h. This means that to build what I have so far requires the following:

  1. Install the Pico C/C++ SDK, toolchain and libraries (as I described here) including setting PICO_SDK_PATH to the location of the Pico SDK installation.
  2. Clone the picodexed repository.
  3. Run the getsubmod.sh script (only run this once for a fresh repository).
  4. Then build:
kevin@ubuntu:~/src/picodexed$ cd build
kevin@ubuntu:~/src/picodexed/build$ cmake ..
kevin@ubuntu:~/src/picodexed/build$ make
kevin@ubuntu:~/src/picodexed/build$

Adding Audio

The Pico has no in-built audio capability, but there is an audio library that can be found in the “pico_extras” repository that supports I2S audio interfaces or PWM audio output on GPIO pins, both implementing using the Pico’s PIO subsystem.

There is an example in the “pico_playground” that shows how to output a sine wave and Pimoroni also have a sample mini-synth application for their I2S Pico Audio Pack.

Links:

In order to build for use with the pico_audio library, the pico_extras repository needs to be cloned into the source area and the location specified using PICO_EXTRAS_PATH.

There is another cmake file that can then be copied into your own project area to include the library.

cp $PICO_EXTRAS_PATH/external/pico_extras_import.cmake .

And then it can be included in the project’s CMakeLists.txt file, pico_audio_i2s can be added to the list of link libraries, and a compilation definition added to define USE_AUDIO_I2S.

include(pico_extras_import.cmake)

target_link_libraries(picodexed PUBLIC synth_dexed pico_stdlib tinyusb_device tinyusb_board pico_audio_i2s)

target_compile_definitions(picodexed PRIVATE
	USE_AUDIO_I2S=1
)

The actual examples for audio output are quite complicated and so far the documentation I’ve found for the pico_audio libraries is pretty minimal.

The Pimoroni build has abstracted most of the functionality out into a separate audio.hpp file with the following interface:

// init_audio initialises the audio library and returns a pointer
// to an object representing the audio buffers.
struct audio_buffer_pool *init_audio(uint32_t sample_rate, uint8_t pin_data, uint8_t pin_bclk, uint8_t pio_sm=0, uint8_t dma_ch=0);

// update_buffer will fill the audio buffer with samples returned
// by the provided callback function.
void update_buffer(struct audio_buffer_pool *ap, buffer_callback cb);

The callback function that is meant to provide samples to the audio subsystem has the following definition:

int16_t getNextSample (void);

Dexed has a getSamples function that can fill a buffer with the next set of samples in either float or int16_t format, but the callback function only returns a single sample at a time.

Rather than attempt to make that fit (I did start off that way) I wrote my own “update_buffer” function to fill the Dexed buffer:

void fillSampleBuffer(struct audio_buffer_pool *ap) {
  struct audio_buffer *buffer = take_audio_buffer(ap, true);
  int16_t *samples = (int16_t *) buffer->buffer->bytes;
  dexed.getSamples(samples, buffer->max_sample_count);
  buffer->sample_count = buffer->max_sample_count;
  give_audio_buffer(ap, buffer);
}

Once slight complication is that the Dexed getSamples function is a protected function. To interface into Dexed I’ve thus created a Dexed Adaptor based on the MiniDexed dexedadaptor. I don’t know how many of these functions will really need to be a wrapper around Dexed, but the capability is there if required.

Typically with embedded audio applications, there is a regular “internal buffer out to audio hardware” layer that has to run at the sample rate and be reliably regular with its timing. This would usually be driven from a timer interrupt or similar. Then there will be a non-timing-critical part of code responsible for filling the buffer itself with samples to be played.

The Pico’s audio library is quite different. Unfortunately, I can’t quite see with the combination of complex C buffer management code, use of the Pico’s DMA and use of the PIO hardware for the I2S protocol itself, quite what the timing expectations of the pico_audio library are and how they should be split up, but from what I can gather, the basic idea is as follows:

  • PIO is automatically pulling data from a set of buffers to send over I2S.
  • DMA is providing that data somehow at a rate independent of the CPU, but maybe driven by interrupts linked to data requests and the PIO somehow.
  • All I have to do is keep the buffer that can be accessed by the DMA fill of audio samples and magic “just happens”.

So for the time being, I’m just calling my update audio equivalent function in a relatively fast loop.

Unfortunately my first attempts all resulting in some kind of horrid growl. But it does turn on and off when the Dexed.keydown and Dexed.keyup calls are made so I was making progress!

Things that eventually fixed that:

  • I updated the init_audio function to allow me to pass in the sample buffer size and chose a buffer size of 128 and then ensured this was set to the same in Dexed itself.
  • I saw the sine example was using a sample rate of 24000 for I2S so I used that sample rate here too.
  • The thing that eventually fixed everything was realising that Dexed is returning a mono stream of data but the default I2S interface is assuming a stereo set of samples. Setting the following in CMakeLists.txt told the I2S library to use mono:
target_compile_definitions(picodexed PRIVATE
    PICO_AUDIO_I2S_MONO_OUTPUT=1
    PICO_AUDIO_I2S_MONO_INPUT=1
    USE_AUDIO_I2S=1
)

The sine library seems to get away with only defining MONO_OUTPUT, but I was getting compilation errors, so I set MONO_INPUT too, even though I’m not using I2S input.

At this point I was getting a nice note playing via Dexed, so I grabbed the voice parameters for Brass 1 out of MiniDexed and dropped them into an array here and with a call to Dexed.loadVoiceParameters, I have a Raspberry Pi Pico playing the Dexed Brass 1 sound!

In terms of sample rates and polyphony, well, things are a little basic! I can currently achieve the following:

  • 2-note polyphony at 44100.
  • 4-note polyphony at 24000.
  • 6-note polyphony, possibly if you squint at it and ask it really nicely, at 12000.

So it isn’t going to win any prizes for quality or quantity, but this is a point that I feel it’s now worth pushing this post out and sticking the progress up on GitHub.

Next, I want some simple MIDI control to act as a real synth. Other things on the “todo” list:

  • See what scope there is for optimisation to increase the polyphony ideally with 44.1kHz.
  • Specifically look into how it is currently doing the floating point maths.
  • Investigate the loading of the core to see how the performance is looking more generally.
  • Get some additional voices in there!

Removing Dependency on CMSIS

It turns out that only a few functions are required from the CMSIS/DSP area, so in order to examining them more closely I created my own version of arm_math.h with a corresponding arm_math.c containing the following functions:

void arm_float_to_q15()

void arm_fill_f32()
void arm_sub_f32()

void arm_scale_f32()
void arm_offset_f32()
void arm_mult_f32()
void arm_biquad_cascade_df1_f32()
void arm_biquad_cascade_df1_init_f32()

with any additional definitions they require. Turns out it isn’t very many. The only thing left is to mirror how Synth_Dexed builds for the Teensy and ensure that the compressor isn’t included. This requires some conditional compilation in dexed.h, dexed.cpp and compressor.h.

There doesn’t appear to be a single obvious way to spot a build for a Raspberry Pi Pico though, but there is a definition of RASPBERRYPI_PICO in <board/pico.h> so I used that.

Find it on GitHub

The current state of progress can be found on GitHub here: https://github.com/diyelectromusic/picodexed

Closing Thoughts

I first wanted to do this when I first installed the Pico SDK back in August 2022! Yes it really has taken me that long to even attempt it. Mostly because my C is still quite rusty; my C++ is very much “learn as you go”; I’ve not worked with a CMSIS ARM embedded system in detail before; and the cmake infrastructure for a Raspberry Pi Pico seems so very complicated for what it is.

But you’ve got to start somewhere as they say, and I learn best from having a reason to do something. So even if this doesn’t amount to anything, it is finally making me learn about all these things in a useful way that will hopefully end up with something I’ll have some fun playing with.

The big limitation might be floating point maths. But the Pico does have some built-in fast floating point routines. I don’t know if any of these are enabled and running at the moment. I suspect the CMSIS DSP code is probably doing a “soft” floating point, so that is something to look into. But if it is already using the faster routines, then it may well keep the polyphony so low that it isn’t particularly practical to use. To be continued…

But once again, my renewed respect to all the above-mentioned people who have essentially already done most of the hard work for projects like this that allow someone like me to bumble along and join the various bits together!

By the way, seeing as MiniDexed exists for the Raspberry Pi and MicroDexed exists for the Teensy, PicoDexed seemed the obvious choice!

Kevin

Published by Kevin

Just another soul meandering my way around the Internet ...

Published

Leave a comment