My Video Synth Setup: Part 1 – The Raspberry Pi Cluster
I figured this would be a good place to detail the video synth setups I’ve built up over the past few years. Video synthesis is something that I’ve gotten more and more into lately and I’ve had a lot of fun exploring the options out there, mixing analog and digital processing and generally mangling video like I do with my eurorack (audio) setup.
The core of my setup is a cluster of raspberry pi’s. Specifically, I’m using a bunch of Raspberry Pi 3 B+ since they are the most compatible with the software detailed below. I’ve always had a bunch of pi’s for various projects, usually for my 3D printers, so when I stumbled across a few different projects including Andrei Jay’s amazing VSERPI that use pi’s, I was all in.
I started out with one pi and switching the apps around and then upsized to a rack of 5 pi’s, each running a different app. I wanted a nice self-contained stack of them and found this great ‘case’ by GeeekPi on Amazon. The assembly wiki shows all the different uses and ways to assemble it.
There was a number of bad reviews of the included microSD card extension adaptors (that move the card slot to the back of the Pi beside the USB ports) but once I installed my stack of Pi’s (without the extensions) I quickly realized what a pain it was to change SD cards on each Pi. So I disassembled everything and added the extensions and they’ve worked fine for me.
Many of these projects work by ‘reacting’ with either connected audio or video. Audio varies but I’ve used cheap USB soundcards with no problem but you can also use a hardware DAC mounted to the pi. Video is handled by a cheap USB capture card (yes, it’s just $12 and works exceptionally well) plugged into the USB port for getting video into the pi’s. These capture cards are pretty robust (make sure you have a metal case version!) and I’ve used them for digitizing old VHS tapes, video input for OBS and videogame footage for broadcast television with no issues. They appear as a generic ‘USB Video’ source on Windows or Macs and can be sources for Zoom and just about anywhere you’d use a webcam. If you have older, analog video sources, use an RCA to HDMI adaptor – these can be hit or miss quality-wise since they are usually under $20 and made of plastic but all of the ones I’ve bought have worked just fine. Weirdly, they all seem to use Mini-USB cables for power so don’t lose the one that comes with the unit.
If you spend a little more on the capture card (mine is similar to this one), you can get one with a pass thru HDMI port so you can have a pre-effect video output along with the effected video as separate sources that can be mixed or further manipulated. I use a small 7″ display (like this) as my pre-output monitor and various projectors or larger HDMI monitors for the output. I recently picked up a standalone 15″ HDMI monitor at an auction for $30 that is great for this as well.
Pi #1: R_E_C_U_R
My cluster starts at the top with R_E_C_U_R which has a small 3.5″ touchscreen mounted on the pi and utilizes a cheap USB numeric keypad as it’s controller. Essentially it’s a video sampler.
You load up a thumb drive with video clips and can create playlists on the fly as well as add effects and shaders. I’ve made stickers (with a label maker) for my keypad to make it easier to remember which key does what.
There is also an open source controller you can get a ‘c_l_i_k_r’ kit from the creator, which I did, but I found the keypad easier/more comfortable to use. Still cool though and always happy to support folks that make awesome shit.
I do kind of wish I could flip the screen up or rotate it on the top of the stack as it feels a little vulnerable being right below the rack’s handle. I (quickly) cut a piece of acrylic to be a cover but my mounting options are limited without adding more holes to the base plate the Pi is mounted to. It is a nice compact setup that is perfect for those dark performance halls though.
Pi #2: Video Looper
Next in the stack is a Video Looper. Similar to R_E_C_U_R in that you load up a thumb drive with videos, this app much more basic and simple to use. It just scans the thumb drive (which can be hot swapped) and plays the first video on the drive and then the next and so on. Set and forget if you want a seamless looping video source. You can connect a keyboard and skip videos as well.
This pi is also the power source for the giant cooling fan on the cluster. These video apps tend to push the pi’s so a big cooling source is nice bonus with this rack. The fan connects via GPIO pins on the pi.
Update: I’ve added a USB macropad controller from Adafruit (sold out but you can get one from Digikey). I was using a regular USB keyboard but the macropod is much smaller and programmable for the handful of keys I use to control the video playback – start/stop/skip/exit. Since its keys are backlit and customizable, it’s perfect to use in the dark. I colour coded the action buttons green and the ‘bad’ ones red. The blue ones are just decoration but are duplicates of the skip button so it acts like a buffer for the show ending red buttons.
Side note about the videos I use: Originally, I just hoarded some of my favourite clips from around the internet including scenes from movies, VJ animations, etc. and left them as-is. With a recent live ‘performance’ at an event where I projected my visuals behind some performers, I decided to switch things up a big and processed most, if not all, of the video clips for that show. Over a couple of weeks, I slowly went through my clip library and processed them using a computer software tool called PhotoMosh Pro. It’s a fantastic program that is cross platform and has 60 different effects you can apply (and stack) to an image or video clip up to 4K. I bought the Pro version almost immediately since it’s a lifetime license. What I really like about it is the ability to tweak each of the presets and save them as a new preset. You can also hit the “Mosh” button and it will essentially randomize everything. I keep doing that until I find something I like, save that as a new preset and then tweak it some more. What makes it even more interesting is using it for live visuals (with a computer) as you can map most of the controls within each preset to a bunch of things including various loop waveforms, random, audio or midi reactive states which takes it way up a notch.
You can view a preview video of the kind of output I was planning on using during a recent event:
and here is a very short clip of my POV from the event:
Pi #3: Artificial Life
Artificial Life is a ‘chaotic generative video oscillator unit’ that can take in video content and lets you mangle it using a bunch of different feedback style effects. You can also connect a USB webcam for live video manipulation. For my cluster rack, I picked up a Kano Webcam (the yellow thing in the photos) because it’s a rigid gooseneck USB camera that easily mounts to a Pi and works great for some live video without having to rig up a regular wired webcam that normally would hang off a monitor. They seem to be harder to find nowadays though so good luck!
Pi #4: Spectral Mesh
Spectral Mesh is in the fourth slot and is arguably my favourite app to use in video synthesis. You can manipulate the incoming captured video using a regular keyboard or even better, using a Korg nanoKONTROL2 midi controller that gives you a ton of sliders and knobs to interact with the video. This control allows you to simulate Rhett Etra effects and really mangle the video with oscillators and positioning control.
Here’s a sample output video of Spectral Mesh processing some Video Looper footage where you can see the Rutt/Etra effect on the processed video:
Another example on TikTok processing the output from my Teenage Engineering OPZ running the Scanner1.5 videopak (stay tuned for more on that in an upcoming post):
@retrocacti Working on some stuff for thr “Northern Lights” synth meetup next weekend #videosynthesis ? POSTERITY – Ludwig Göransson
And an Instagram Reel:
Pi #5: Phosphorm
Phosphorm is the final install that simulates an ocilliscope style vector display with real time audio reactive visuals.
I power all this with a powered USB block from Aukey (similar to these on Amazon). It is more than adequate to power all the Pi’s at once if necessary although I rarely have them all running at the same time. It’s also handy for charging anything else that needs to be.
I’ve also been experimenting with some other pi based programs to feed into the video monster. The benefit of this rack setup is that I can easily mix and match the software across multiple pi’s simply by switching SD cards that boot on each of the pi’s. Daisy chaining the output of one into the input of another (using the capture cards) can really give you some unique results.
My latest experiment has been a self-booting Hatari install which is an Atari ST emulator that loads one of my favourite video synth apps from the 90s, Trip-A-Tron by Jeff Minter. It’s a light synth that has a very unique look and is also midi capable (in emulation too!) although I haven’t had a chance to connect anything midi to it yet.
That about wraps it up for the Pi Cluster portion of my video synth gear. Drop a comment if you have any questions about the setup.
Next up will be some eurorack based modules that I also use to feed into the cluster that are sound reactive and customizable.
