About

In October 2000 I decided to organize a psytrance party. A couple of months later, the party was nearing and I had a bit of a problem that, although I would be DJ-ing, I had not much skill in doing so. Up to that point I had always been using winamp to throw two songs together and hope the collision between them would be somewhat satisfactory. After a while one becomes quite good at estimating how long chaos can last before it becomes disturbing. Nevertheless... That approach was unsatisfactory, and the problem of tempo detection, had been growing in my mind for some time. I did some estimates on how accurate tempo should be measured to keep two tracks in sync for a number of bars and figured out that the answer was: much more accurate than any tap based BPM counter will tell you.

Aha-erlebnis: In December 2000 I realized that auto-differencing was a good strategy to measure the tempo and small tests under Linux made clear that it would indeed work. The first command line bpm counter and music player was created. As well as the first beatgraph. Just imagine going to a party and typing commands at a Linux prompt to mix two tracks. Those were the 'good old days'...

Somewhat later I started to extend the command line tools with a selector that would allow me to spawn player processes by choosing a song and clicking on it (imagine that, a user interface !). Around that time I also started to realize that once the tempo of a track is known, a rhythmical pattern can be extracted, which can then be used to measure similarity between tracks. I also figured that mixing music required that the frequency bands were at least somewhat compatible, so I added a spectrum analyzer. Then I believed measuring energy intensity histograms could be useful as well, and added those. During this time I learned a lot from Julius Orion Smith his pages on signal processing, and to be honest, I didn't understand much of it.

Along the way I also wrote a paper on tempo detection and submitted it to a number of journals. None of them would have it. One even said that it was 'too readable' for their target audience. In hindsight that was the best thing that could happen because it forced me to publish my research on Internet, without going through journals that would almost certainly bury the article behind a paywall. Since its online publication around 2003, the article has received quite a lot of attention. Much more so than any other research I conducted in official capacity.

In 2003, BpmDj was usable both as a) a test environment and b) as a robust live DJ-ing tool. Of course, because it spawned different processes for each song, two stereo output were used. And those stereo outputs had to be crossfaded using an external mixing desk. To make beat matching possible, one had to nudge the incoming song backward and forward with the + and - keys. Or set and use a cue as appropriate. Bottom line was that neither installing it nor using it was for the faint hearted.

End 2003, I obtained my PhD on distributed and concurrent systems. In 2004, I started working at Norut IT to help them expand their bioinformatics activities. At Norut there was an earth observation group that provided signal processing expertise to both NASA and ESA. And those people really wanted to see some of their algorithms used in a different setting. Needles to say, I took that opportunity with both hands. Functioning as a bridge between biologists and signal processing experts allowed me to quickly learn a thing or two about biology and signal processing. During this time I gave talks about BpmDj at various universities and music colleges.

Linux started to get some traction and things started to go well with my little project.

In 2005, BpmDj was nominated in the multimedia/audio category at Trophee Libres (a European award for open source software), but since I didn't know what 'nominated' meant and because the email was in broken English, I ignored it. As a consolation prize, I later became jury member for Trophee Libres. I like how the french have such a free spirit regarding musical software. They are not bound by preconceptions on how things should be, nor do they expect a certain label.

In 2006, an article on BpmDj appeared in Linux+ magazine. A small year later a similar article was printed in some Dutch magazine.

In October 2006, I played as DJ at the northern Norwegian Insomnia Festival. A very entertaining event. As an experiment, I played all songs at 85% of their normal speed. And of course, every time I saw a camera zipping by I would duck behind the mixing desk. I was too shy.

In the meantime I still worked as bioinformatician at Norut IT, but that was coming to an end. The harsh reality of politics at universities and an incompetent technology transfer office (TTO Nord) forced me to resign my job. The idiots wanted to patent 'correlation' and no matter how I tried to explain them that it was a waste of money, they just thought I it would be a good idea to patent that.

2007, a party at the Kultuurkaffee in Brussels...

2007->2008, BpmDj and I played at a private party at a castle in France. Very nice setting. Great atmosphere.

In 2008, I started working as a bioinformatician/senior research associate at ETH Zürich. Interesting work, but the science routine started to set in. I knew what to do, how to do it and when to do it. I knew how to explain things to biologists and I knew how to understand what they were trying to get out of the data. Not that the work I was doing was easy or routine. It was all quite advanced stuff, but none of it really interested me. I was more interested in audio. A year later I left ETH Zürich.

In 2009, Bernard Fortz helped to advance BpmDj by adding JACK and Midi support to it. During this time I also upgraded BpmDj from QT3 to QT4. That was 3 months of fulltime work. After the upgrade I shelved BpmDj for a year. I had absolutely no interest anymore in the project.

Around this time, I started to investigate the K nearest neighbors search problem and had some serious problems wrapping my head around it. Why was it so difficult ? Why did I have to look at every single value to find the k nearest neighbors. Why did I need to access 300 Mb just to know the first 10 most similar songs ? Why was that ? Spending 6 months banging your head against a mathematical problem is no fun, certainly not if you do not intuitively understand why things are as they are. Online I read a lot about the 'curse of dimensionality' and that 'distances have no meaning anymore' and that the question might be flawed. I found that each paper I read on the topic, was either deluded or sugarcoated their not-so-bright results. After a while I stopped reading articles about it.

In 2010, the University of Dortmund showed interest in my work and wondered whether I could make a tempo detector work in 95% of the cases. Up to that point, the BpmDj counter was 'sufficient' but would often report tempo harmonics: E.g: assuming 3 or 5 beats per bar. I accepted the challenge, spend two months looking at the problem and found a solution to deal with it. Later, I presented this work at the Chaos Communication Camp in Berlin (a hackerscamp), and found that this type of events produces a much more interested audience that the typical academic track.

In 2011, Andre Michelle contacted me. He needed a solution to measure the tempo of music for a Nokia mobile app. I tried to help him with this but in the end, the phone turned out to be too slow. After this small cooperation, I started working for Audiotool. During 15 months I created a variety of mastering tools, filter banks, helped creating a synthesizer (the Audiotool Heisenberg FM Synth), deepened my understanding of nearest neighbor detection and really enjoyed working there.

End 2012, the company went broke and fired everybody. Allthough, somewhat later they wanted to hire me again, I never received a good letter of recommendation. Therefore it was, from a game-theoretical point of view, impossible to continue working with them.

Somewhere in 2012, the independendent TedX bricklane event referred to BpmDj in a track on the future of DJ-ing.

2013. No job, no letter of recommendation. No minimum wages. Bad economy. Banks on a plundering spray...

I started to think about a small business plan on how I could provide audio analysis to big Internet audio platforms. However, it quickly turned out that that would be such a large project that I first wanted to test whether I could create a small project and bring that to live. Therefore I decided to recreate BpmDj from scratch.

As a targetplatform Android was choosen. While reimplementing it, I removed a lot of unnecessary features and introduced something I had wanted for a long time: overlapping beatgraphs. By visually overlaying the beatgraphs, and choosen the colors carefully one could - to a certain extent - mix visually. The colors were initially choosen for maximal contrast. A loudness analysis was added to each frequency band as to make the beatgraphs feel more 'solid'. The entire tempo analysis module was redone and turned out to be a lot faster and more accurate than the old algorithms.

I also developed a method to access only about ~33% of the data to find the k nearest neighbors. The technique is based on the realisation that the most significant bit in an integer is equally as powerfull as all other bits combined. When preparing a talk for OHM2013 (a hackers camp in the Netherlands), where I would announce my great discovery, I found that a Swiss guy, called Weber, had already invented a very similar technique in 1998. Basically reducing my great discovery to a brief tutorial. Lucikly I require 20% less datastorage than him, so there was still something to be said...

A last advance I made had to do with the creation of a weight matrix that defines a distance metric. By training a test set of music (which songs are closer to each other), I was able to create a metric that should resembles our human perception somewhat better. Hopefully this improvement will live up to my expectations.

In 2015, we improved BpmDj with two aspects. The first was a time stretcher (Zathras-1). The second was that we ported BpmDj to desktop again. To do that we first implemented our own database backend (FlowDb). Once the datamodel was unified, we were able to reimplement the user interface in JavaFx.

As time progresses we might add

If you think it worthwhile to support this project, you can make a donation

Donate >