The Database

Before BpmDj can use your songs, they have to be analyzed. The analysis process takes a jolly good time (about 3 times faster than real-time on a test tablet. If you have 240 minutes worth of music, then the analysis will take about 80 minutes). During analysis, meta-data is written at the same place where the mp3 is stored. E.g if you have an MP3 in /mnt/sdcard/Music/BigFatTechno.mp3, then BpmDj will generate a new file /mnt/sdcard/Music/BigFatTechno.mp3.bpmdj1. The size of the meta-data is about 200k per song, which means that you should always leave some room on the sdcard. Don't fill it up entirely.

The meta-data associated with each MP3 contains a key that uniquely identifies the mp3. That means that whenever you delete the .bpmdj1 meta-data, any mix that might refer to that song will no longer know what song you are talking about. Therefore, once the analysis is performed it is crucial to keep the .bpmdj1 files.

When you browse your song collection using the file system browser, each encoutnered mp3 that was not yet analyzed will be analyzed automatically.

When a song is queued for analysis it is drawn in yellow. If it could not be analyzed it will be drawn red. Hover with your mouse over the song to see the reason why.

If you want to analyze an entire tree at the same time, then go to menu Library|Scan Tree

Nevertheless, for indexing purposes, all that data is imported into a database (stored in the flow-13 subdirectory). There are some commandline options available to check the health of your BpmDj databse.

How many MP3 files do we know ?

Which beatgraphs does it know ? Are there beatgraphs without mp3 file ? Are there mp3 files without beatgraph file and so on. To list these start bpmdj as follows:

  java -jar bpmdj.jar --check-file-tables

An example output is (the numbers between parantheses are remote files):

werner@bulldozer:/I/home/werner$ bpmdj --check-file-tables
BpmDj v4.7.9
(c) 2000-2017 Werner Van Belle - all rights reserved
  [db] mp3 files: 155400 (78041)
  [db] beatgraph files: 154830 (78041)
  [db] other files: 910
  [db] usable tracks: 152323 (77770)
  [db] unusable tracks:
  [db] 1. /E/SomeExtraSpace/Acer-October-2014/.medieval_software/.BlueFTP_temp/blueftp_view_-1681246623
  [db] mp3's marked as 'do not analyze': 829
  [db] mp3's that no longer exist on disk: 2239
  [db] bg's that don't exist on disk: 2456
  [db] unusable bg's because mp3 is missing: 51
  [db] unusable mp3's because bg is missing: 9

In this case, for some reason some medieval non-track ended up in the database. Not much of a problem. However, if you see a lot of unusable tracks, you can trigger an inspection of them by adding the option --trigger to the commandline.

How are the nn-vectors doing ?

To know that, use the --check-nn-tables option. This option is slightly slower, because it will read all known vectors (~500M). An example output is:

werner@bulldozer:/I/home/werner$ bpmdj --check-nn-tables
BpmDj v4.7.9
(c) 2000-2017 Werner Van Belle - all rights reserved
  [db] normal nnvectors: 224658
  [db] empty nnvectors: 168
  [db] nn vectors without usable song: 543
  [db] nn vectors with usable song: 221873
  [db] nn vectors with multiple usable songs: 2410
  [db] usable songs without nnvector: 0 (0)
  [db] usable songs with empty nnvector: 0 (0)
  [db] usable songs with normal nnvector: 152323 (77770)

Compressing your database

The database does store various versions of the same object. That is both for performance reasons (all data is put out in the same pages), as well as consistency reasons (we can go abck to an old state if the current one is incomplete). That does however mean that the space on disk might be somewhat larger than necessary.

To clean and compact the database start BpmDj with --gc

  java -jar bpmdj.jar --gc

This will take a while, but afterwards your database folder (flow-13) folder should be somewhat smaller. Also, if your database was corrupt somehow then this process will gloriously screw up your database even further. From that perspective it is a good database check as well.