Engine DJ 5.0.0 drops playlists on export: investigation and SQL fix

When Engine DJ 5.0.0 came out I did what everyone does: I updated, exported my library to a USB drive, and checked that everything was in place. Everything was: thousands of tracks and all my playlists, ready for the next gig. The bug was already in, it just had not bitten me yet. The Engine DJ community forum started filling with reports of exports that produced drives with every track and zero playlists, and the people affected were mostly those who had exported from an already exported drive. Then it bit me too, the day I used my drive as the source of another export, which is exactly what the merged libraries feature that made me fall in love with Engine is for.

Losing the music would have been better. Tracks are replaceable commodities, while playlists are years of curation, and for a DJ they are the actual work product: the sets, the moods, the transitions and the crates refined gig after gig.

So I spent the following weeks digging into the SQLite databases of my libraries in their various broken states, trying to infer what was wrong with the new 3.0.2 database schema of Engine DJ 5.0.0. This article is what I found, and the one-query fix that came out of it.

How the bug spreads

Since updating to Engine DJ 5.0.0, exporting a library to a USB drive produces a drive with every track on it and zero playlists. The export finishes without any error, the players read the drive normally and the music is all there, but the playlist menu is empty.

The failure propagates. A drive written by the 5.0.0 export still holds your playlists and works on the players and in the desktop app, but if you later use that library as the source of another export, for instance from another computer through the merged libraries feature, the new drive gets no playlists either. Once a library has been through the 5.0.0 export, every export derived from it loses the playlists. Downgrading to 4.x does not help, because 4.x cannot correctly process a library that was upgraded to the new database schema. Tracks and all their associated data export fine; only playlists are lost.

Forensics on m.db

Engine DJ is proprietary (what a stupid choice), so I cannot read its code or contribute in any way other than reverse engineering it, and I’m VERY bad at reverse engineering stuff, so everything that I tried to do was to play with the database files. Luckily enough they were not so stupid to encrypt or obscure the database format, which is an unencrypted collection of SQLite files that simply happen to be overly complicated (overengineering? stratification of ideas over time? no shame in that). The “what” below is verified fact and the “why” is deduction, since the export logic itself is not visible to me.

The library sits in Engine Library/Database2/. Playlist data lives only in the main file m.db; the companion files hm.db, sm.db, and stm.db hold history and streaming data and do not matter here. Two tables matter. Playlist holds the folder tree, with title, parentListId for nesting, nextListId for ordering, and two boolean flags named isPersisted and isExplicitlyExported. PlaylistEntity holds which tracks belong to which playlist.

I compared three databases: a healthy library that exports correctly, a 5.0.0-exported copy of that library which fails to export, and a broken export produced from a library in that failed state. The findings:

1. On broken exports the playlists were never written, which rules out deletion and file corruption. SQLite keeps a per-table insert counter called sqlite_sequence, created on the first insert into a table, and it survives row deletion and database compaction. The broken exports have no counter row for the playlist tables at all, zero reclaimable pages, and a clean integrity check. The export never executed a single playlist insert.
2. The database schema is identical between the healthy and broken databases, both at version 3.0.2. Whatever breaks the export is a difference in data, since the schema did not change.
3. The playlist data in the failed library is fully intact. I checked the tree structure and the track memberships down to the linked-list ordering, and everything is valid, which is why players and the desktop app read the library without complaints. The data is present and readable, and the 5.0.0 export simply does not emit it.
4. The 5.0.0 export zeroes the two flags on the copy it writes. Playlist IDs survive export unchanged, so every source row matches an exported row one to one.
5. Once the flags are gone, the export writes tracks only. The exact internal condition stays hidden in the proprietary code, and the failed library even kept two flagged root folders without recovering, so the condition is not a simple per-playlist filter. What I could say at this point was that the flags were the only meaningful difference left, so I tested whether restoring them restores the export.

The experiment

I built patched copies of the failed library, changed one variable in each, and ran real 5.0.0 exports against them from a Windows machine. Three exports were enough:

The flags alone make the difference between a library that exports and one that does not. The other candidates I had isolated (sync history, playlist timestamps, library identity) turned out to be irrelevant, and I never needed to test them individually.

The fix

Some context on how I got here and what that means for you. I am a Linux user, and all of this research was done going back and forth between my Linux machine and Windows lab machines, because Engine DJ only runs on Windows and macOS. Denon should seriously consider releasing a native Linux client at this point. The fix itself is only a SQL query against the SQLite file, so any SQLite tool on any operating system can run it. I did my patching on Parrot Security with the sqlite3 command and sqlitebrowser. I have not run the Windows procedure end to end yet, since what I describe for Windows is a reconstruction of what I did on Linux. I will try to reproduce the exact steps on a Windows machine, but until I confirm them, treat that part as untested and take the precautions seriously.

The procedure: quit Engine DJ completely and make a backup copy of m.db. Open the file with your SQLite tool and run the query below, then save the changes. The file to edit is the library you export from. On Windows that is C:\Users\<you>\Music\Engine Library\Database2\m.db and on macOS it is ~/Music/Engine Library/Database2/m.db. For a drive library, edit <drive>/Engine Library/Database2/m.db. It works even if you apply the change straight to the database on your USB drive, which makes much more sense, since external drives are where the bug mostly manifests itself.

UPDATE Playlist SET isPersisted = 1, isExplicitlyExported = 1;

Then launch Engine DJ, export again, and check the new drive:

SELECT COUNT(*) FROM Playlist;         -- 0 means the bug is still there, more than 0 means it worked
SELECT COUNT(*) FROM PlaylistEntity;

The query flags every playlist, which is deliberate: it satisfies whatever flag condition the export checks, without needing a healthy reference library to compare against, which most people do not have. The side effect is that every playlist shows as explicitly exported in the software, which was cosmetic in my testing. Seeing so many previously deleted empty playlists popping up again in your collection will bring you back memories :)

If a 5.0.0 export already gave you a drive with no playlists, your source library is affected, so apply the fix to it and export again.

Limits of the fix

• You cannot repair an already broken exported drive in place: the USB drive with no playlists cannot be fixed, only the library that has playlists but cannot export them, because the playlists were never written to that drive. Fix the source library and re-export.
• The fix holds on the source library, because the export strips the flags only on the copy it writes. Every drive written by 5.0.0 is itself poisoned as a future source, so if you ever export from such a drive, run the fix on it first.
• Until Denon patches this, check the playlist count on every export before a gig.

Why this slipped through

Most of us like to export to USB drives and use such USB drives as backups, counting on the old merged libraries feature to kick in and allow old data retrieval. I presume the change in behavior passed silently to the production version because the Engine devs never took that use case into consideration, since Engine already offers a backup feature, but unfortunately not everyone lives with only one computer, only one library and only one source of truth for the latest state of the music collection.

DJ gear should be open source

I found this bug and its fix only because Denon happened to leave the database unencrypted, and nothing guarantees that a future version will not encrypt those files. If that happens, a bug like this one becomes fixable only by the vendor, on the vendor’s schedule, if the vendor believes it exists.

A DJ’s library is work product. The tracks are licensed commodities, but the playlists, the cue points, the loops and the hand-tuned beat grids are years of skilled labor, and today that labor lives inside a format that one vendor controls and one proprietary application can break with a routine update. Clubs and festivals run on this equipment, which makes it professional infrastructure with a closed software layer.

The irony is that these consoles already rely on open source components in their software stack, and the community has been reading their file formats for years without any help or permission. The ingredients for an open DJ platform all exist. What is missing is a vendor willing to open its platform, or a collective project willing to replace it. Until one of those happens, every DJ is one silent update away from finding out how much of their work they actually own.