Nokia music library fix bash script

In my last post, I outlined a fix to a problem with Nokia E-series phones in which the music library fails to refresh if music files with certain properties are present.  I also mentioned:

I have saved a bash script in the root of my memory card which I can run at any time if I accidentally transfer new "bad" mp3s to my phone in the future.

Below are instructions to create and use this script.

First, create a text file in your Linux distribution - in Ubuntu, this simply involves opening gedit from the Applications menu or the command line.

Next, paste the text below into the empty file:

#!/bin/bash

# This script removes POPM frames from mp3 files so the Nokia music library does not choke on them.
# After the files are processed, the music library files are removed from the phone to force a refresh.
# This script should be placed in root of microSD card and run from there (i.e. when it is mounted as a filesystem).
# See http://e63tips.blogspot.com/2010/05/solution-to-music-library-does-not.html for more information.

if dpkg -s python-mutagen 2>/dev/null 1>/dev/null; then
    echo "Package python-mutagen is installed."
else
    echo "Installing python-mutagen..."
    gksudo apt-get install python-mutagen
    echo "Installation complete!"
fi
echo "Removing POPM frames from mp3 files in Music folder"
find Music/ -iname '*.mp3' -iname *.mp3 -exec mid3v2 --delete-frames=POPM '{}' \; &&
rm Private/101FFC31/mpxv1.mpd && rm Private/101ffca9/harvesterdb.dat
exit

Don't forget to replace /Music with the path to the top-level folder on your phone.  Now mount your phone via USB (or mount the microSD card using a card reader) and save the file to the root directory of the mounted filesystem as, for example, music_fix.sh.

Ensure that you make the script executable.  In Ubuntu, the an easy way to do this is to right-click on the saved file in Nautilus, choose Properties, click on the Permissions tab, and tick the checkbox next to the "Allow executing file as program option."

Now, if the music library refresh problem ever recurs, you can simply mount the phone or memory card and run this script.  In Ubuntu, you can do this by double-clicking on the file in Nautilus and selecting "Run in Terminal."

Solution to Music Library Does Not Refresh Problem

(OK, technically this is a workaround, not a solution - but that's just semantics, right?)

The problem

Users of Nokia's E63, E71 and other E-series phones have been complaining for years on the Nokia Support Discussions site about a frustrating bug in which the music library does not recognize new music files that have been added to the phones memory (or more typically, the memory card).  Typically, the first time the music player is launched after the files have been added, it attempts to refresh but hangs after adding only some (or in other cases, none) of the songs to the library.  These music files appear in the file browser, and can even be played from there, but are not added to the library.

Many responses in the Nokia forums suggest solutions that do not make intuitive sense, or are quite drastic, such as performing a software reset of the phone (which erases all data); defragmenting or reformatting the memory card; or transcoding all files to a different format (such as from mp3 to aac.  While a few users may have success with these solutions, I have found them either ineffective or impractical on my handset.

Many users have also figured out how to force the phone to rebuild the music library by deleting key database files from the memory card.  This is useful after the root cause of the problem has been solved, but is not a solution in itself.

Root cause

So what is the root cause?

A few more savvy users have correctly identified that this problem is associated with select "bad" mp3 files that the player doesn't seem to "like".  However, the workaround they have offered - identifying these files through painstaking trial and error and delete them from the phone, or re-encode them - is not acceptable to me.  I want a permanent fix!  The good news?  After isolating some of these "bad" files in my music collection, and analyzing them, I think I finally found one!

Metadata describing mp3 files - e.g. the artist, album, song title, track number, composer, etc. - is stored within the file in ID3 tags.  While the Nokia E-series music library appears to have no problems processing these common fields, I have discovered that it chokes on files with POPM frames.  These frames are added to mp3s (and possibly other formats which use ID3 tags) by some media player applications, typically to store ratings information entered by the user (e.g. "5 stars").  The Nokia player appears to add new music files to the library database in batches of ten files.  If any one of the 10 files has a POPM frame, none of the files in the batch will be added to the library and scanning will hang, requiring a manual stop.

Solution - Ubuntu

It's simple - find software that can remove POPM frames from your music library! 

If you use Ubuntu (or probably any other modern Linux distribution), the command line mid3v2 package can do this for you with minimum fuss.  First, open a terminal window and install the package using the following command:

sudo apt-get install mid3v2

EDIT: After upgrading to Ubuntu 10.04 (Lucid Lynx), I discovered mid3v2 is no longer available in the repository as a separate package - it is now part of python-mutagen, which can be installed using the following command: sudo apt-get install python-mutagen

Then simply run the following command to strip all of your mp3 files of their POPM frames:

find /path/to/music/collection/ -iname *.mp3 -exec mid3v2 --delete-frames=POPM '{}' \;

(The path to your music collection could be on your PC - e.g. ~/Music - or on your phone - e.g. /media/microsd).

EDIT: Commenter Thameera reports that he had to use a slightly different syntax than that of the command above.

If you sanitize the files on your PC, then you can simply transfer them to your phone and you should be able to refresh the music library and see all of them.  If the library refuses to refresh, try exiting the Nokia music player, connecting your phone to the PC via USB as a mass storage device, or connecting the memory card, and deleting the following files:

• Private/101FFC31/mpxv1.mpd
• Private/101ffca9/harvesterdb.dat

(These are E63 file locations - they may vary across other E-series devices).

This will force the music library to rebuild itself.  You may find that it appears to hang once again, but if you stop it and wait a few seconds, it will have added some files, and when you then manually refresh the library you will be able to see its progress.

Because I would never be able to memorize this command, I have saved a bash script in the root of my memory card which I can run at any time if I accidentally transfer new "bad" mp3s to my phone in the future.

Solution - Windows/Mac

Unfortunately, I rarely use Windows, and do not use Mac, so I do not have ready solutions to offer for these operating systems.  If you discover any, please let readers know in the comments below.  (For what it is worth, I remember using Windows freeware The GodFather for tagging music files some years back - it is very full featured and it wouldn't surprise me if it is capable of this task).

However, if there are no native software solutions available, the Ubuntu option could still potentially be deployed on PCs running these platforms either by downloading, burning and then booting from a LiveCD or installing Ubuntu within Windows using Wubi.

There is even a possibility that mid3v2 can be installed on Macs given that, like Ubuntu, both operating systems are based on Unix - but I will rely on those more knowledgeable and experienced than me to confirm or refute that possibility.

Conclusion

After diagnosing this bug and the workaround explained above, I am happy to report that I have experienced ZERO problems with the Nokia Music application on my E63 even after loading more than 1200 songs.  Hopefully, my investigations can help other justifiably frustrated users of E-series and other affected Nokia handsets, too.