Solution to Music Library Does Not Refresh Problem

(OK, technically this is a workaround, not a solution - but that's just semantics, right?)

The problem

Users of Nokia's E63, E71 and other E-series phones have been complaining for years on the Nokia Support Discussions site about a frustrating bug in which the music library does not recognize new music files that have been added to the phones memory (or more typically, the memory card).  Typically, the first time the music player is launched after the files have been added, it attempts to refresh but hangs after adding only some (or in other cases, none) of the songs to the library.  These music files appear in the file browser, and can even be played from there, but are not added to the library.

Many responses in the Nokia forums suggest solutions that do not make intuitive sense, or are quite drastic, such as performing a software reset of the phone (which erases all data); defragmenting or reformatting the memory card; or transcoding all files to a different format (such as from mp3 to aac.  While a few users may have success with these solutions, I have found them either ineffective or impractical on my handset.

Many users have also figured out how to force the phone to rebuild the music library by deleting key database files from the memory card.  This is useful after the root cause of the problem has been solved, but is not a solution in itself.

Root cause

So what is the root cause?

A few more savvy users have correctly identified that this problem is associated with select "bad" mp3 files that the player doesn't seem to "like".  However, the workaround they have offered - identifying these files through painstaking trial and error and delete them from the phone, or re-encode them - is not acceptable to me.  I want a permanent fix!  The good news?  After isolating some of these "bad" files in my music collection, and analyzing them, I think I finally found one!

Metadata describing mp3 files - e.g. the artist, album, song title, track number, composer, etc. - is stored within the file in ID3 tags.  While the Nokia E-series music library appears to have no problems processing these common fields, I have discovered that it chokes on files with POPM frames.  These frames are added to mp3s (and possibly other formats which use ID3 tags) by some media player applications, typically to store ratings information entered by the user (e.g. "5 stars").  The Nokia player appears to add new music files to the library database in batches of ten files.  If any one of the 10 files has a POPM frame, none of the files in the batch will be added to the library and scanning will hang, requiring a manual stop.

Solution - Ubuntu

It's simple - find software that can remove POPM frames from your music library! 

If you use Ubuntu (or probably any other modern Linux distribution), the command line mid3v2 package can do this for you with minimum fuss.  First, open a terminal window and install the package using the following command:

sudo apt-get install mid3v2

EDIT: After upgrading to Ubuntu 10.04 (Lucid Lynx), I discovered mid3v2 is no longer available in the repository as a separate package - it is now part of python-mutagen, which can be installed using the following command: sudo apt-get install python-mutagen

Then simply run the following command to strip all of your mp3 files of their POPM frames:

find /path/to/music/collection/ -iname *.mp3 -exec mid3v2 --delete-frames=POPM '{}' \;

(The path to your music collection could be on your PC - e.g. ~/Music - or on your phone - e.g. /media/microsd).

EDIT: Commenter Thameera reports that he had to use a slightly different syntax than that of the command above.

If you sanitize the files on your PC, then you can simply transfer them to your phone and you should be able to refresh the music library and see all of them.  If the library refuses to refresh, try exiting the Nokia music player, connecting your phone to the PC via USB as a mass storage device, or connecting the memory card, and deleting the following files:

• Private/101FFC31/mpxv1.mpd
• Private/101ffca9/harvesterdb.dat

(These are E63 file locations - they may vary across other E-series devices).

This will force the music library to rebuild itself.  You may find that it appears to hang once again, but if you stop it and wait a few seconds, it will have added some files, and when you then manually refresh the library you will be able to see its progress.

Because I would never be able to memorize this command, I have saved a bash script in the root of my memory card which I can run at any time if I accidentally transfer new "bad" mp3s to my phone in the future.

Solution - Windows/Mac

Unfortunately, I rarely use Windows, and do not use Mac, so I do not have ready solutions to offer for these operating systems.  If you discover any, please let readers know in the comments below.  (For what it is worth, I remember using Windows freeware The GodFather for tagging music files some years back - it is very full featured and it wouldn't surprise me if it is capable of this task).

However, if there are no native software solutions available, the Ubuntu option could still potentially be deployed on PCs running these platforms either by downloading, burning and then booting from a LiveCD or installing Ubuntu within Windows using Wubi.

There is even a possibility that mid3v2 can be installed on Macs given that, like Ubuntu, both operating systems are based on Unix - but I will rely on those more knowledgeable and experienced than me to confirm or refute that possibility.

Conclusion

After diagnosing this bug and the workaround explained above, I am happy to report that I have experienced ZERO problems with the Nokia Music application on my E63 even after loading more than 1200 songs.  Hopefully, my investigations can help other justifiably frustrated users of E-series and other affected Nokia handsets, too.