2010-05-29

Solution to Music Library Does Not Refresh Problem

(OK, technically this is a workaround, not a solution - but that's just semantics, right?)

The problem

Users of Nokia's E63, E71 and other E-series phones have been complaining for years on the Nokia Support Discussions site about a frustrating bug in which the music library does not recognize new music files that have been added to the phones memory (or more typically, the memory card).  Typically, the first time the music player is launched after the files have been added, it attempts to refresh but hangs after adding only some (or in other cases, none) of the songs to the library.  These music files appear in the file browser, and can even be played from there, but are not added to the library.

Many responses in the Nokia forums suggest solutions that do not make intuitive sense, or are quite drastic, such as performing a software reset of the phone (which erases all data); defragmenting or reformatting the memory card; or transcoding all files to a different format (such as from mp3 to aac.  While a few users may have success with these solutions, I have found them either ineffective or impractical on my handset.

Many users have also figured out how to force the phone to rebuild the music library by deleting key database files from the memory card.  This is useful after the root cause of the problem has been solved, but is not a solution in itself.

Root cause

So what is the root cause?

A few more savvy users have correctly identified that this problem is associated with select "bad" mp3 files that the player doesn't seem to "like".  However, the workaround they have offered - identifying these files through painstaking trial and error and delete them from the phone, or re-encode them - is not acceptable to me.  I want a permanent fix!  The good news?  After isolating some of these "bad" files in my music collection, and analyzing them, I think I finally found one!

Metadata describing mp3 files - e.g. the artist, album, song title, track number, composer, etc. - is stored within the file in ID3 tags.  While the Nokia E-series music library appears to have no problems processing these common fields, I have discovered that it chokes on files with POPM frames.  These frames are added to mp3s (and possibly other formats which use ID3 tags) by some media player applications, typically to store ratings information entered by the user (e.g. "5 stars").  The Nokia player appears to add new music files to the library database in batches of ten files.  If any one of the 10 files has a POPM frame, none of the files in the batch will be added to the library and scanning will hang, requiring a manual stop.

Solution - Ubuntu

It's simple - find software that can remove POPM frames from your music library! 

If you use Ubuntu (or probably any other modern Linux distribution), the command line mid3v2 package can do this for you with minimum fuss.  First, open a terminal window and install the package using the following command:
sudo apt-get install mid3v2
EDIT: After upgrading to Ubuntu 10.04 (Lucid Lynx), I discovered mid3v2 is no longer available in the repository as a separate package - it is now part of python-mutagen, which can be installed using the following command: sudo apt-get install python-mutagen

Then simply run the following command to strip all of your mp3 files of their POPM frames:
find /path/to/music/collection/ -iname *.mp3 -exec mid3v2 --delete-frames=POPM '{}' \;

(The path to your music collection could be on your PC - e.g. ~/Music - or on your phone - e.g. /media/microsd).

EDIT: Commenter Thameera reports that he had to use a slightly different syntax than that of the command above.

If you sanitize the files on your PC, then you can simply transfer them to your phone and you should be able to refresh the music library and see all of them.  If the library refuses to refresh, try exiting the Nokia music player, connecting your phone to the PC via USB as a mass storage device, or connecting the memory card, and deleting the following files:
  • Private/101FFC31/mpxv1.mpd
  • Private/101ffca9/harvesterdb.dat
(These are E63 file locations - they may vary across other E-series devices).

This will force the music library to rebuild itself.  You may find that it appears to hang once again, but if you stop it and wait a few seconds, it will have added some files, and when you then manually refresh the library you will be able to see its progress.

Because I would never be able to memorize this command, I have saved a bash script in the root of my memory card which I can run at any time if I accidentally transfer new "bad" mp3s to my phone in the future.

Solution - Windows/Mac

Unfortunately, I rarely use Windows, and do not use Mac, so I do not have ready solutions to offer for these operating systems.  If you discover any, please let readers know in the comments below.  (For what it is worth, I remember using Windows freeware The GodFather for tagging music files some years back - it is very full featured and it wouldn't surprise me if it is capable of this task).

However, if there are no native software solutions available, the Ubuntu option could still potentially be deployed on PCs running these platforms either by downloading, burning and then booting from a LiveCD or installing Ubuntu within Windows using Wubi.

There is even a possibility that mid3v2 can be installed on Macs given that, like Ubuntu, both operating systems are based on Unix - but I will rely on those more knowledgeable and experienced than me to confirm or refute that possibility.

Conclusion

After diagnosing this bug and the workaround explained above, I am happy to report that I have experienced ZERO problems with the Nokia Music application on my E63 even after loading more than 1200 songs.  Hopefully, my investigations can help other justifiably frustrated users of E-series and other affected Nokia handsets, too.

34 comments:

  1. This seems great except that my knowledge of computing wouldn't allow me to do this as a lot of it I don't understand. This does remain a frustrating problem with music files. I have a new Nokia X6 and I experience the same problem. I thought it was all going to be great, just press the Sync button in Nokia's own OVI Player and all would be fine, but no, a lot of the files show in mass storage but not on the phone or are in the wrong place on the phone. Funny how Nokia don't mention this in their advertising - NOT. Anyway, thanks for trying to explain this fix but I am afraid my knowledge isn't large enough to be able to do it. Shame you are no working for Nokia as perhaps you could fix it for them.

    ReplyDelete
  2. I was having the very same problem with a 6110 Navigator. After checking with the GodFather that none of the tracks had Rating information, I tried deleting the library, stopping the auto refresh in the beggining, and making a manual refresh, which locked on a number (1120 out of 1500 for me). By trial and error on Windows Explorer, I closed down on the first track that wasn't added to the library, deleted it and tried a new manual refesh. And bam! the rest of the tracks were added :) Hope this info is useful!

    ReplyDelete
  3. Thanks for the tip. However the given command did not work for me, instead I used:

    find /path/to/music/collection/ -name '*.mp3' -exec mid3v2 --delete-frames=POPM '{}' \;

    It works like magic!

    ReplyDelete
  4. i'm having the same prob wid my new e63 and i don't kniow a thing how to solve dis prob and the tips u people are talking bout, so if any 1 of u will plz help me...........PLZ

    ReplyDelete
  5. i'm having the same prob wid my new e63 and i don't kniow a thing how to solve dis prob and the tips u people are talking bout, so if any 1 of u will plz help me using windows xp...........PLZ

    ReplyDelete
  6. @Anonymous:
    Here's my blog post on the issue: http://goo.gl/5PX7
    I'm sorry, this was done in Ubuntu and I've no idea how to do this in Windows, so please get the help of another or use a linux box for this. Thanks.

    ReplyDelete
  7. @Anonymous:
    You could try installing The GodFather (http://www.jtclipper.eu/thegodfather/index.html) and see if allows you to remove ratings tags from all of the mp3 files on your phone. Then delete the two files listed in the post from your E63 to force the music library to rebuild. Please post back if you have any luck.

    ReplyDelete
  8. @Jon :
    i m sry im vry bad at comptrs culd u plz giv me da detailed procedure hw to operate godfather as i hv dwnloaded it........thanx

    ReplyDelete
  9. i hv tried removing tags through th godfather but cud nt find the files to delete.........sum1 help me ....PLZ

    ReplyDelete
  10. It's not strictly required to delete the files. Anyway to see these files and delete them you need a third party file browser like YBrowser: http://www.drjukka.com/YBrowser.html

    ReplyDelete
  11. i cudn't find any files by dat name using "Ybrowser" also..........n da prob still remains........plz help

    ReplyDelete
  12. I have fixed the "Music Library Does Not Refresh Problem" by flowing method.

    http://discussions.europe.nokia.com/t5/Pool-of-Knowledge/Problems-with-Refresh-Deleting-Tracks-and-Album-Art-from-Music/td-p/393652

    ReplyDelete
  13. i have restarted my nokia e63 with *#7370*...now my memory card is not recognised.if we refresh it,i am sure it will.now i want to know the steps to refresh

    ReplyDelete
  14. I got solution
    1.just connect to pc via cable in mass storage
    2.rename private to private1
    3.disconnect
    4.search music
    5.reconnect to pc
    6.rename parivate1 to private
    7.disconnect
    8.search for music.
    its working
    ..prem

    ReplyDelete
    Replies
    1. Prem, thanks a lot. I have been having this problem for the past 1 year. i used your method and it worked fine. Kudos to you.

      Thanks

      Delete
    2. Hey Prem... Thanks a ton brother... it worked... it was frustrating. Thank god i didn't go with above posts n reply... but how did you noticed that one...

      Delete
    3. excellent...simple and effective solution..thanks Prem. In my case after renaming to Private1 and removing from Pc, after searching a new file called private is created and it is working fine. I left the old private1 as it is. A good non-technical solution..great! Thanks

      Delete
    4. Very usefull trick

      Thanx Prem

      Delete
    5. hi prem where do i find private? roanna007@yahoo.com

      Delete
    6. thank you so much! :)

      Delete
    7. Enter your reply...thanks prem brother worked very well for me thank you very much

      Delete
  15. thanks man ,u r a genius .
    I solved it on ubuntu ,as mentioned .
    Its feeling good to see again that message 'Refresh Complete'

    ReplyDelete
  16. the solution by prem works like magic

    ReplyDelete
  17. Where is this private in e 63 I can't seem to find it

    ReplyDelete
  18. I was having the same issue in a N958GB, but for what I've seen is still going on, even on N8.

    I took three actions:
    -deleted rating field using mp3tag, following the directions on their forum
    - then I re-wrote all files to idv2.3 based on what I read on allaboutsymbian about this same issue.
    - I realized some files didn't have any ID3 tag, so I filled them. All above was made with mp3tag connecting phone as "mass memory", directly on the music files on the phone.

    I can't tell which of those three was the one solving it, but the fact is that I have a working music library again.

    Thanks for your help.

    ReplyDelete
  19. I tried the deleting part and it worked for me very well. Just deleted the 2 files mentioned in Private folder.

    Thanks for the tip.

    ReplyDelete
    Replies
    1. Thank you, deleting the two files in private worked for me too.

      Delete
  20. Thank you. Deleting the files from private helped me.

    ReplyDelete
  21. @Prem! When i reconnect the phone again, there are 2 folders named private1 and private. What do i do?

    ReplyDelete
    Replies
    1. You can delete the private1 folder.

      Delete
  22. Greate thanks! Finaly my phone decided that 3GB of music is not just 20 songs!

    ReplyDelete
  23. simple solution is that-
    Just cut all the data in your memory card and paste it in your laptop, after that scan your data with anti virus and then format your memory card completely, and then paste your scanned data back to your memory card, it will b definately solved...personal experience

    ReplyDelete
  24. Excellent, deleting the two private files worked on my windows computer. Phone finally refereshing

    ReplyDelete