Still here in 1.0; quite a serious bug imo...

I have this same problem, i think this should be listed as critical.

*** Bug 537957 has been marked as a duplicate of this bug. ***

*** Bug 538154 has been marked as a duplicate of this bug. ***

fwiw the SysInfo file is not populated by default, and nor is it necessary -
e.g. Songbird is able to create the itunesdb correctly without the firewireguid
entry in SysInfo.

Bug #537957 has the output of podsleuth and mentions some ipod-sharp logs printed on the console, bug #538154 has some different traces output in the console.

Loading the ipod with amarok afterwards shows all the files (just the ipod
can't seem to see them). It's kinda like it was before libgpod was able to
write the cryptographic hashes for newer models (pre 0.6.0).
If you then add a song from amarok and have amarok write the database (using
libgpod) all songs (also the ones banshee put on) show up properly.


My installed versions (Gentoo)
banshee-1.0.0
ipod-sharp-0.8.0
podsleuth-0.6.1

The ipod is a classic 80GB.

If you need any additional information I'm happy to provide it.

it looks like i got the same problem, ipod classic black 80gb + linux mint elyssa + latest banshee version (repository)

it hope it is going to be fixed, banshee is a wonderful piece of software :)

Same problem. I think the most recent (non-touch) iPods are affected. Rebuilding the song database on my 3rd Gen Nano has Banshee recognize the music, but not my iPod D:

Hope this gets fixed soon.

More info in bug 546776
Confirmed in Ubuntu 8.04, using Banshee 1.2.0

My research show that this only happen with 3rd Gen iPods, older ones works perfect. After you transfer the music the iTunesDB get corrupted, then you can get the database fixed with gtkpod...

*** Bug 546776 has been marked as a duplicate of this bug. ***

*** Bug 538107 has been marked as a duplicate of this bug. ***

*** Bug 538412 has been marked as a duplicate of this bug. ***

Confirmed based on duplicates.

Changing the summary, 3rd gen iPods generally refer to really old ipod models (grayscale and no click wheel)

Still here in today's SVN.

I tried with a fresh db, and a classic iPod filled with songs. gtkpod has no problem with it, I can also see the tracks and play them on the device if I upload them with gtkpod, however banshee doesn't even display the songs present.

I think that both problems -- saving the database and reading it -- are connected with non-latin1 chars present in some tracks name.

For example, I've got this album name in the db:
"Štěpán Rak (R) - John Duarte (D)"

If I look at the console, however, I see this upon loading of the iPod:

 Exception executing command: SELECT CoreAlbums.AlbumID,CoreAlbums.ArtistID,CoreAlbums.MusicBrainzID,CoreAlbums.ReleaseDate,CoreAlbums.IsCompilation,CoreAlbums.Title,CoreAlbums.ArtistName FROM CoreAlbums WHERE 1=1 AND CoreAlbums.ArtistID = 105 AND CoreAlbums.Title = '`tp�n Rak (R) - John Duarte (D)' AND CoreAlbums.IsCompilation = 0
[Warn  22:07:07.628] Caught an exception - unrecognized token: "'`t" (in `Mono.Data.SqliteClient')
  at Mono.Data.SqliteClient.SqliteCommand.GetNextStatement (IntPtr pzStart, System.IntPtr& pzTail, System.IntPtr& pStmt) [0x00000] 
  at Mono.Data.SqliteClient.SqliteCommand.ExecuteReader (CommandBehavior behavior, Boolean want_results, System.Int32& rows_affected) [0x00000] 
  at Mono.Data.SqliteClient.SqliteCommand.ExecuteReader (CommandBehavior behavior) [0x00000] 
  at Mono.Data.SqliteClient.SqliteCommand.ExecuteReader () [0x00000] 
  at (wrapper remoting-invoke-with-check) Mono.Data.SqliteClient.SqliteCommand:ExecuteReader ()
  at Hyena.Data.Sqlite.HyenaSqliteCommand.Execute (Hyena.Data.Sqlite.HyenaSqliteConnection hconnection, Mono.Data.SqliteClient.SqliteConnection connection) [0x00071] in /home/matteo/Public/banshee-bzr/banshee/src/Libraries/Hyena/Hyena.Data.Sqlite/HyenaSqliteCommand.cs:126

Under the "Music" view of the iPod device inside banshee, no track is shown at all (like it was empty).

So, maybe some encoding problems? Or are there just some unescaped entities in the query strings? gtkpod uses UTF-8, btw.

Can someone else on this list say if they're using non-ASCII/non-latin1 id3 tags for their songs or not?

If I remember correctly, all strings on the ipod are stored in utf-16/ucs2, so anything else in the DB would indicate some buggy app wrote to the ipod.

Yes, you're right: http://gtkpod.wikispaces.com/Setting+iPod+Properties
"Please note that if necessary you can change the charset each time you add files or directories: the iTunesDB itself is using UTF16, so once tags are imported correctly, changing the charset has no influence."

So, may this instead be a bug of ipodsharp vs. libgpod?

As for comment #16, should I report a separate bug?

But accented characters appear OK in the iPod UI itself, so that indicates the format in the database is correct.

Matteo: Not being able to see tracks on an ipod does sound like a different bug. I can see the ones on mine, just not update the DB without it being corrupted.

The accented characters thing is an interesting idea and should be pretty easy to test - empty the ipod and copy over a few albums with only Latin characters?

Created attachment 117969 [details] [review]
Patch

This patch fixes the issue.  The issue was ipod-sharp had a bit of code that altered the hash it would produce when the db file reached a certain point.  So for me, I couldn't add more than 309 songs without triggering that, the hash failing, and so the ipod saying I had no songs.

This problem has been fixed in the development version. The fix will be available in the next major software release. Thank you for your bug report.

Hi Gabriel,

thanks for addressing this issue. Unfortunately, this bug is still here for me in today's SVN.

I've tried to empty completely my iPod, and transferring tracks to it via banshee.

Not only now it takes > 7 hours to sync my library (before, it took ~1.5h, but I ain't sure it's related), but upon disconnect, it shows empty.

As usual, running gtkpod for recalc the hashes fixes it.

(In reply to comment #23)
> Hi Gabriel,
> 
> thanks for addressing this issue. Unfortunately, this bug is still here for me
> in today's SVN.

Are you running ipod-sharp from svn?  That's where the bug was.

Yes.

matteo@Tulip:~/Public/ipod-sharp$ svn info
Percorso: .
URL: http://anonsvn.mono-project.com/source/trunk/ipod-sharp
Repository: http://anonsvn.mono-project.com/source
UUID del Repository: e3ebcda4-bce8-0310-ba0a-eca2169e7518
Revisione: 112388
Tipo di Nodo: directory
Azione: normale
Autore dell'Ultimo Cambiamento: gburt
Revisione dell'Ultimo Cambiamento: 112218
Data dell'Ultimo Cambiamento: 2008-09-04 00:50:59 +0200 (gio, 04 set 2008)

And you did make && make install, right?  How are you running banshee, with make run or banshee-1 (or the launcher)?

Can you test w/ just 4 or 500 songs?  How big is your library?

Also try with only a handful of songs.  Also, please confirm you don't have any packaged version of ipod-sharp installed.

I'm using banshee-1 (the launcher); my library is around 5300 songs.
I've got latin, japanese and eastern countries' characters in song names, album names and among artists.

I'm sure I haven't also another packaged version of ipod-sharp installed because I made specifically a package of it taking the last SVN revision, and removing the distro-provided .deb in the process.

I can add a variable number of songs and they may or may not show up in the ipod, depending on what I chose.

However, adding just *one single track*, the one having these info:

Album Artist: "Prague Guitar Quartet"
Track Artist: "Prague Guitar Quartet"
Album: "Štěpán Rak (R) - John Duarte (D)"
Title: "R: České Pohádky Op.43c"
Genre: "Acoustic"
Track number: 13
Track count: 22
Disc: 1
Year: 1995

results in no tracks showing up on the ipod. If I reconnect the ipod with banshee open, I can't see this track showing up, either; it's as it were completely lost (but gtkpod is able to restore it).

Other tracks with non-ASCII, non-latin characters trigger this.

Moreover, I noticed that, using the new automatic syncronization feature of banshee, all songs containing russian, japanese and serbian characters (e.g.: 'Š', 'ě') are automatically removed when connecting the ipod.

On the left, just above the progress bar, I don't see the original names of the songs being removed, but strange characters like "�▯�▯�▯�▯�▯�▯�▯".

Then, the original songs with the right UTF-8 names are added again to the device.

So, are we really sure for example that the hash is computed on a string of the right encoding (UTF-16 and not UTF-8, if I understood it correctly)? Could be something related to charsets anyway?

I'm insisting on this because:
* adding these songs with banshee results in no item showing up in the ipod upon disconnection, or occasionally in an item with some characters missing (for example, "Queensrÿche" is shown as "Queensrche" between the ipod artists);
* running gtkpod fixes the problem: the tracks added are showing up, and I see the non-ASCII characters just fine in the ipod (so I believe they are in UTF-16, and the hash is correctly calculated on this);
* if I start banshee again with the autosync feature enabled, it removes all and only the tracks with non-ASCII characters in the name/album/artist, showing their name garbled in the UI while removing them, which makes me think about an character encoding problem.

Else, may the hash algorithm in ipod-sharp be differing from that of libgpod in some non-obvious way?

Please discard everything I say if I'm wrong; I'm just trying to help but I admit not having dived into this completely (yet).

Sorry by the spam:

by the way, when a non-ASCII tagged item is shown in the ipod, I see some text garbled in the album name or in the artist name, but as far as I tried the song title is always okay.

Don't know if it is a rule, though, so don't relay on it. I've just tried removing/adding ~20 songs from different albums one at a time.

Still an issue at least for me. Is it okay for everyone else?

Matteo,

Please confirm you are using podsleuth from svn, ipod-sharp from svn, and banshee from svn?

I'm using ipod-sharp from SVN, banshee from SVN but *not* podsleuth from SVN. I'm gonna check it out now, and I'll tell you if this solves the issue.

Can anybody else who commented they had this problem confirm they still have it when running the latest versions of everything?  (Banshee 1.3.1 or 1.3.2, etc)

Matteo, are you still seeing all the problems you saw before?  If you only transfer American songs to the device, does it work?  Can you concisely describe the problem(s) with the most recent version of everything, and how to reproduce it/them consistently?

Reopening the bug.

Still seeing this also with latest podsleuth from SVN.

Transferring only songs with metadata from the ASCII set seems okay as far as I've tried (but I'll try some more; not really dug through it since most of my collection *doesn't* use ASCII).

Also note that, if I check "automatically sync..." in the iPod properties, *every time* I close/re-open banshee, the same tracks are deleted (with non-latin1 characters substituted with "�▯" and similar above the progress bar) and re-added again.

Even closing and *immediately* re-opening banshee brings up this problem.

Both gtkpod and amarok have no problem whatsoever with managing my library or transferring songs to the device (if banshee hasn't put its hands on it before, of course ;-)).

I'd attach an MP3 which presents this problem here, but I don't know if this is okay due to copyright laws. Maybe a 10 seconds sample would be okay? Please tell me.

Created attachment 119682 [details]
Screenshot of the issue

This shows what I see everytime I open banshee. I also forgot to add that gtkpod, after a banshee synchronization, finds a lot of duplicates and removes them - typically the songs with the 'correct' title (correctly encoded) and leaves on the device the tracks with the 'wrong' one (the ones banshee just added).

Created attachment 119685 [details]
Banshee log

This log shows some errors when executing SQL statements. Maybe it's related.

In the future please attach logs as plain text, or even paste them inline if they're short.

Matteo, what distribution are you on?  Is your sqlite built some funky settings?  I really don't get what's going on with your setup.

Is anybody else seeing any of Matteo's problems?

I'm trying to confirm this, but I have problems building from the SVN... I'll try a little more and see what happens...

I'm on Ubuntu Intrepid.
sqlite here is version 2.8.17-4build1, compiled by the distribution packager with these custom parameters (other than distro defaults):

DEB_CONFIGURE_EXTRA_FLAGS = \
      config_TARGET_TCL_INC="-I/usr/include/tcl8.4" \
      config_BUILD_CFLAGS="$(CFLAGS) -DTHREADSAFE=1" \
      config_TARGET_LIBS="-lpthread" \
      --enable-utf8

PS. sorry for the gzipped log, but bugzilla wasn't letting me attach that as plain text since it was too big. 

I'm running Intrepid and having no such problems

I wiped my ipod last night to test out the syncing support in SVN and it wrote the itunesdb just fine.
It transferred over 4110 tracks, although I'm not aware of any having non-latin names.

I guess I could artificially inject some unicode characters into ID3 tags and see what happens.

I did notice the same thing as Matteo wrt tracks being removed each time I synced, but I think that was because they were ones which hadn't been transcoded yet (about 10% of my collection is ogg and I ran out of time and had to kill the transcoding).

Confirmed in Banshee Unstable PPA (1.3.2). (For some sick reason I can't build from source, sorry) And iPod Nano Video 4GB.

I've added more or less 200 tracks with US-characters and everything was fine, even the covers transfered OK (That's the reason I want to use Banshee with the iPod instead of gtkPod or Rythmbox).

But, if in that library I add some tracks with accents or ñ (spanish characters) the whole thing mess up and sometimes a "No music" appears in the iPod OR the covers get all wrong (The cover of album A shows in 3 albums, or something like that).

I'm trying to add various albums of "La Oreja de Van Gogh", both the filenames of the songs and ID tags have accents and ñ's... Sometimes the iPod fails, sometimes is ok...

First I added 15 songs (1 album) and everything went smoothly, then I tried 62 items (4 different albums) and the covers started to messed up...

If you need more info, just ask... (sorry for my poor English)

Ok, I've been trying a little more... All the spanish songs transfered ok, BUT I have an album that have the first song with an accent (El Último Vals)... THAT specif album got a messed cover, and sometimes corrupts the database... If i delete the accent, then everything is fine...

More info as long as I'm discovering...

I am experiencing messed up covers all of the time. btw there is a patch for rhythmbox to fix album art or of course you could just use amarok to fix it after syncing. 

Happened to me yesterday.

After I edited most of my files to comply with MusicBrainz, some artist names were rewritten in UTF-8 (using Picard, a recommended tagger). The tags are OK. I first cleaned my DB, then uploaded about 800 songs to the iPod. I then listened to random tracks (some of them had non-Latin tags) and everything was working (artist/track name, cover art, sound, shows up). This morning, I had to upload one more track I forgot about and it just messed up everything! The tag was pure ASCII and was successfully uploaded, but after plugging out the iPod, the database was just dead. Now, I can see the ASCII artists/tracks, but the others are just short strings with random ASCII characters. I cannot see any cover art. The sound works however.

I used to have some non-ASCII artists before the update of my library, and I remember experiencing that kind of trouble, but I didn't care at the stage. I also remember having a lot of SQL error once (I'll check the logs from yesterday and today tonight).

BTW, I got a bug once, where every single string in the iPod was split (even the main menu: "Music" appeared "Usic"!) but I didn't report it since I couldn't reproduce it. I however had to firmware-upgrade the iPod to make it work correctly again. May be related.

Configuration:
  * Ubuntu 8.10
    * Banshee 1.4.1 (trunk, yesterday)
    * Podsleuth (trunk, less than a week)
    * ipod-sharp (trunk, less than a week)
    * SQLite (from Ubuntu repository)
  * iPod (nano 4g 8 GB)

I can confirm comment #34. From a fresh iPod file system, uploading songs to the iPod works great (cover art, artist/track name), but closing and immediately reopening Banshee corrupts the database.

Therefore, I guess that Banshee tries to write something to the iPod when it shows up. What is it?

BTW, I know it's a bit off-board, but is there a working piece of software to reset an iPod ("rm -rf /media/IPOD/*" just don't work, gtkpod don't support recent iPods and "wine iTunes.exe" is just the thing I don't want to do - even if it worked) ?

I can see another strange behavior. If after unmounting the iPod (whether it's done with Nautilus, Banshee or directly "umount"), its screen still says "Connected". I have to unplug it so that it says "OK to disconnect", and then goes back to the main menu. This is a recent behavior. Can anyone confirm ? I can't tell if it's related, I'm not even sure this is a bug.

Sorry for the spam, hope this will help.

Bug still exists. I'm investigating.

Most simple step to reproduce:
1) Have a clean iPod and a recent Banshee version
2) Run Banshee
3) Plug the iPod
4) Upload a song with non-ASCII artist name
5) Eject iPod
6) Look at the iPod, everything should be OK
7) Plug the iPod again
8) Upload another song from the same artist
9) Eject iPod
10) Look at the iPod, now there are two artists, one good (the new song you uploaded) eg. "植松伸夫" and one bad (the previous song) eg. "i~g8O+Y". And this goes on and on each time Banshee reads the iPod. However, the songs are playable on the iPod.

And please, put this bug to confirmed. I'm looking at the source of ipod-sharp, please tell me if I'm not looking in the right direction.

Ok, if I hard-unplug my iPod, Banshee doesn't have time to screw up the DB, but if I use Eject, it does.

First time I upload a song to the iPod, it's OK. The next time I plug the iPod, Banshee badly reads the iTunesDB (the artist name is not good when I list the song of the iPod in Banshee), then, if I eject it, Banshee syncs the DB (why?) and makes a mess.

So my guess is that the bug is not the the writing but in the reading of the iTunesDB!

Most simple step to reproduce (updated):
1) Have a clean iPod and a recent Banshee version
2) Run Banshee
3) Plug the iPod
4) Upload a song with non-ASCII artist name
5) Eject iPod
6) Look at the iPod, everything should be OK
7) Plug the iPod again
8) Eject iPod
9) Look at the iPod, now the artist name is wrong eg. "i~g8O+Y" instead of "植松伸夫". However, the songs are playable on the iPod.

Actually, Banshee even screw up the DB with tracks with ASCII tags. The name is right but the artwork disappeared.

I think I located the bug. It's not in Banshee, it's in ipod-sharp. DetailRecord.Value is wrong for non-ASCII tags.

Trying to fix it.

Apparently, there are two separate bugs.

First is tag database being screw up after reading it with Banshee.
Second is artwork disappearing after ejecting the iPod.

First problem has been solved. A patch is coming.
I didn't look at the second problem yet, but I guess it's somewhat related.

Created attachment 125072 [details] [review]
Fix iPod database reading (always use UTF-16 instead of UTF-8)

This is a patch for *ipod-sharp* (maybe it's not where I'm supposed to propose the patch).

Apply the patch to ipod-sharp trunk, make, make install.
Go to your banshee source tree, touch src/Dap/Banshee.Dap.Ipod/Banshee.Dap.Ipod/IpodSource.cs, then make && make install.

Please review (and test), so that it can go to trunk ASAP.

Warren, this is the right place for ipod-sharp bugs.  This change will need a lot of testing.  Please confirm the types of iPods you've verified this works with.

I have only tested the patch with an orange iPod nano 4th gen. 8GB (yes, the firmware is different depending on the color – don't ask me why…).

BTW, libgpod (a C library for iPod access) has recently added supported for relatively new iPods (such as mine) and works great. Maybe some work could be shared.

Again, if somebody goes through here and has a different iPod, it would be great if he/she could confirm that the patch won't cause a kernel panic :) (oh, and that it fixes iPod support).

@Warren: unfortunately this patch, whilst it doesn't panic the kernel, doesn't fixes this issue for me. Doesn't happen always; seems to happen when transferring more than a certain number of tracks.

Still no track shown on my iPod after transferring them with banshee. gtkpod works okay as always, but now it doesn't even tell me the database is corrupted: it shows no tracks, no warning, and I've to "check for orphan files" to get all tracks.

It seems that the database isn't even written to disk:

matteo@Tulip:/media/MATTEO SETT/iPod_Control/iTunes$ ls -l
totale 336
-rwx------ 1 matteo root     36 2008-06-05 01:43 BansheeIPodName
-rwx------ 1 matteo root    138 2009-04-04 17:22 gtkpod.prefs
-rwx------ 1 matteo root   4032 2009-04-04 19:31 iTunesDB
-rwx------ 1 matteo root   4032 2009-04-04 19:28 iTunesDB.bak
-rwx------ 1 matteo root     78 2009-04-04 17:22 iTunesDB.ext
-rwx------ 1 matteo root   1232 2008-12-06 14:56 iTunesPrefs
-rwx------ 1 matteo root     18 2009-04-04 17:22 iTunesSD
-rwx------ 1 matteo root    616 2008-08-27 11:30 PhotosFolderAlbums
-rwx------ 1 matteo root     96 2008-08-27 11:30 PhotosFolderPrefs
-rwx------ 1 matteo root 171848 2009-04-02 19:03 Play Counts.bak
-rwx------ 1 matteo root    245 2008-12-06 14:56 Rentals.plist

There are > 6000 tracks on the device, but iTunesDB is ~4KB. So it's not even a proper "database corruption": before, gtkpod was able to rebuild it, now it doesn't since it isn't even there.

I have iPod Classic 160GB (silver) and reproduce the issue that is described by
Warren Seine (Comment #50) at a 100% rate.

I used a few songs with Serbian characters in them (ш, љ, њ, ђ...).

I also see that Banshee will not pick up album art for these songs even if it's
sitting in the same folder. This works fine for songs with latin characters
only.

(iPod Classic 160GB (silver))...

Built from SVN last night, and songs with Serbian characters work fine, but ASCII ones do NOT. They don't get transfered.

Also, iPod reports wrong number of songs on the device (not sure if the number is random or not, but with having close to a 1K of songs loaded, the total number says 8).

Worth noting: I think this appears for Winamp as well. I have dual-boot with Windows 7, where I used Winamp to handle my iPod, but Banshee seems to fail at recognizing tracks

Just tested Warren's patch and want to add that it works on my silver 160GB classic.

I do not get any album art (present on ipod itself), but can live without that for sure...

Thanks Warren, well done.

my initial impression is that Warren's patch appears to improve things on my 120GB iPod Classic.

I guess there are multiple bugs. But I'm pretty sure my patch fixes one of them.

I'm sorry I can't help anymore as I sold my iPod for an iPhone a while ago (any good soul willing to support this device? :) ). Good luck anyway!

I have got the problem, too. It's the 120 GB iPod classic (black, it's very important ;) )

I'm using Banshee 1.6 Beta 4 (1.5.3) from the Ubuntu Launchpad PPA.

An interesting thing is that it takes several hours (!) to empty the iPod. I don't know if it helps.

For me it's solved in the daily PPA.

The problem should no longer exist in banshee 1.7.5 and higher as it uses libgpod now.

I just bought a ipod classic 160 GB and plugged it to banshee (before itunes) and I got the issue as described in this bug, when I disconnect it from my notebook it does not recognize any music.

I just plugged it to gtkpod and it fixed everything.

Frederico, did you notice that this bug is marked as FIXED? What version of Banshee are you using? Can you provide a log?