#musicbrainz-devel

totoro is apparently correct

maybe something else has to be done before it's actually usable

I'd be surprised if the ram is installed, since IIRC ruaok is still a few hundred miles north of our servers and they didn't get rebooted?

I'm guessing 'added' was 'to the cart wherever it's being bought online' :P

hello, i am interested in solving the bug http://tickets.musicbrainz.org/browse/PICARD-644, can somebody help me to get started

I am new to this org

kartikgupta0909: can you reproduce it ?

zas i dont get you?

zas: i dont get you?

zas: i mean are you not able to reproduce this error?

kartikgupta0909: i cannot reproduce it here, i added a comment, so i want to know if you are able to (and what is your env)

zas: i use ubuntu 13.04 , i havent tried reproducing it

zas: lemme try it then

then please try to reproduce it, it will be hard to fix without being able to reproduce it imho (python/unicode/env mess)

zas: do you know for what kind of bad names does this happen?

it is described, check original comment from Freso

should (03, 04, 05, 07, 08).mp3 give me an error? am i getting it correct?

zas: should (03, 04, 05, 07, 08).mp3 give me an error? am i getting it correct?

yes, check Freso's description, it is clear enough for me

zas: i cant reproduce it on ubuntu

Freso: ^^^

kartikgupta0909: please add a comment about that in the tracker

zas: one thing i noticed just now is that when only one file is added and you click on cluster it does nothing

zas: dont you think that track should be added to already existing clusters or at least the user should be informed that he needs to add more than one track

kartikgupta0909: There's no "(03, 04, 05, 07, 08).mp3" in the download...

Freso: i think i misunderstood the bug, what i did was in my picard on my machine, i tried to add a file named "(03, 04, 05, 07, 08).mp3"

Freso: and it got added

kartikgupta0909: files are in the zip

Freso: what is your env ?

zas: I linked you last night...

Freso: ok, i didn't see, ty

Anyway, I commented with some more thoughts and included sprunge link.

kartikgupta0909: Those files are the ones that error for me.

Ie., {03,04,05,07,08}*.mp3 in the archive.

Freso: oh okay

(If you're not using a *nix, that glob won't do you much good.)

Freso: your filesystem encoding is utf8 ?

Read the entire issue, please. The very first sentence tells you where you can get the problem files, though it shouldn't be too difficult to create some of your own.

zas: Yep.

Everything is.

really weird i cannot reproduce it

Python 2.7.6 here

I don't think it depends on Python version.

As per my last comment on the ticket.

Freso: can you reproduce in debug mode and post the log ? also, can you check the output of sys.getfilesystemencoding() ?

zas: https://pastee.org/8xgg9

my log is similar, but no exception, files are loading without issue

I just tried extracting to an ext4 partition, same thing happens.

What do you use to unpack the .zip?

unzip (UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.)

If I extract with `7z x`, I can load the files fine.

perhaps your unzip version differs than mine ??

"UnZip 6.00 of 20 April 2009, by Info-ZIP. Maintained by C. Spieler."

So looks like you have some Debian patches in yours.

check patches

Mineo or JonnyJD would have the same unzip version as me.

one concerns encodings

Freso: yeah, reading and I will test that bug

Indeed.

zas: But it also looks like it's a patch imported from ArchLinux, meaning that it would be in my version as well.

Anyway, I have no idea how to check the patches. I've never learned to navigate launchpad well, and I'm not very familiar with Ubuntu's structure, so I don't know what the various names mean.

well, picard is still bugging so the issue isn't really the unzip command ;)

but it explains why i cannot reproduce it somehow

those python2/unicode issues suck (i hope moving to python 3 will help ....)

zas: Do you have convmv on your system?

now, yes ;)

there are 6 patches in the ubuntu package:

01-manpages-in-section-1-not-in-section-1l 05-fix-uid-gid-handling

02-branding-patch-this-is-debian-unzip 06-unzip60-alt-iconv-utf8

03-include-unistd-for-kfreebsd series

04-handle-pkware-verification-bit

If you have a file with UTF-8 characters, you can try and "reverse" the convmv. E.g., `convmv -f utf8 -t iso88591` or something.

06 sounds most intersting

Freso: i doubt it will work, as utf8 chars in filenames cannot be converted to latin1

iso-8859-1 doesn't cover all needed characters for: "./08 MisforstЖ mig ret.mp3"

and I can't find that patch on debian (possibly ubuntu only or because I can find the -9 on debian because it is not included in any release)

zas: That should actually be "Misforstå", and yes, "å" is in 8859-1.

interesting, on my system it unzips as "08 MisforstЖ mig ret.mp3" with "Ж" not "å"

the å looks fine on my system, but 4 and 5 look weird

oh, no, that is only in the unzip output. On system I do have "?" for both

but yeah, in unzip output 04 Dilemma - Bær mig væk.mp3, that only looked really weird the font I am using. Unzip output looks fine, but ? with "ls"

(and having that stacktrace in Picard when opening)

so basically we have badly encoded characters in the filenames and Picard fails to handle them

yep

zas: and you have a patch with this subject in your unzip: Subject: unzip files encoded with non-latin, non-unicode file names

basically just not making this happen

yes i have it apparently

I might make this problem go away on Arch with including that patch, but that is not what we want. We want to fix Picard so these are ignored, right? something like "replace" somewhere

imho we should test unicode conversion, and handle exception

kartikgupta0909: you could reproduce this problem, right?

JonnyJd: my net is working slow, so it was not able to download, once it downloads i ll check that.

kartikgupta0909: what OS are you on?

JonnyJd: ubuntu 13.04

Freso, JonnyJD : can you try "unzip -l Dynamikken_MensviventerEP.zip | python -c 'import sys; print sys.stdin.read(2000)'"

kartikgupta0909: in case you can't reproduce that due to a patch in your unzip: I'll create a tar.gz from my unzip and upload that somewhere

JonnyJD: Ideally, the scenario with the bad encoding mixup would never happen, but Picard should handle it more gracefully. Ideally actually loading the files, but at the least it should give a (more) helpful error message.

JonnyJd: once it downloads i ll let you know

zas: *python2 :) But same output :)

zas: I get invalid syntax at the openening (

JonnyJD: Use python2, not python. ;)

oh, forgot ;

and yes, same output now

unzip -l Dynamikken_MensviventerEP.zip | python2 -c 'import sys; print repr(sys.stdin.read(2000).decode("UTF-8"))'

08 Misforst\u0416 mig ret.mp3\n

here

"UnicodeDecodeError: 'utf8' codec can't decode byte 0xe5 in position 438: invalid continuation byte"

here we go ;)

no error here

btw, http://www.fileformat.info/info/unicode/char/04... is "Ж"

decode("utf-8", "replace") works, but than we don't have the correct path I guess

unzip -l Dynamikken_MensviventerEP.zip | python -c 'import sys; print repr(str(sys.stdin.read(2000)))' -> what is 08 file path for you ?

08 Misforst\xd0\x96 mig ret.mp3

time to prepare lunch, afk

"Dynamikken - Mens vi venter EP/08 Misforst\xe5 mig ret.mp3"

zas, kartikgupta0909: this should reproduce it without using unzip (and I removed half of the files): kraehen.org/tmp/dynamikken_names.tar.gz

Freso: that patch got actually removed from Arch Linux (no note on why though): https://projects.archlinux.org/svntogit/package...

Freso: https://bugs.archlinux.org/task/18691

yes, because it had other problems and was actively rejected upstream. This is probably also the reason why it was also removed (later) on Debian/Ubuntu)

p7zip should be used is said

Well, `7z x` also "works". Using `convmv` also works.

The issue is not so much getting the files to behave, as to get Picard to behave when given badly formatted filenames.

sure, but now I know we don't have to open a bug for unzip on Arch ;-)

Haha. Right. :)