#musicbrainz-devel

/

      • CatCat
        ridiculus
      • pretend i spelled that right
      • ianmcorvidae is up to the VACUUM at the end, whoo
      • ocharles
        loading recording data here
      • musicpain
        hi ocharles: appreciate you're right in the middle of a major db upgrade so thank you for your time today. Solved the "role does not exist" error after some googling and for the benefit of others: su postgres (create a UNIX passwd for that account if required) then "createuser -d -a -P musicbrainz" without the quotes to create the psql role. Now have an error importing the database "key name is duplicated"
      • warp
        load recording_meta on bullet-bill and load vote on 18-volt.
      • 23:00, I'm going to run off.
      • ruaok: I'll check the schema change doc / e-mail / irc logs tomorrow morning.
      • goodnight mb dash devel!
      • Freso
        How closed is the Agenda for IA Meeting page? I'd like to propose "Support for other image types than JPEG", but not sure if that's good or bad karma. :)
      • ianmcorvidae
        that's not something that needs discussing with the IA :)
      • warp
        why not?
      • ianmcorvidae
        unless I'm misunderstanding
      • they already create thumbnails for a variety of formats, *we* don't allow them
      • there's some that don't work, but
      • nikki
        we're waiting for the cors thing for pngs
      • we already told them about gif thumbnails, but haven't heard anything back yet
      • ianmcorvidae
        oh? why did we need cors for that?
      • warp
        Freso: anyway, add whatever you can think of. We'll sort things into some kind of prioritized list before going to the archive.
      • nikki
        because it's easier to do with warp's multi-upload thingy
      • I think we set the format in the upload page, so the current one makes it tricky to switch, or something like that
      • I dunno, ask warp :P
      • Freso
        warp: Alright. I just noticed Nikki's CORS thing being about PNG, so I guess I don't really have anything to add anyway. :D
      • ianmcorvidae
        ah
      • warp
        but, sleep now. woosh.
      • ianmcorvidae
        ah! my thing is done
      • ianmcorvidae starts upgrade
      • jdamcd joined the channel
      • people doing slave migrations: the URL you'll need to put into upgrade.sh is:
      • rest of migration/export/etc. still in progress for master, but that makes more of step #3 possible :)
      • ianmcorvidae waits for merge-duplicate-credits, that one always takes a long time
      • ocharles
        should only be about 5 minutes?
      • ianmcorvidae: thanks, i'm doing a slave migration
      • ianmcorvidae
        longer than any of the other scripts :)
      • ocharles
        yea
      • just making sure it didn't suddenly take half an hour for you
      • ianmcorvidae
        the whole migration tends to take about 30 minutes, so no
      • ocharles: __PACKAGE__->meta->make_immutable; doesn't seem to work on Controller::Role::WikipediaExtract, complains about being unable to find make_immutable
      • ocharles: none of our other controller roles have that, seems like maybe it shouldn't be there?
      • ocharles
        oh, sorry
      • it's a role
      • ignore that comment :(
      • ianmcorvidae
        k, no worries, just wanted to check :)
      • ocharles
        http://skillsmatter.com/podcast/home/front-end-... LOL. that at 49 minutes on is hilarios
      • hilarious*
      • slave import seems to have failed...
      • track 12750848 99% 169448Error loading /tmp/MBImport-hjbaNpFo/mbdump/track: DBD::Pg::db pg_putcopyend failed: ERROR: invalid input syntax for integer: "2011-11-16 23:39:52.777558+00"
      • ianmcorvidae
        hm
      • some column orders have changed or something?
      • ocharles
        i don't know if i trust that error
      • i feel like it's out of position
      • v-2012-05-15-schema-change is the right tag, no?
      • ianmcorvidae
        should be, but you'd be the person to ask
      • did we do any fixes to that release?
      • I do seem to remember something about that
      • ocharles
        table looks fine o.o
      • ianmcorvidae
        we did
      • ocharles
        as in the track table schema looks fine
      • ianmcorvidae
        v-2012-05-15-import-fix
      • ocharles
        hm
      • ianmcorvidae
      • ocharles
        shame there's no description
      • ocharles drops and tries again
      • ruaok
        nikki: for some reason the IA never got back to me on the other image types.
      • I plan to pester the IA people in person next week
      • ocharles
        HEAD is now at 18e31b9... Ensure that the CreateTable scripts use the same column order as what the main database uses, otherwise imports will fail.
      • heh.
      • great.
      • ianmcorvidae
        I imagine that was the reaction when it happened too :P
      • ianmcorvidae noted this complication in the schema change doc, not that it'll necessarily be needed
      • RootWyrm_: if you do a slave import/migration, take note :)
      • ruaok
        ok, one import is done.
      • what branch should I migrate on?
      • ianmcorvidae
        beta.
      • RootWyrm_
        ianmcorvidae: of which?
      • ianmcorvidae
        master has the same stuff
      • RootWyrm_: if you import the 20121013 dump, make sure to use v-2012-05-15-import-fix, not v-2012-15-15-schema-change
      • otherwise the import will fail
      • as ocharles just discovered
      • RootWyrm_
        ianmcorvidae: ah, okay. Huge wall of text plus bad glasses = confusion.
      • ianmcorvidae
        whoo, master migration done
      • ocharles
        isn't converting a standalone excatly the same as converting a master?
      • like master is just a standalone conversion + triggers?
      • ianmcorvidae
        no, because of replication stuff
      • ocharles
        what 'stuff' is that
      • ianmcorvidae
        triggers :)
      • note: artist_idx_uniq_name_comment doesn't work with the 1013 dump.
      • ianmcorvidae will drop that and label_idx_uniq_name_comment and rerun not-null-comments-master; when we test imports of this data we'll need to comment out the index.
      • ruaok
        ianmcorvidae: the 1013 dump also cleanly imports on master. was that wrong?
      • ianmcorvidae
        no, it imports fine
      • it's the migration that screws it up
      • when null becomes '', then there are duplicate '' comments
      • ruaok
        k
      • ianmcorvidae running not-null-comments-master again, then I'll do a data export
      • I'm making a copy of my cleanly imported DB. then I will do the standalone with raw and edit migration
      • ianmcorvidae
        then I'll turn on ianmcorvidae.mbsandbox.org, collect stats, screw with some cover art, maybe add a release or two, and produce some replication packets :)
      • okay, master fully migrated, doing a full dump
      • ruaok
        sounds dirty.
      • make sure you light a match when you're done.
      • ianmcorvidae
        haha
      • heh, this testing is the hardest work rika has ever had to do
      • the graphs have a nice cliff starting on friday :P
      • reosarevok
        Clearly we need to strain it more
      • (normally)
      • ianmcorvidae
        yes!
      • ruaok: by the by, the archive.org search for empty buckets is 'collection:coverartarchive AND NOT format:JPEG'
      • ruaok
        k
      • ianmcorvidae
        currently 45064 entries; some of those have faulty index.json and such
      • er, 45063, forgot the CSV header :P
      • there are other buckets that are functionally worthless that are a lot harder to find (after release merges, no-longer-valid gids retain their cover art; our delete doesn't appear to work)
      • but there are probably at least 2 orders of magnitude fewer of those :P
      • ruaok starts a migration on beta
      • ruaok
        (branch beta)
      • ianmcorvidae
        yes, that should be correct
      • do you still have {artist,label}_idx_uniq_name_comment?
      • ruaok
        I have made no modifications of the code.
      • not on import, nor on upgrade.
      • ianmcorvidae
        k
      • then one of the scripts will fail, but it won't stop the whole migration
      • ruaok
        why is that still there?
      • why aren't we just dumping it at this stage?
      • ianmcorvidae
        this is the problem which is why I added dropping those indexes and re-adding them afterwards to the migration doc :)
      • ruaok
        ianmcorvidae: you're listed as flying to Seattle after BCN. You sure you got the right airport code?
      • SEN
      • ianmcorvidae
        oh, whoops, no
      • ehrgeiz has left the channel
      • ijabz joined the channel
      • the reservation itself is fine, easyjet doesn't fly to seattle, just screwed up the wiki page
      • ruaok
        I figured as much.
      • reosarevok
        :D
      • That'd have been fun
      • ianmcorvidae
        I think if I'd ended up in seattle I'd have just given up, bought some books at the japanese bookstore, and flown home :P
      • nikki
        it would however have been a really cheap flight if you could get from barcelona to seattle :P
      • ianmcorvidae
        true
      • ruaok
        ianmcorvidae: migration ran/died as expected.
      • reosarevok
        kepstin: have you booked'
      • ruaok
        just re-run that script now?
      • reosarevok
        ?
      • ianmcorvidae
        ruaok: DROP INDEX artist_idx_uniq_name_comment; DROP INDEX label_idx_uniq_name_comment;
      • reosarevok
        (remember to modify http://wiki.musicbrainz.org/MusicBrainz_Summit/12 when you're done - navap, same with your tickets)
      • ianmcorvidae
        ruaok: then rerun the script
      • ruaok
        k
      • ianmcorvidae
        without dropping the indexes it'll just fail the same way again
      • reosarevok
        And someone tell mariana to sign up there too
      • ianmcorvidae
        the label index may or may not actually be required but at this point I don't care :P
      • ruaok
        reosarevok: I just did.
      • sweet. two women at the summit.