#metabrainz

/

      • reosarevok
        Afraid most of us here are
      • What was this whole having a life thing like???
      • CatQuest
        unless the date range fits the recording's release date :D
      • Scruloose: ahaha, but only in a non-coder way :D
      • Scruloose
        Heh! What have I got myself into
      • CatQuest
        hahahaha
      • welcome to MusicBrainz <3 :D
      • Scruloose
        I think I'm going to ignore the release date thing. The idea of my plugin is to simplify the on-disk directory sorting, not to achieve perfect granularity
      • CatQuest
        sounds fine
      • Scruloose
        so chuck anything with an end date, prioritize 'Primary' if it exists, grab first locale en (or en_<region>), and if none of those exist fall back on top-level "artist" dict values
      • CatQuest
        I mean. if oyur user wants english
      • i'mnorwegian o probably woudl wnat norsk names? :D
      • CatQuest is helpful :P
      • norsk is another can of worms. so i'll leave yo uat that :D
      • Scruloose
        Oh! Yes definitely. I'm also planning to learn how to get user's locale out of beets config if they've set it, and only use en as default
      • CatQuest
        i hope it has a en-
      • erh
      • i hope it has a en_GB option as well. if i cna i use en_gb over en_US
      • if not.. miht wanna ask for it upstream.
      • hmm.. wasn't it Freso working on/with beets.. no that was whipper wanst it
      • CatQuest really goes
      • Scruloose
        Thanks for the info!
      • chaban joined the channel
      • yvanzo joined the channel
      • adhawkins
        ruaok: Can you point me at some documentation regarding tuning postgres' memory usage for the musicbrainz server? About to increase the RAM on my VM, so want to make the appropriate change there.
      • ruaok
        not sure I have a good doc to give you, but the setting that needs adjusting is shared_buffers, which should be set to 1/4 - 1/3 of available ram if you can give it that.
      • if you're using the VM, then most of the settings are tuned well enough that you only need to tweak shared_buffers.
      • revi
        reosarevok: (about the username change docs we were discussing 2 days ago) https://musicbrainz.org/doc/Account_FAQ might be a fine place to host it? :P
      • reosarevok
        I guess :D I'm just used to nobody reading the FAQs
      • But sure
      • revi
        well I know that
      • but it does carry more weight when we are asking the mighty Google.... than the community.metabrainz
      • adhawkins
        ruaok: This is my own hand-crafted VM. Thanks, will take a look at what it's set to now and increase accordingly.
      • revi
        I think I also get quite a few tickets on our helpdesk without reading "do XXX" :P
      • adhawkins
        Trying to avoid having to ask you everytime ruaok :D
      • ruaok
        heh. :) 'tis alright -- its good to be useful at times.
      • reosarevok
      • revi
        oh you have a wiki
      • revi feels like joining
      • lemmesee....
      • reosarevok
        We're planning to move some docs to github eventually, but for now they're all wiki-based
      • revi
        well I don't know about deep-inner-workings of this project but sounds good to me
      • on Wikimedia we limit people to 2 rename requests
      • (for third and more you need to have reallllllllllllly convincing reason other than "I changed my nick)
      • reosarevok
        Seems sensible :)
      • I'll transclude the change then. Thanks!
      • Scruloose
        Is it safe to assume that an artist dict returned by musicbrainzngs.get_artist_by_id will *always* have both a name and a sort-name in the top-level dict?
      • Or, to put basically the same question another way, are bolded fields in the flowchart ruaok linked mandatory?
      • reosarevok
        Yes
      • Scruloose
        Yay! I found something that's as simple as I thought!
      • reosarevok
        bitmap, yvanzo: I know I've asked before but I forget - do we know why our Selenium tests have random failures so often?
      • ruaok
        Scruloose: yes, seems so.
      • yvanzo
        reosarevok: race conditions, but I don't recall exactly.
      • KassOtsimine has quit
      • Cyna
        bitmap: so are you considering having a wrapper for mocha to html test ?
      • currently the supported export types are JUnit, Pytest and Js mocha
      • for our case js mocha would be the better option
      • reosarevok: could you link me the doc where instruction for running test for mb-server is present please ?
      • I tried finding it but failed
      • bitmap
        no plans atm, what we have is working for now so I'm not too interested in changing it yet
      • chaban has quit
      • Cyna: HACKING.md
      • BrainzGit
        [bookbrainz-site] sbvkrishna opened pull request #311 (master…revisions-not-found-message): fix(display): add 'Revision History not found' message https://github.com/bookbrainz/bookbrainz-site/p...
      • xtarget joined the channel
      • [musicbrainz-server] mwiencek merged pull request #1251 (master…eslint-rule-script): Add a script to check a specific eslint rule https://github.com/metabrainz/musicbrainz-serve...
      • [musicbrainz-server] yvanzo opened pull request #1253 (master…mbs-10389-irc-footer): MBS-10389: Add “IRC” link to footer https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-10389: Add "Chat" or "IRC" link to footer https://tickets.metabrainz.org/browse/MBS-10389
      • BrainzGit
        [musicbrainz-server] yvanzo opened pull request #1254 (master…mbs-10371-songfacts-logo): MBS-10371: Update the Songfacts logo https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-10371: Update the Songfacts logo used in the sidebar https://tickets.metabrainz.org/browse/MBS-10371
      • BrainzGit
        [musicbrainz-server] yvanzo merged pull request #1253 (master…mbs-10389-irc-footer): MBS-10389: Add “IRC” link to footer https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-10389: Add "Chat" or "IRC" link to footer https://tickets.metabrainz.org/browse/MBS-10389
      • c1e0 joined the channel
      • Cyna
        got it thanks bitmap
      • BrainzGit
        [musicbrainz-server] yvanzo merged pull request #1254 (master…mbs-10371-songfacts-logo): MBS-10371: Update the Songfacts logo https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-10371: Update the Songfacts logo used in the sidebar https://tickets.metabrainz.org/browse/MBS-10371
      • BrainzGit
        [listenbrainz-server] paramsingh merged pull request #635 (master…param/revert-username-changes): Revert "Update the musicbrainz ID of users if it has changed" https://github.com/metabrainz/listenbrainz-serv...
      • [listenbrainz-server] paramsingh merged pull request #660 (master…dependabot/pip/sphinx-2.2.1): Bump sphinx from 2.2.0 to 2.2.1 https://github.com/metabrainz/listenbrainz-serv...
      • bitmap
        reosarevok: yvanzo: whoops, lost track of time, do you wanna go over the milestone or any prs?
      • BrainzGit
        [listenbrainz-server] paramsingh merged pull request #661 (master…dependabot/pip/pytest-5.2.2): Bump pytest from 5.2.1 to 5.2.2 https://github.com/metabrainz/listenbrainz-serv...
      • reosarevok
        Heh, was just asking
      • I'm happy to - anything you've seen and want to check?
      • BrainzGit
        [musicbrainz-server] mwiencek merged pull request #1245 (master…pot-git-ls-files): Get po/Makefile prereqs with git-ls-files https://github.com/metabrainz/musicbrainz-serve...
      • reosarevok
        I'm currently fighting some new bugs with the genres code, I suspect I messed up a rebase somewhere
      • iliekcomputers
        ruaok: hi, not sure if you're around or even what timezone you're on, but i wanna chat about some LB stuff I was thinking over.
      • I'm not sure where we are on the cluster / what we decided about it being always on or vms that switch on and off.
      • ruaok
        heya!
      • iliekcomputers
        hey!
      • so we have two different patterns of getting data into the cluster for both those choices.
      • ruaok
        I'm pretty much ready to get a dedicated 8 node cluster from hetzner.
      • iliekcomputers
        we came up with incremental dumps because we thought we wouldn't be able to stream via rabbitmq
      • ruaok nods
      • and the jobs PR I have up (https://github.com/metabrainz/listenbrainz-serv...) depends on it being always on.
      • >I'm pretty much ready to get a dedicated 8 node cluster from hetzner.
      • well that was simple :P
      • then i guess we should look back into streaming listens to the cluster too
      • ruaok
        yes, to everything.
      • iliekcomputers
        not sure where that leaves this PR: https://github.com/metabrainz/listenbrainz-labs...
      • should I close it? (and reopen the spark-writer PR I have)
      • had*
      • bitmap
        reosarevok: left some small comments on https://github.com/metabrainz/musicbrainz-serve...
      • ruaok
        my life is returning to normal finally (mom & bro are on the way back to the US) and after moving the MSB mapping two steps forward, I'm going to buy us a cluster to play with.
      • iliekcomputers
        kinda same with my life.
      • ruaok
        \ø/
      • I suppose if we have an always on cluster we should aim to keep it in sync at all times.
      • bitmap
        reosarevok: the only other PR from daddy shark I haven't explicitly approved is https://github.com/metabrainz/musicbrainz-serve... though I think it's fine, just curious what yvanzo thinks
      • iliekcomputers
        right, makes sense.
      • ruaok
        also, I wonder if we can start with four nodes and not 8.
      • because I really dont understand out load demands at all.
      • our
      • iliekcomputers
        for stats calculation, 4 would be more than enough imo
      • i'm not sure about the recommendation stuff.
      • ruaok
        yeah, this has to work for stats and recommendation.
      • might be enough, not sure.
      • iliekcomputers
        the hetzner vms did stats calculation on the MLHD very fast.
      • I think getting the cluster right / making it easy to add or delete nodes is really important here.
      • ruaok
        but given that I am thinking of getting machines that are connected via virtual LAN, it gives me much more flexibility to add/remove nodes.
      • yes, so the first thing I need to do is get networking right (firewalls, not exposing ports, etc) before we ever load data on it.
      • iliekcomputers
        right.
      • ruaok
        and once that is stable, then we coordinate adding/removing nodes in a careful fashion. since we saw that randomly adding/dropping nodes causes havoc.
      • not that all havoc is bad, either, but in this case...
      • iliekcomputers
        hmm. setting up the cluster is a blocker for a few PRs for now for me.
      • will we have consul on the cluster? not sure if too early to ask?
      • for configurations etc
      • chaban joined the channel
      • ruaok
        the cluster is substantially going to be the same as it is now. just on proper dedicated hardware.
      • yvanzo
        bitmap, reosarevok: about the react-table thing, both issues have been reported upstream and have open PR in progress.
      • ruaok
        so, no consul.
      • iliekcomputers
        ah
      • bitmap
        yvanzo: great
      • iliekcomputers
        ok.
      • right, so I can test stuff in the leader cluster and it'd be basically the same.
      • cool.
      • ruaok
        another thought... when the new cluster comes online, can we add the new nodes to the existing nodes, have them propagate and then take out the old nodes?
      • yes, exactly.
      • and perhaps we can migrate the cluster to the new machines, without having to start over.
      • yvanzo
        Just found a new bug with UserAccountTabs which expects EditorT but likely receives incomplete instance when visiting user collections.
      • iliekcomputers
        I don't have enough context onn how the data storage in hdfs etc works to answer that. but i can look.
      • we made starting over really easy initially, so it shouldn't be much of a problem though.
      • unless something has changed in the past few weeks.
      • ruaok
        not really, no. starting over should be easy.
      • but I am wondering if trying to migrate the cluster allows us to learn how to add / remove nodes without breaking everything
      • also, you only used a partial data set for the MLHD, right?
      • my next question: how much disk should we get?
      • iliekcomputers
        what do we plan to do with the cluster?
      • Is mlhd still in the plan, what with the inaccuracy of mbids
      • ruaok
        MHLD, user stats, collab filtering. for starters.
      • iliekcomputers
        I think if we provide a sorta inaccurate dataset with some gamification in MusicBrainz, it might be okay.
      • ruaok
        and I would like to run the whole MHLD to see how that comes out. and then put the dataset up and let everyone poke at it. perhaps its useful, perhaps we can identify problems and maybe even fix them.
      • iliekcomputers
        maybe a gsoc idea.
      • ruaok
        have you seen the MSID -> MBID mapping?