#metabrainz

/

      • yef joined the channel
      • 2021-05-10 13056, 2021

      • yef has quit
      • 2021-05-10 13056, 2021

      • yef joined the channel
      • 2021-05-10 13003, 2021

      • sumedh joined the channel
      • 2021-05-10 13056, 2021

      • thomasross has quit
      • 2021-05-10 13016, 2021

      • Etua joined the channel
      • 2021-05-10 13036, 2021

      • Etua has quit
      • 2021-05-10 13052, 2021

      • Etua joined the channel
      • 2021-05-10 13039, 2021

      • _lucifer
        ruaok: oh! that makes sense.
      • 2021-05-10 13050, 2021

      • Etua has quit
      • 2021-05-10 13046, 2021

      • yvanzo
        CatQuest: After the next version has been released; This is a schema change release so there will be no beta update, only test.mb.o. “Fixed” is the expected resolution (because a fix has been merged), it becomes real when the ticket is closed.
      • 2021-05-10 13049, 2021

      • Zhele has quit
      • 2021-05-10 13013, 2021

      • reosarevok
        yvanzo: hi! What can I help you with if anything? :)
      • 2021-05-10 13046, 2021

      • Zhele joined the channel
      • 2021-05-10 13032, 2021

      • Etua joined the channel
      • 2021-05-10 13046, 2021

      • reosarevok
        Also, yvanzo, Freso: please do check the open syswiki PR when you have time :)
      • 2021-05-10 13027, 2021

      • Etua has quit
      • 2021-05-10 13052, 2021

      • Freso
        I don't have access to syswiki.
      • 2021-05-10 13042, 2021

      • reosarevok
        Oh, huh
      • 2021-05-10 13051, 2021

      • reosarevok
        I thought all contractors did
      • 2021-05-10 13015, 2021

      • reosarevok
        Ok, I'll PM you
      • 2021-05-10 13034, 2021

      • Rotab has quit
      • 2021-05-10 13059, 2021

      • alastairp
        zas: morning. currently ftp.acousticbrainz.org points to 138.201.228.110 and is only available over ftp. I remember that you recently set up http/s data download for musicbrainz, can you help me to move AB download to http too?
      • 2021-05-10 13034, 2021

      • alastairp
        _lucifer: hi, to confirm - is http://ftp.musicbrainz.org/pub/musicbrainz/listen… the main location of LB dumps?
      • 2021-05-10 13039, 2021

      • _lucifer
        alastairp: according to my understanding, that is the US mirror. https://musicbrainz.org/doc/MusicBrainz_Database/… mentions a couple of others as well. I think the ftp.eu one is the main location.
      • 2021-05-10 13007, 2021

      • alastairp
        when the dumps script uploads something, where does it put it? the storagebox?
      • 2021-05-10 13041, 2021

      • _lucifer
        i am not sure. ruaok would know about that.
      • 2021-05-10 13019, 2021

      • alastairp
        I found the FtpServer page on syswiki
      • 2021-05-10 13007, 2021

      • _lucifer
        alastairp: i checked and found `//u209105.your-storagebox.de/backup 2.2T 263G 1.9T 13% /mnt/dumps`. the temp dir is created in the storage box https://github.com/metabrainz/docker-server-confi… and https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-10 13034, 2021

      • _lucifer
        after this succeeds, the dump is rsynced to backup and ftp
      • 2021-05-10 13040, 2021

      • alastairp
        yep, right
      • 2021-05-10 13008, 2021

      • alastairp
        and ftp is on williams, though the documentation is a bit out of date, and AB ftp isn't there
      • 2021-05-10 13039, 2021

      • reosarevok
        Do we have a way to query how many CB reviews an editor has written?
      • 2021-05-10 13055, 2021

      • _lucifer
        reosarevok: https://critiquebrainz.readthedocs.io/api/endpoin… with the user_id parameter should do it methinks.
      • 2021-05-10 13030, 2021

      • alastairp
        perhaps it might be useful to be able to call this with a limit of 0 so that it only returns a count (and saves a db query?)? Not sure if it's worth it given the size of the db though
      • 2021-05-10 13043, 2021

      • _lucifer
        but i see its need the user's UUID not the username.
      • 2021-05-10 13008, 2021

      • alastairp
        yeah, we need to migrate to usernames on these endpoints too
      • 2021-05-10 13051, 2021

      • _lucifer
        yup, another thing to do when we migrate the user accounts to MeB.
      • 2021-05-10 13031, 2021

      • BrainzGit
        [listenbrainz-server] alastair opened pull request #1441 (master…docs-dumps-location): LB-783: Add URL to ListenBrainz dumps https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-10 13023, 2021

      • reosarevok
        So, not easily doable for now? I was wondering about linking to a user's reviews (and count) from MB profiles
      • 2021-05-10 13016, 2021

      • _lucifer
        yeah not easy currently but should be sometime soon.
      • 2021-05-10 13034, 2021

      • reosarevok
        Ok. Do you have a ticket for those changes?
      • 2021-05-10 13039, 2021

      • alastairp
        I think so
      • 2021-05-10 13040, 2021

      • reosarevok
        (I can then add a ticket with a "depends on")
      • 2021-05-10 13045, 2021

      • alastairp
        let me look
      • 2021-05-10 13017, 2021

      • yvanzo
        reosarevok: about artist award, how do you think winner/nominated/... should be represented?
      • 2021-05-10 13049, 2021

      • reosarevok
        My original idea was to just add winners, but this could be decided later on - the main thing is for it to be possible :)
      • 2021-05-10 13010, 2021

      • reosarevok
      • 2021-05-10 13029, 2021

      • yvanzo
        Could it be implemented using either separate relationship types or attributes?
      • 2021-05-10 13016, 2021

      • yvanzo
        If using separate relationship types, would ordering_attribute become useful?
      • 2021-05-10 13026, 2021

      • alastairp
        reosarevok: CB-416
      • 2021-05-10 13027, 2021

      • BrainzBot
        CB-416: Identify users by musicbrainz username, not uuid https://tickets.metabrainz.org/browse/CB-416
      • 2021-05-10 13001, 2021

      • reosarevok
        yvanzo: it *could*, but I think it would be more straightforward to just have two separate series, one for nominees and one for winners, linked to each other with a rel
      • 2021-05-10 13008, 2021

      • reosarevok
        alastairp: thanks!
      • 2021-05-10 13016, 2021

      • yvanzo
        reosarevok: so separate series for jury members, sponsors...
      • 2021-05-10 13056, 2021

      • reosarevok
        I wouldn't expect us to store sponsors. Jury members, maybe? I dunno, I also wouldn't expect to see that as part of the main series
      • 2021-05-10 13006, 2021

      • reosarevok
        The only question is winners vs nominees
      • 2021-05-10 13010, 2021

      • reosarevok
        IMO
      • 2021-05-10 13025, 2021

      • reosarevok
        And I think it's sensible to have a big showcase series for winners
      • 2021-05-10 13027, 2021

      • yvanzo
        Any other artist series in mind?
      • 2021-05-10 13032, 2021

      • reosarevok
        But dunno, we might decide something else later
      • 2021-05-10 13048, 2021

      • reosarevok
        I don't have any other personally, but I'm sure people will find uses
      • 2021-05-10 13023, 2021

      • yvanzo
        I have no specific idea either, just trying to guess potential requests :)
      • 2021-05-10 13020, 2021

      • reosarevok
      • 2021-05-10 13026, 2021

      • yvanzo
      • 2021-05-10 13027, 2021

      • BrainzBot
        MBS-11652: Add support for artist series
      • 2021-05-10 13043, 2021

      • Rotab joined the channel
      • 2021-05-10 13029, 2021

      • reosarevok
        Thanks :)
      • 2021-05-10 13048, 2021

      • yvanzo
        Updating my branch to refer to that new ticket...
      • 2021-05-10 13018, 2021

      • ruaok
        Dump tmp is on the storage box, alastairp
      • 2021-05-10 13057, 2021

      • ruaok
        Zas: is it possible for you to setup a "is the dump on the ftp site older then x hours"?
      • 2021-05-10 13011, 2021

      • ruaok
        I'd like an alert is a miss does t show up on time.
      • 2021-05-10 13019, 2021

      • alastairp
        right, the tmp. got it. but actual ftp is on williams
      • 2021-05-10 13021, 2021

      • ruaok
        A dump...
      • 2021-05-10 13029, 2021

      • ruaok
        alastairp: yes.
      • 2021-05-10 13036, 2021

      • alastairp
        For some reason we set up a different ftp for AB, I think it was because we assumed that AB dumps would get too big for williams. but I don't know where ftp.acousticbrainz.org config is
      • 2021-05-10 13048, 2021

      • ruaok
        And disk is still a problem, hence me wanting to upgrade lemmy
      • 2021-05-10 13013, 2021

      • ruaok
        Best ask zas about that.
      • 2021-05-10 13027, 2021

      • alastairp
        yep, waiting for a response :)
      • 2021-05-10 13043, 2021

      • yvanzo
        reosarevok: The description of “number” attribute refers to “work” only (see https://musicbrainz.org/relationship-attributes#n… ) while the code uses it for other entity types. Should either the code or the description be changed?
      • 2021-05-10 13000, 2021

      • Rotab has quit
      • 2021-05-10 13015, 2021

      • Rotab joined the channel
      • 2021-05-10 13051, 2021

      • _lucifer
        ruaok: LB-884
      • 2021-05-10 13052, 2021

      • BrainzBot
        LB-884: Listens count not updated after old listens import https://tickets.metabrainz.org/browse/LB-884
      • 2021-05-10 13003, 2021

      • sumedh has quit
      • 2021-05-10 13028, 2021

      • zas
        alastairp, ruaok: hmmm, there are different things here: 1) http(s?) in addition of ftp 2) dump older than x hours 3) lemmy's upgrade
      • 2021-05-10 13052, 2021

      • alastairp
        zas: yes, 3 things
      • 2021-05-10 13029, 2021

      • zas
        about 1) I didn't set up AB ftp, and current williams's FTP to HTTP was a quick hack, so we perhaps need to rethink how we serve files, and unify
      • 2021-05-10 13016, 2021

      • alastairp
        right. I just realised that I looked on boingo (AB web) but not on frank (AB DB), and I just found that we're running metabrainz/docker-anon-ftp:latest on frank for AB FTP
      • 2021-05-10 13034, 2021

      • alastairp
        see my comment just earlier - we host AB here instead of on williams because of the size of AB dumps
      • 2021-05-10 13023, 2021

      • alastairp
        possibly we could have an equivalent docker-anon-http image or similar that can host http dumps in the same way?
      • 2021-05-10 13025, 2021

      • zas
        about 2) basically you want an alert, I'll check what I can do, but perhaps the simplest would be to touch a file (lastupdated) on each dump, and check that, dunno
      • 2021-05-10 13056, 2021

      • zas
        alastairp: yes, the problem is mainly about https certs
      • 2021-05-10 13003, 2021

      • alastairp
        understood
      • 2021-05-10 13010, 2021

      • alastairp
        for the current issue (users can't download submission tool), I will move these to williams FTP, as the filesize isn't large. I'm happy to wait a while until we come up with a better solution for http/s download
      • 2021-05-10 13026, 2021

      • alastairp
        for your answer to 2), good idea. if we don't have a lastupdated file for LB dumps then we should definitely add that
      • 2021-05-10 13010, 2021

      • zas
        ruaok: about lemmy's upgrade, how do you want to proceed?
      • 2021-05-10 13036, 2021

      • zas
        alastairp: about ftp.acoustibrainz.org to http, the problem is that https://acoustibrainz.org is using a Letsencrypt cert (generated on kiki/herb), but I guess we want direct access to files via http for serving those files, so it will require a different hostname, and its own cert, or a way to generate LE certs on frank. None of those are great (the nginx container we use for mb ftp is using hardcoded mb cert)
      • 2021-05-10 13004, 2021

      • alastairp
        can we do https://data.acousticbrainz.org terminated on kiki/herb, with http proxy_pass to an nginx instance on frank?
      • 2021-05-10 13039, 2021

      • alastairp
        ah, I see that's basically your comment " so it will require a different hostname, and its own cert"
      • 2021-05-10 13052, 2021

      • zas
        we can, actually that's a simpler approach, but I'm not sure if it's a good idea, because gateways are already quite busy, and those downloads don't stricly need to pass through them
      • 2021-05-10 13051, 2021

      • alastairp
        right, understood.
      • 2021-05-10 13027, 2021

      • ruaok
        zas: lets do lemmy's upgrade later this week. it isn't urgent -- I have too many loose ends to pick this up now.
      • 2021-05-10 13037, 2021

      • ruaok
        the FTP check sooner, though.
      • 2021-05-10 13007, 2021

      • alastairp
        !m everyone who works in MeB in not their native language
      • 2021-05-10 13007, 2021

      • BrainzBot
        You're doing good work, everyone who works in MeB in not their native language!
      • 2021-05-10 13013, 2021

      • zas
        my idea is to make a LE-compatible nginx image for this (in fact, I tried for williams FTP, but ended using hardcoded certs because we have have a wildcard one for this domain).
      • 2021-05-10 13021, 2021

      • ruaok
        alastairp: danke. :)
      • 2021-05-10 13024, 2021

      • alastairp
        I'm trying to have a technical discussion with people in Spanish, and it's not going well
      • 2021-05-10 13034, 2021

      • ruaok
        rough.
      • 2021-05-10 13051, 2021

      • ruaok
        zas: this dir should be checked to make sure that the dump is no more than 16 days old: ftp://ftp.eu.metabrainz.org/pub/musicbrainz/liste…
      • 2021-05-10 13023, 2021

      • ruaok
        and the latest here should be no more than 26 hours old ftp://ftp.eu.metabrainz.org/pub/musicbrainz/liste…
      • 2021-05-10 13002, 2021

      • alastairp
        ruaok: did you see zas' suggestion? We could add a LATEST file with the date of the dump. that means that such a check would just need to read this file and parse the date
      • 2021-05-10 13018, 2021

      • ruaok
        yeah, I knew that was coming. I was hoping to not add more on my plate.
      • 2021-05-10 13050, 2021

      • alastairp
        is it correct to call osuosl "main" and hetzner "mirror"? Or should we just say Protocol (Location)? https://usercontent.irccloud-cdn.com/file/WDgGKB7…
      • 2021-05-10 13036, 2021

      • ruaok
        germany is main. OSU is mirror. now that we have unlimited bandwidth, we can easily do this.
      • 2021-05-10 13003, 2021

      • alastairp
        will change
      • 2021-05-10 13009, 2021

      • Sophist-UK joined the channel
      • 2021-05-10 13008, 2021

      • BrainzGit
        [listenbrainz-server] alastair merged pull request #1441 (master…docs-dumps-location): LB-783: Add URL to ListenBrainz dumps https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-10 13035, 2021

      • BrainzGit
        [listenbrainz-server] alastair closed pull request #1435 (master…dependabot/pip/master/eventlet-0.31.0): [Security] Bump eventlet from 0.30.2 to 0.31.0 https://github.com/metabrainz/listenbrainz-server…
      • 2021-05-10 13002, 2021

      • reosarevok
        yvanzo: the description, will change
      • 2021-05-10 13055, 2021

      • zas
      • 2021-05-10 13057, 2021

      • zas
        still experimental, but it works (with LE), I'll refine the stuff a bit though, very rough atm
      • 2021-05-10 13020, 2021

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #2096 (master…MBS-11650): MBS-11650: Add tag and rating statistics to profile page https://github.com/metabrainz/musicbrainz-server/…
      • 2021-05-10 13020, 2021

      • Mr_Monkey
        Is anyone using test.LB.org to test some stuff out?
      • 2021-05-10 13020, 2021

      • Mr_Monkey
        I wanted to build and deploy PR #1436 (nodejs upgrade) to test it out.
      • 2021-05-10 13057, 2021

      • zas
        alastairp: I hit the LE cert rate limit :(, I'll need to figure why
      • 2021-05-10 13054, 2021

      • alastairp
        no problem, thanks. To confirm, this URL will be exactly the same as current ftp but available over http/s? On frank
      • 2021-05-10 13015, 2021

      • zas
        Yes
      • 2021-05-10 13040, 2021

      • alastairp
        Mr_Monkey: I'm not using it - in fact I think last week we turned it and beta off, with the idea that the next person who needed it would turn it on
      • 2021-05-10 13053, 2021

      • Mr_Monkey
        Yep, I remembered that, and test is back up, so I figured maybe someone was using it
      • 2021-05-10 13026, 2021

      • alastairp
        ah
      • 2021-05-10 13059, 2021

      • BrainzGit
        [musicbrainz-server] reosarevok opened pull request #2097 (master…MBS-11578): MBS-11578: Link tracks in RecordingTrackDifferentName report https://github.com/metabrainz/musicbrainz-server/…
      • 2021-05-10 13030, 2021

      • BrainzGit
        [acousticbrainz-server] alastair opened pull request #398 (master…downloads-http): Move download links to http https://github.com/metabrainz/acousticbrainz-serv…
      • 2021-05-10 13008, 2021

      • ruaok
        Mr_Monkey: trying to build a new branch off master: https://gist.github.com/mayhem/ec9db506db7c5879fa…
      • 2021-05-10 13012, 2021

      • ruaok
        any ideas?
      • 2021-05-10 13023, 2021

      • ruaok
        zas: never mind the FTP check. I've implemented it already.
      • 2021-05-10 13046, 2021

      • Mr_Monkey
        ruaok: Not much to go from with those logs, but the error code I haven't seen before. I found this: https://github.com/nimiq/core-js/issues/295#issue…
      • 2021-05-10 13057, 2021

      • Mr_Monkey
        Possibly some resource constraint?
      • 2021-05-10 13029, 2021

      • Mr_Monkey
      • 2021-05-10 13055, 2021

      • Mr_Monkey
        Have you tried turning it off and on again ? :p
      • 2021-05-10 13012, 2021

      • Mr_Monkey
        I had to do a `docker system prune` myself this morning…
      • 2021-05-10 13030, 2021

      • Mr_Monkey
        (unrelated issue)
      • 2021-05-10 13044, 2021

      • alastairp
        'Killed' looks suspiciously like out of memory
      • 2021-05-10 13056, 2021

      • Mr_Monkey
        Indeed
      • 2021-05-10 13056, 2021

      • ruaok restarts docker
      • 2021-05-10 13006, 2021

      • ruaok
        hopefully a replacement machine will arrive this week or next
      • 2021-05-10 13028, 2021

      • ruaok grumble pandemic grumble chip shortage grumble
      • 2021-05-10 13049, 2021

      • BrainzGit
        [musicbrainz-server] yvanzo opened pull request #2098 (master…mbs-11652): MBS-11652: Add support for artist series https://github.com/metabrainz/musicbrainz-server/…
      • 2021-05-10 13003, 2021

      • ruaok
        killed again. I can't wait to be rid of mac os. My linux box at home is a much better dev env.