#metabrainz

/

      • monkey
        Thanks !
      • 2023-03-20 07907, 2023

      • mayhem
        thanks for running the meeting!
      • 2023-03-20 07920, 2023

      • TOPIC: 🎉 MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | BookBrainz: #bookbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews
      • 2023-03-20 07923, 2023

      • mayhem
        whatchagot, lucifer ?
      • 2023-03-20 07938, 2023

      • reosarevok
        mayhem or rdswift: Since I'm traveling next Monday, maybe one of you two can run the meeting? :)
      • 2023-03-20 07900, 2023

      • reosarevok
        I might be able to attend, but I wouldn't dare say I can run it : )
      • 2023-03-20 07903, 2023

      • aerozol
        bitmap: chaban did some more digging into why external links wont load: https://tickets.metabrainz.org/browse/MBS-12996
      • 2023-03-20 07904, 2023

      • BrainzBot
        MBS-12996: 'External Links' fails to load when creating new artist
      • 2023-03-20 07905, 2023

      • mayhem
        I have a dinner I am supposed to be at, but I think I can manage, since the dinner starts right at the end of themeeting.
      • 2023-03-20 07921, 2023

      • mayhem
        I can run the meeting, but might need to pass the baton if it runs longer than 45 minutes.
      • 2023-03-20 07923, 2023

      • lucifer
        mayhem: about the listens schema migration, i ran the script initially and made a few fixes. it seems fine rn but we should probably be careful with it.
      • 2023-03-20 07942, 2023

      • mayhem
        were the diskspace alerts related to that>
      • 2023-03-20 07943, 2023

      • mayhem
        ?
      • 2023-03-20 07954, 2023

      • lucifer
        so question is how careful and how do we make sure that its actually correct?
      • 2023-03-20 07959, 2023

      • lucifer
        yes
      • 2023-03-20 07916, 2023

      • mayhem
        run it, then point test.lb to the new table?
      • 2023-03-20 07927, 2023

      • lucifer
        there's an old volume from last year's timescale migration which can be removed to free up 0.5TB. i'll do that.
      • 2023-03-20 07935, 2023

      • mayhem
        ok
      • 2023-03-20 07951, 2023

      • mayhem
        also, inserting the rickroll into all daily jams needs to happen next week.
      • 2023-03-20 07900, 2023

      • mayhem
        I'll crank out a PR for it.
      • 2023-03-20 07910, 2023

      • lucifer
        well yes, but that needs a copy of listen metadata and other tables.
      • 2023-03-20 07918, 2023

      • bitmap
        Pratha-Fish: fyi the mechanize library doesn't execute javascript, which is probably why it doesn't see those interactive fields. maybe there is a way to inject those fields yourself
      • 2023-03-20 07946, 2023

      • lucifer
        that refer values based on the new table and not the old one. there is also a datatype difference in listened_at column in old and new schema.
      • 2023-03-20 07911, 2023

      • lucifer
        i'll create a copy of those tables for the time being then.
      • 2023-03-20 07933, 2023

      • Pratha-Fish
        bitmap: Hi, thanks a lot for the ping.
      • 2023-03-20 07933, 2023

      • Pratha-Fish
        I'll be happy to try out other methods too. Can you recommend me a place to get started? I remember something about mechanize not supporting js a while ago, and some people seemed to recommend selenium instead
      • 2023-03-20 07937, 2023

      • lucifer
        do we want to go one step further and insert to both tables in prod for a while?
      • 2023-03-20 07940, 2023

      • mayhem
        what if we rent a decent server for a week and isolate the work there?
      • 2023-03-20 07940, 2023

      • Pratha-Fish
        But ig that would be too big of a shift
      • 2023-03-20 07903, 2023

      • bitmap
        aerozol: thanks, I wasn't able to reproduce it earlier but I'll be sure to check again
      • 2023-03-20 07932, 2023

      • mayhem
        inserting to both tables seems like a good idea, but may be a bit of work.
      • 2023-03-20 07944, 2023

      • yvanzo
        zas: I forgot we planned to restart trille today, is it too late for you? Otherwise which time would work for you tomorrow?
      • 2023-03-20 07946, 2023

      • lucifer
        that a new server idea also sounds good to me.
      • 2023-03-20 07901, 2023

      • lucifer
        yeah i guess it between either more work for zas or me. :)
      • 2023-03-20 07905, 2023

      • bitmap
        Pratha-Fish: yes selenium is an option, but maybe there's a way to inject the fields yourself with the mechanize api. otherwise you can make POST request manually with the requests library or something
      • 2023-03-20 07908, 2023

      • mayhem
        if we rented a new server, we could make a complete clone, convert the data and then test it properly.
      • 2023-03-20 07918, 2023

      • mayhem
        renting another server is not much work.
      • 2023-03-20 07933, 2023

      • lucifer
        cool sounds good to me
      • 2023-03-20 07934, 2023

      • mayhem
        with all the ansible stuff that zas and atj have been doing I would expect this work to be less than an hour.
      • 2023-03-20 07942, 2023

      • mayhem
        ok, when do you want to do this>
      • 2023-03-20 07943, 2023

      • mayhem
        ?
      • 2023-03-20 07955, 2023

      • lucifer
        anytime this week sounds good
      • 2023-03-20 07903, 2023

      • Pratha-Fish
        bitmap: I see. Let me check if there's still a way to use mechanize in this case.. Or I'll just try to emulate a request instead
      • 2023-03-20 07918, 2023

      • mayhem
        this week gets more dicey for me. traveling wed - fri.
      • 2023-03-20 07924, 2023

      • mayhem
        but I can get you a server right now.
      • 2023-03-20 07928, 2023

      • lucifer
        i see, well next week then i guess
      • 2023-03-20 07947, 2023

      • mayhem
        do you need me to do this work? if so, then yes, next week. if not, let me get you a server.
      • 2023-03-20 07947, 2023

      • mayhem
      • 2023-03-20 07950, 2023

      • mayhem
        pick one!
      • 2023-03-20 07955, 2023

      • Pratha-Fish
        P.S. bitmap when do you clock out for today? I'll try my best to wrap up all my queries before that :)
      • 2023-03-20 07901, 2023

      • mayhem
        2 x 4.0 TB Enterprise HDD in a mirror configuration, so 8TB?
      • 2023-03-20 07910, 2023

      • mayhem
        clock out??
      • 2023-03-20 07919, 2023

      • lucifer
        yeah that sounds about right
      • 2023-03-20 07923, 2023

      • mayhem wonders if metabrainzer have clocks.
      • 2023-03-20 07932, 2023

      • mayhem
        64GB ram enough>
      • 2023-03-20 07941, 2023

      • Pratha-Fish
        #clockbrainz
      • 2023-03-20 07946, 2023

      • mayhem
        zas: ping
      • 2023-03-20 07902, 2023

      • mayhem
        how much disk space do you need lucifer ?
      • 2023-03-20 07904, 2023

      • lucifer
        64 should be enough.
      • 2023-03-20 07915, 2023

      • lucifer
        2 TB at least would be good.
      • 2023-03-20 07924, 2023

      • mayhem
        4TB should be good then?
      • 2023-03-20 07927, 2023

      • lucifer
        yup
      • 2023-03-20 07931, 2023

      • mayhem
        then we can use a standard raid-1 setup.
      • 2023-03-20 07934, 2023

      • bitmap
        Pratha-Fish: I'll be around for a while, might be afk for a bit to eat but I will return :)
      • 2023-03-20 07958, 2023

      • lucifer
        cool
      • 2023-03-20 07904, 2023

      • Pratha-Fish
        That's cool :)
      • 2023-03-20 07904, 2023

      • Pratha-Fish
        I am thinking of pulling an all nighter today, so I might be around for the next ~5 hours
      • 2023-03-20 07912, 2023

      • mayhem
        lucifer: do you want the server to be setup to be a MeB machine?
      • 2023-03-20 07922, 2023

      • mayhem
        read: zas and/atj doing that.
      • 2023-03-20 07929, 2023

      • mayhem
        or do you just want the server and set it up as you like?
      • 2023-03-20 07951, 2023

      • lucifer
        mayhem: as long as i can copy data from gaga to it easily. anyway is fine.
      • 2023-03-20 07912, 2023

      • lucifer
        because the script will read listens from table on gaga and write to the database on this new machine
      • 2023-03-20 07914, 2023

      • mayhem
        so virtual network setup? or are you ok with scp?
      • 2023-03-20 07933, 2023

      • mayhem
        or SSH tunnels in that case?
      • 2023-03-20 07934, 2023

      • lucifer
        don't think scp will work.
      • 2023-03-20 07941, 2023

      • lucifer
        tunnels could work yeah
      • 2023-03-20 07953, 2023

      • lucifer
        fine with tunnels if vlan is more work
      • 2023-03-20 07957, 2023

      • mayhem
        but it sounds like you prefer the viurtual network, yes>
      • 2023-03-20 07902, 2023

      • monkey
        reosarevok: I reckon I need some sort of special access in Discourse to get user information by external id as described in MBS-11937 ?
      • 2023-03-20 07902, 2023

      • monkey
        https://community.metabrainz.org/u/by-external/mu… > "You are not permitted to view the requested resource."
      • 2023-03-20 07903, 2023

      • BrainzBot
        MBS-11937: Link to Discourse profile from MB profile https://tickets.metabrainz.org/browse/MBS-11937
      • 2023-03-20 07914, 2023

      • lucifer
        yup vlan sounds more robust
      • 2023-03-20 07915, 2023

      • mayhem
        let me get zas/atj to chime in.
      • 2023-03-20 07917, 2023

      • mayhem
        ok.
      • 2023-03-20 07926, 2023

      • monkey
        Or is it some other url I should be hitting?
      • 2023-03-20 07945, 2023

      • lucifer
        cool
      • 2023-03-20 07949, 2023

      • yvanzo
        reosarevok: I’m always available to help with weekly meeting, email support, account admin or anything else if needed, especially as you take some weeks off.
      • 2023-03-20 07953, 2023

      • monkey
        This one does get the info I need, but tied to username: https://community.metabrainz.org/u/mr_monkey.json
      • 2023-03-20 07910, 2023

      • atj
        won't you just use ssh over the vlan anyway?
      • 2023-03-20 07926, 2023

      • yvanzo
        mayhem: Will the new datasets UI require any change to the MB mirror setup which downloads dumps from FTP?
      • 2023-03-20 07935, 2023

      • mayhem
        atj: why would that be necessary?
      • 2023-03-20 07947, 2023

      • atj
        is it db connections rather than file transfer?
      • 2023-03-20 07957, 2023

      • mayhem
        yvanzo: no. I think the changes will mostly be limited to wikidoc pages about where to download the data.
      • 2023-03-20 07904, 2023

      • mayhem
        atj: db connections.
      • 2023-03-20 07902, 2023

      • atj
        i don't think the pg docker containers are listening on the vlan yet?
      • 2023-03-20 07928, 2023

      • mayhem
        lucifer: ^^
      • 2023-03-20 07938, 2023

      • lucifer
        nope they arent
      • 2023-03-20 07906, 2023

      • lucifer
        that the was the next thing i was going to talk about, we need a short downtime to restart database to listen on vlan as well
      • 2023-03-20 07922, 2023

      • mayhem
        ah, ok, so atj , they will soon. :)
      • 2023-03-20 07952, 2023

      • mayhem
        lucifer: prep the PRs needed for that and then we can do that change... tomorrow evening?
      • 2023-03-20 07958, 2023

      • lucifer
        let's plan that for tomorrow since you'll travelling wed-fri
      • 2023-03-20 07901, 2023

      • lucifer
        yup i'll do that
      • 2023-03-20 07945, 2023

      • mayhem
        ok, sounds good.
      • 2023-03-20 07909, 2023

      • mayhem
        I'm in a cooking class in the first half of the day, and I suppose I should be finishing in about 26 hours from now.
      • 2023-03-20 07917, 2023

      • mayhem
        I hope that timing works.
      • 2023-03-20 07900, 2023

      • lucifer
        so we start at 26 hours from now?
      • 2023-03-20 07928, 2023

      • mayhem
        can we be flexible about this. I am unsure if I will be exactly available then.
      • 2023-03-20 07928, 2023

      • atj
        this is a bizarre conversation
      • 2023-03-20 07942, 2023

      • lucifer
        sure
      • 2023-03-20 07945, 2023

      • mayhem
        how is this different from normal, atj ?
      • 2023-03-20 07956, 2023

      • mayhem
        lucifer: ok, no problem then.
      • 2023-03-20 07927, 2023

      • jasje has quit
      • 2023-03-20 07934, 2023

      • atj
        i guess i've not been fully indoctrinated yet ;)
      • 2023-03-20 07907, 2023

      • yvanzo
        mayhem: Ok, I would suggest to add "custom" to MB DB dump’s description - "XZ compressed custom PostgreSQL table dumps" - since it’s neither created with "pg_dump" nor directly loadable with "psql" as people frequently expect it to be, it actually requires MB Server (or alternatively mbdata mirroring scripts) to load it properly.
      • 2023-03-20 07914, 2023

      • aerozol
        pratha-fish: jasje: jivte: Are you on the forums? With different usernames? Let me know and I’ll try remember to tag you in the meeting notes (https://community.metabrainz.org/t/notes-from-met…)
      • 2023-03-20 07908, 2023

      • Pratha-Fish
        aerozol: Hi, I am on the forums with the username "snaekboi"
      • 2023-03-20 07922, 2023

      • aerozol
        :D
      • 2023-03-20 07926, 2023

      • jivte
        aerozol: My username is jivteshs20
      • 2023-03-20 07935, 2023

      • mayhem
        yvanzo: ah, interesting. can you please summarize that in an email to me? Otherwise I will forget.
      • 2023-03-20 07942, 2023

      • mayhem needs to run off in a sec
      • 2023-03-20 07946, 2023

      • aerozol
        Thanks tīmu
      • 2023-03-20 07901, 2023

      • yvanzo
        zas: tomorrow 4pm UTC would work for me resetting trille.
      • 2023-03-20 07918, 2023

      • zas
        yvanzo: ok
      • 2023-03-20 07937, 2023

      • mayhem
        zas/atj: how much work is it to take a new server from hetzner and get the VLAN setup?
      • 2023-03-20 07900, 2023

      • pixelpenguin uploaded an image: (37KiB) < https://libera.ems.host/_matrix/media/v3/download/matrix.org/YrpulwstFDxnqwJYMFdEHejE/Screenshot%20from%202023-03-21%2000-37-37.png >
      • 2023-03-20 07903, 2023

      • mayhem
        lucifer needs a machine for a few days only, so one of the server auction servers would work great, but we need VLAN setup
      • 2023-03-20 07910, 2023

      • aerozol
        reminisces about when akshaaatt started and mayhem told him to use a username with less a’s and t’s. Guess you won that one akshaaatt :P (or am I misremembering?)
      • 2023-03-20 07913, 2023

      • pixelpenguin
        mayhem: I completed the setup for data-set-hostler,... (full message at <https://libera.ems.host/_matrix/media/v3/download/libera.chat/4e41b85161e34769615368be1b3b297a1e99143c>)
      • 2023-03-20 07925, 2023

      • mayhem
        pixelpenguin: yay!
      • 2023-03-20 07912, 2023

      • zas
        mayhem: to get vlan up it is a matter of minutes (we need to add it on hetzner interface, and run ansible with proper config for this server)
      • 2023-03-20 07937, 2023

      • mayhem
        great, let me order the server then
      • 2023-03-20 07944, 2023

      • akshaaatt
        Or maybe I could do aksha9273939t91739 as a username, aerozol?
      • 2023-03-20 07915, 2023

      • akshaaatt
        I am happy I have the same username on every platform ever😂
      • 2023-03-20 07925, 2023

      • aerozol
        Doesn’t look secure enough, needs some symbols I think
      • 2023-03-20 07941, 2023

      • akshaaatt
        XD
      • 2023-03-20 07956, 2023

      • mayhem
        zas: server ordered. and you or atj do the minimal user and VLAN setup on that machine once we get credentials?
      • 2023-03-20 07954, 2023

      • mayhem
        lets call it quest. atj's choice.
      • 2023-03-20 07924, 2023

      • mayhem
        gsoc page finally fixed, lucifer. :)
      • 2023-03-20 07950, 2023

      • lucifer
        awesome
      • 2023-03-20 07904, 2023

      • mayhem
        ok, really. I need to find food. back in a while.
      • 2023-03-20 07941, 2023

      • jasje joined the channel
      • 2023-03-20 07909, 2023

      • jasje
        aerozol: mine is Jasjeet
      • 2023-03-20 07957, 2023

      • aerozol
        Thanks!
      • 2023-03-20 07914, 2023

      • jasje
        yippies
      • 2023-03-20 07931, 2023

      • akshaaatt
        jasje: what’s with the replacement about sharedpref?
      • 2023-03-20 07915, 2023

      • jasje
      • 2023-03-20 07930, 2023

      • akshaaatt
        I’m looking at it until we have reasons to migrate
      • 2023-03-20 07933, 2023

      • jasje
        basically it provides sharedPrefs in form of flows
      • 2023-03-20 07937, 2023

      • akshaaatt
        Not looking*
      • 2023-03-20 07955, 2023

      • jasje
        since normal variables used to trigger changes in XML