#metabrainz

/

      • tandy1000 joined the channel
      • 2022-03-09 06813, 2022

      • elomatreb[m] joined the channel
      • 2022-03-09 06825, 2022

      • Lotheric joined the channel
      • 2022-03-09 06854, 2022

      • aerozol
        akshaaatt, Monkey: have a look at the figma mockup when you get a chance and give me your feedback
      • 2022-03-09 06833, 2022

      • aerozol
      • 2022-03-09 06831, 2022

      • aerozol
        I'm getting a headache with how we've got multiple designs going at the same time - LB vs MB vs BB
      • 2022-03-09 06849, 2022

      • aerozol
        I'd be interested in knowing which look you think 'feels best' for MetaBrainz right now (e.g. current LB or akshaaatt's modern MB mockup)
      • 2022-03-09 06856, 2022

      • aerozol
        (others as well)
      • 2022-03-09 06800, 2022

      • gcrkrause2 has quit
      • 2022-03-09 06851, 2022

      • Shubh joined the channel
      • 2022-03-09 06832, 2022

      • gcrkrause2 joined the channel
      • 2022-03-09 06830, 2022

      • lucifer
        zas: does exim keep complete logs of messages sent in the last week or so?
      • 2022-03-09 06823, 2022

      • lucifer
      • 2022-03-09 06803, 2022

      • lucifer
        for instance i am trying to view the headers/body of a message sent earlier today but get this error.
      • 2022-03-09 06846, 2022

      • lucifer
        ok it appears exim does not retain sent email :/
      • 2022-03-09 06811, 2022

      • akshaaatt
        Wow aerozol! Love the additions you've made! 👌👌👌
      • 2022-03-09 06835, 2022

      • akshaaatt
        aerozol the modern MB look looks fire to me and I think we should be keen to have it that way 👌
      • 2022-03-09 06813, 2022

      • aerozol
        I would expect whatever styles we settle on to get used, more or less, in future MetaBrainz develpments, so it's good to make sure
      • 2022-03-09 06818, 2022

      • aerozol
        Including LB etc
      • 2022-03-09 06857, 2022

      • akshaaatt
        aerozol I think LB would be flexible with any design changes you might have. We even need to better work on some CSS for LB rn.
      • 2022-03-09 06846, 2022

      • zas
        lucifer: exim doesn't retain emails but if they are still in queues. I guess that's exim relay for lb running as container, so docker logs is your friend.
      • 2022-03-09 06841, 2022

      • zas
        lucifer: see my PM about this
      • 2022-03-09 06837, 2022

      • aerozol
        Great! LB has some square buttons but also some very rounded elements in-page so I'm trying to find some solid ground
      • 2022-03-09 06830, 2022

      • texke joined the channel
      • 2022-03-09 06808, 2022

      • gayangi81 joined the channel
      • 2022-03-09 06840, 2022

      • zas
        yvanzo, atj: I don't understand the use of sudo -A in sudo -A rsync -avh --progress --rsync-path='sudo -A rsync' user@remote:/remote_path/ /local_path command, if you want to preserve env, use -E, but -A is to ask pass, which doesn't look needed. Now I'm not sure to understand what you are trying to do, what do you try to copy, from where to where?
      • 2022-03-09 06815, 2022

      • zas
        atj: SSH_AUTH_SOCK to env_keep -> not sure if this is needed if one uses sudo -E: try sudo -E printenv | grep SSH
      • 2022-03-09 06847, 2022

      • yvanzo
        zas: Thanks, -E is what I intended to use actually ^^
      • 2022-03-09 06806, 2022

      • yvanzo
        zas: Copied the docker volumes musicbrainz-sitemaps and musicbrainz-sitemaps-logs from yehudi to rakim on yesterday.
      • 2022-03-09 06822, 2022

      • yvanzo
        O’Moin
      • 2022-03-09 06858, 2022

      • gayangi54 joined the channel
      • 2022-03-09 06847, 2022

      • yvanzo
        zas: Are port numbers in PORT_LIST supposed to be unique? I guess that sharing the same number for both musicbrainz-redis-cache and musicbrainz-redis-cache-test doesn’t clash because these containres are not running on the same host. Should those be differentiated instead?
      • 2022-03-09 06852, 2022

      • gayangi81 has quit
      • 2022-03-09 06821, 2022

      • zas
        they are supposed to be unique to avoid clashes when moving services around, but they don't have to, only the key have to be unique (it works like a dict)
      • 2022-03-09 06804, 2022

      • yvanzo
        Moved the container musicbrainz-sitemaps (and its redis instance) to rakim yesterday.
      • 2022-03-09 06850, 2022

      • yvanzo
        Planning to move musicbrainz-redis-cache today.
      • 2022-03-09 06836, 2022

      • yvanzo
        Moving musicbrainz-redis-store will require a downtime, will plan with bitmap and reosarevok later today.
      • 2022-03-09 06852, 2022

      • zas
        ok, perfect, tell me when all is done
      • 2022-03-09 06838, 2022

      • zas
        atj: what's with shorewall & IPv6?
      • 2022-03-09 06842, 2022

      • atj
        zas: Shorewall is IPv4 only, there is a separate package called Shorewall6 that handles IPv6
      • 2022-03-09 06804, 2022

      • atj
        they share a lot of code, and the configuration is almost identical
      • 2022-03-09 06822, 2022

      • atj
        yvanzo: I can confirm that "sudo -E" works, did you try that before I made the env_keep change?
      • 2022-03-09 06845, 2022

      • zas
        atj: I guess we need it to be deployed then
      • 2022-03-09 06802, 2022

      • yvanzo
        atj: No I just didn’t type the correct option.
      • 2022-03-09 06810, 2022

      • atj
        yvanzo: ok, zas solved it then :) I've removed the env_keep option
      • 2022-03-09 06823, 2022

      • yvanzo
        atj: yes, thank for your help :)
      • 2022-03-09 06830, 2022

      • atj
        no worries. I clearly need to brush on my sudo knowledge!
      • 2022-03-09 06850, 2022

      • mayhem
      • 2022-03-09 06856, 2022

      • mayhem
        lucifer: ^^ I think we need to delay how long before we send the mail about a user's spotify connection being disabled. it seems that a lot of people are not re-connecting. And it makes sense, just because spotify wobbles, our users shouldn't have to do anything.
      • 2022-03-09 06847, 2022

      • lucifer
        alastairp: more fun `timescaledb-2-2.6.0-postgresql-13` is present in the repo but 2.2.0 is not :/ for ubuntu.
      • 2022-03-09 06822, 2022

      • lucifer
        mayhem: i see makes sense.
      • 2022-03-09 06804, 2022

      • lucifer
        mayhem: it looks like from the errors in admin ui that the spotify issues were caught here https://github.com/metabrainz/listenbrainz-server…
      • 2022-03-09 06849, 2022

      • lucifer
        those are invalid grant errors so we remove the user's access token we have in that case and can't reconnect without user intervention.
      • 2022-03-09 06816, 2022

      • lucifer
        for other errors, we don't remove the token so could retry connecting ourself.
      • 2022-03-09 06837, 2022

      • mayhem
        that really sucks.
      • 2022-03-09 06837, 2022

      • mayhem
        I had to log back into spotify to make it work, I wonder if everyone had to do that.
      • 2022-03-09 06822, 2022

      • lucifer
        yeah they'll need to do that if they weren't already logged in to spotify or spotify login session got borked too.
      • 2022-03-09 06849, 2022

      • atj
      • 2022-03-09 06848, 2022

      • lucifer
        :(
      • 2022-03-09 06859, 2022

      • atj
        was this a spotify error or did LB handle the downtime badly?
      • 2022-03-09 06817, 2022

      • mayhem
        spotify was down.
      • 2022-03-09 06835, 2022

      • lucifer
        spotify had a downtime and the error reported by spotify was probably misconstrued as the user having revoked access to LB.
      • 2022-03-09 06836, 2022

      • mayhem
        I dont think any of this was our problem. (but dont quote me on that)
      • 2022-03-09 06830, 2022

      • gayangi54 has quit
      • 2022-03-09 06830, 2022

      • atj
        so the user then tried to grant access again to LB, which actually revoked access, but Spotify was down so it didn't work anyway?
      • 2022-03-09 06816, 2022

      • lucifer
        when we detect access revocation eerror, we delete the access tokens on our side and mail the user to reconnect again.
      • 2022-03-09 06812, 2022

      • lucifer
        but spotify was down so yeah it didn't work and we cannot reconnect those users ourself without them reapproving spotify side because we deleted the token
      • 2022-03-09 06813, 2022

      • mayhem
        I wonder if there is any way to retry after the revocation error... because it may have been erroneous/
      • 2022-03-09 06824, 2022

      • lucifer
        we retry 5 times.
      • 2022-03-09 06853, 2022

      • mayhem
        which what delay?
      • 2022-03-09 06815, 2022

      • BrainzGit
        [critiquebrainz] release 03v-2022-03-09.0 has been published by 14github-actions[bot]: https://github.com/metabrainz/critiquebrainz/rele…
      • 2022-03-09 06826, 2022

      • texke has quit
      • 2022-03-09 06834, 2022

      • mayhem
        because maybe we should have some backoff -- and only ever give up after X hours.
      • 2022-03-09 06838, 2022

      • alastairp
        hi
      • 2022-03-09 06840, 2022

      • texke joined the channel
      • 2022-03-09 06845, 2022

      • mayhem
        essentially allowing for this case.
      • 2022-03-09 06850, 2022

      • alastairp
        lucifer: hmm, is that a specific apt package?
      • 2022-03-09 06803, 2022

      • lucifer
        yes that makes sense. we don't have a delay yet.
      • 2022-03-09 06820, 2022

      • mayhem
        lucifer: lets pick this up tomorrow after the TS upgrade.
      • 2022-03-09 06851, 2022

      • alastairp
        lucifer: I installed `timescaledb-2-postgresql-13` and it came with `/usr/lib/postgresql/13/lib/timescaledb-2.2.0.so`
      • 2022-03-09 06859, 2022

      • lucifer
        alastairp: yes. TS repo has apt package for each TS and PG version combination.
      • 2022-03-09 06841, 2022

      • lucifer
        oh let me try that. i was looking for timescaledb-2-2.2.0-postgresql-11 because the other one is timescaledb-2-2.6.0-postgresql-13.
      • 2022-03-09 06847, 2022

      • lucifer
        mayhem: 👍
      • 2022-03-09 06805, 2022

      • alastairp
        timescaledb-2-postgresql-13 and timescaledb-2-postgresql-11 installed fine for me, though I didn't try to start it up with a database
      • 2022-03-09 06854, 2022

      • riksucks
        monkey: thats a good idea, since, if we do further changes, then whoever is doing changes need not worry about putting Nullish coalescing operator all the time. Is it okay if I implement it in the PR which would have the implementation of hiding?
      • 2022-03-09 06800, 2022

      • monkey
        riksucks: Sure, that'll work
      • 2022-03-09 06806, 2022

      • texke has quit
      • 2022-03-09 06817, 2022

      • texke joined the channel
      • 2022-03-09 06834, 2022

      • alastairp
        CB released, gravatars removed
      • 2022-03-09 06838, 2022

      • reosarevok
        Yay
      • 2022-03-09 06802, 2022

      • alastairp
        monkey's review of the uuid cccccc-cccccccc-cccc-cccccccc is gone too!
      • 2022-03-09 06816, 2022

      • monkey
        Nooooo! Not cccccc-cccccccc-cccc-cccccccc !
      • 2022-03-09 06844, 2022

      • alastairp
        it was a good release, huh?
      • 2022-03-09 06835, 2022

      • reosarevok
        I c what you did there
      • 2022-03-09 06847, 2022

      • alastairp
        oh, it was unintentional!
      • 2022-03-09 06848, 2022

      • riksucks
        also monkey, I haven't thought much on the exact implementation (have recently moved into a dorm), so I am going to say what I have in mind, please refine wherever you feel like. My plan is to store keys of the events that I am gonna hide in localStorage. Key would probably be something like `${event.event_type}-${event.timestamp}` (since not all events have id available, like listens).
      • 2022-03-09 06805, 2022

      • alastairp
        hmmm, poking around the CB DB I found a _lot_ of spam comments, just looking into them now to see if they're public or not
      • 2022-03-09 06831, 2022

      • reosarevok
        lucifer: I guess you have seen LB-1074 ?
      • 2022-03-09 06832, 2022

      • BrainzBot
        LB-1074: Reconnecting Spotify violates uniqueness constraint https://tickets.metabrainz.org/browse/LB-1074
      • 2022-03-09 06833, 2022

      • mayhem
        OT: GSoc Mentors have been invited. if you didn't get an invite but were expecting one, ping me.
      • 2022-03-09 06822, 2022

      • lucifer
        reosarevok: yeah this has been reported many times before but we have been unable to reproduce it reliably.
      • 2022-03-09 06826, 2022

      • reosarevok
        Oh, ok
      • 2022-03-09 06810, 2022

      • lucifer
        alastairp: mayhem: how should we backup timescale docker volume?
      • 2022-03-09 06817, 2022

      • alastairp
        lucifer: `cp`
      • 2022-03-09 06821, 2022

      • monkey
        riksucks: Yes, I think something along those lines would work. I've come to the same conclusion for deleting listens where we have a similar issue (we recently moved to a mechanism that marks a listen to be deleted, with listens being deleted in batch every hour)
      • 2022-03-09 06838, 2022

      • mayhem
        oh. I have a couple of tables that I could get rid of.
      • 2022-03-09 06840, 2022

      • mayhem
        hang on.
      • 2022-03-09 06850, 2022

      • lucifer
        oh, that seems simple.
      • 2022-03-09 06800, 2022

      • mayhem
        canonical_recordings for instance. that will make the backup easier.
      • 2022-03-09 06818, 2022

      • alastairp reviews migration document
      • 2022-03-09 06821, 2022

      • mayhem
      • 2022-03-09 06828, 2022

      • mayhem
        these are the tables that remain.
      • 2022-03-09 06849, 2022

      • mayhem
        the __delete_* tables were old mapping tables. they *should* be safe to delete, but I renamed them just in case.
      • 2022-03-09 06800, 2022

      • mayhem
        if nothing breaks we should delete them before we proceed with the backup.
      • 2022-03-09 06857, 2022

      • mayhem
        75M rows in those tables. I'm sure things would move faster without those tables. :)
      • 2022-03-09 06815, 2022

      • alastairp
        db is 787gb, and we have 2.2tb free on /. gaga-old drive is 690gb in total, so we can't use that as a copy
      • 2022-03-09 06801, 2022

      • mayhem
        my day is going to shit. I may not be available until 16h today. :(
      • 2022-03-09 06810, 2022

      • alastairp
        looks like if we're going to make the backup on the same disk, we will have to use --link in pg_upgrade to set up the data files in both pg versions
      • 2022-03-09 06829, 2022

      • alastairp
        we'll have to test how many items pg_upgrade will link, and how many it needs to make a new version of
      • 2022-03-09 06850, 2022

      • alastairp
        lucifer: do you have something running in bono?
      • 2022-03-09 06800, 2022

      • alastairp
        mayhem: have we tweeted?
      • 2022-03-09 06809, 2022

      • mayhem
        not yet, let me do that.
      • 2022-03-09 06823, 2022

      • mayhem
        alastairp: thoughts on dropping those __delete tables?
      • 2022-03-09 06835, 2022

      • mayhem
        should I proceed? nothing seems to have broken.
      • 2022-03-09 06839, 2022

      • lucifer
        alastairp: yes, encountered some more errors, trying to resolve those now.
      • 2022-03-09 06851, 2022

      • alastairp
        I've wondered about the 2 sets of mapping tables before, it seems that the old ones are a fine candidate to delete
      • 2022-03-09 06859, 2022

      • alastairp
        mayhem: I could quickly pg_dump them if you want a backup
      • 2022-03-09 06824, 2022

      • mayhem
        I don't want a backup, but I think just for safety we should do that. please proceed with that.
      • 2022-03-09 06830, 2022

      • mayhem
        once dumped, drop those tables, please.
      • 2022-03-09 06834, 2022

      • alastairp
        on it, will do
      • 2022-03-09 06856, 2022

      • mayhem
        do we have an estimate of how long this will take?
      • 2022-03-09 06802, 2022

      • mayhem
        Or should I just say 2-3 hours?
      • 2022-03-09 06811, 2022

      • alastairp
        the upgrade? no, lucifer is still testing the postgres upgrade steps
      • 2022-03-09 06851, 2022

      • mayhem
        lucifer: do you have a gut feeling? minutes or hours?
      • 2022-03-09 06809, 2022

      • lucifer
        hours
      • 2022-03-09 06819, 2022

      • mayhem
        k
      • 2022-03-09 06818, 2022

      • mayhem
        Tweeted, claiming 2-3 hours.
      • 2022-03-09 06852, 2022

      • mayhem
        heh, I love that MeB takes things seriously, even while still having fun.
      • 2022-03-09 06854, 2022

      • mayhem
      • 2022-03-09 06826, 2022

      • riksucks
        monkey: can we also employ a similar algo to delete keys of listens from localStorage, will prevent stale storage
      • 2022-03-09 06801, 2022

      • monkey
        Yeah, i was just thinking about how to do that. Any ideas?
      • 2022-03-09 06846, 2022

      • riksucks
        an algorithm that checks the oldest timestamp available in the feed, then deletes any event that's in the localstorage that's older than that
      • 2022-03-09 06843, 2022

      • riksucks
        we should probably store these keys in an array, so that the time series aspect is retained
      • 2022-03-09 06853, 2022

      • alastairp
        mayhem: hah, amazing
      • 2022-03-09 06800, 2022

      • alastairp
        !m reosarevok and others working on that ticket
      • 2022-03-09 06800, 2022

      • BrainzBot
        You're doing good work, reosarevok and others working on that ticket!
      • 2022-03-09 06822, 2022

      • reosarevok
        Thanks, I guess
      • 2022-03-09 06824, 2022

      • monkey
        "oldest timestamp available in the feed" does that take pagination into account?
      • 2022-03-09 06833, 2022

      • riksucks
        no, only the first page, because I thought the hide should only work for the first page? that's the assumption I had in mind, because you would want to hide something that is appearing multiple times in your whole UX
      • 2022-03-09 06804, 2022

      • riksucks
        but if something is in seocnd page, it's kinda already hidden. that is how I had in my mind