#metabrainz

/

      • alastairp
        I wonder if we could perform the filtering only in a production database
      • 2022-03-01 06051, 2022

      • alastairp
        Ansh: which sample database did you import?
      • 2022-03-01 06029, 2022

      • Ansh
        I am currently working on the January's release
      • 2022-03-01 06023, 2022

      • alastairp
        https://critiquebrainz.readthedocs.io/intro.html#… did you use these "smaller sample dumps" ?
      • 2022-03-01 06012, 2022

      • Ansh
        yes
      • 2022-03-01 06012, 2022

      • alastairp
        OK. I'm just going into a meeting now, but let me think about it and I'll see if we can come up with a solution
      • 2022-03-01 06058, 2022

      • BrainzGit
        [musicbrainz-server] 14reosarevok opened pull request #2436 (03master…MBS-11481): MBS-11481 (MBS-10847): Add refine link to edit/subscribed https://github.com/metabrainz/musicbrainz-server/…
      • 2022-03-01 06034, 2022

      • reosarevok
        yvanzo, bitmap: https://github.com/metabrainz/musicbrainz-server/… has the stuff remaining from last milestone in it for now, feel free to remove / add stuff as you see fit
      • 2022-03-01 06039, 2022

      • mayhem
        gah. I had forgotten what mobile world congress week was like. I didn't miss it.
      • 2022-03-01 06058, 2022

      • mayhem
        but then BAM everyone is in town and ready for drinks and dinner. all in one go.
      • 2022-03-01 06004, 2022

      • mayhem has a rough life
      • 2022-03-01 06023, 2022

      • reosarevok
        (ok, added a couple other PRs)
      • 2022-03-01 06039, 2022

      • reosarevok
        mayhem: too late to go hiking for the week? :p
      • 2022-03-01 06038, 2022

      • mayhem
        yeah, way too late.
      • 2022-03-01 06046, 2022

      • mayhem
        I just don't think I am going to get anything done this week.
      • 2022-03-01 06029, 2022

      • reosarevok
        Some people would say networking is work
      • 2022-03-01 06031, 2022

      • reosarevok shrugs
      • 2022-03-01 06038, 2022

      • reosarevok
        I mean, not me, but some people woul d:D
      • 2022-03-01 06041, 2022

      • mayhem
        computer networking or human?
      • 2022-03-01 06005, 2022

      • agatzk has quit
      • 2022-03-01 06033, 2022

      • agatzk joined the channel
      • 2022-03-01 06002, 2022

      • yvanzo
        hi reosarevok, I didn’t really follow the milestone for the last release. I still have PRs related to accounts on my to-review list at the moment.
      • 2022-03-01 06041, 2022

      • reosarevok
        Ok. Well I won't touch the milestone for now, you can add those to it if you want
      • 2022-03-01 06019, 2022

      • BrainzGit
        [musicbrainz-server] 14reosarevok opened pull request #2437 (03master…slave-to-mirror): Change terminology from "slave" to "mirror" https://github.com/metabrainz/musicbrainz-server/…
      • 2022-03-01 06042, 2022

      • yvanzo
        hi alastairp: Sure, this afternoon would work for me, however the new instance is not available at the moment.
      • 2022-03-01 06052, 2022

      • reosarevok
        bitmap: when around, let's talk about that ^ for the actual names of things like RT_SLAVE and indices and whatnot, see what can be changed and what not etc
      • 2022-03-01 06024, 2022

      • agatzk has quit
      • 2022-03-01 06037, 2022

      • agatzk joined the channel
      • 2022-03-01 06023, 2022

      • BrainzGit
        [bookbrainz-site] 14MonkeyDo merged pull request #742 (03master…dev): Added No. of Entities in Collection Page https://github.com/metabrainz/bookbrainz-site/pul…
      • 2022-03-01 06032, 2022

      • lucifer
        mayhem: for listen count of current month, i am thinking just to use timescale. it'll use 6-7 chunks at most so should be much of an issue. we can cache it for some time as well. alternatively, we can calculate in spark in store as a stat LB side.
      • 2022-03-01 06027, 2022

      • mayhem
        seems sensible
      • 2022-03-01 06039, 2022

      • lucifer
        👍
      • 2022-03-01 06017, 2022

      • BrainzGit
        [bookbrainz-site] 14MonkeyDo merged pull request #787 (03master…dependabot/npm_and_yarn/react-datepicker-4.7.0): chore(deps): bump react-datepicker from 2.16.0 to 4.7.0 https://github.com/metabrainz/bookbrainz-site/pul…
      • 2022-03-01 06045, 2022

      • alastairp
        thanks yvanzo. I had some other things come up this afternoon. I was looking at some CB PRs and saw that there will be some significant translation changes to be made. If you can ping me when the instance is ready then I'd be happy to help to move CB translations to it. I'm in no hurry to do so
      • 2022-03-01 06045, 2022

      • yvanzo
        alastairp: Thanks, will keep you informed of any update. I hope it will be in the next days as a new version of Weblate has just been released yesterday.
      • 2022-03-01 06058, 2022

      • alastairp
        Ansh: regarding the missing data for the sample database, I'm thinking of two options: one is that for a development database (e.g. DEBUG=True, or another setting) we can construct dummy metadata for items that are not in the database. To do that we just need an extra check here: https://github.com/metabrainz/critiquebrainz/blob…
      • 2022-03-01 06032, 2022

      • alastairp
        another option is that we create dummy reviews for all of the entities that are in the sample MB database, that someone can import if needed
      • 2022-03-01 06042, 2022

      • alastairp
        lucifer: your feedback on this would be welcome too ^
      • 2022-03-01 06053, 2022

      • lucifer
        why do we need separate behaviour for dev?
      • 2022-03-01 06030, 2022

      • alastairp
        because now we hide reviews if the reviewed entity isn't in the MB database. and if the developer only has the sample MB database instead of the full one, they can only see about 6 reviews out of the 5000+
      • 2022-03-01 06027, 2022

      • lucifer
        oh so this is for importing review dump and MB sample dump?
      • 2022-03-01 06038, 2022

      • alastairp
        yes
      • 2022-03-01 06035, 2022

      • lucifer
        i guess do nothing or always show a fixed name for those reviews in dev.
      • 2022-03-01 06057, 2022

      • alastairp
        yeah right - my first suggestion is the same as your second option
      • 2022-03-01 06028, 2022

      • alastairp
        `if config.get('DEV_HAS_SAMPLE_DB'): <fill in missing metadata with dummy value>`
      • 2022-03-01 06006, 2022

      • lucifer
        yup makes sense
      • 2022-03-01 06015, 2022

      • alastairp
        cool, thanks
      • 2022-03-01 06053, 2022

      • reosarevok
        Seems sensible to me
      • 2022-03-01 06007, 2022

      • reosarevok
        Do you have a sample DB for CB? I guess there's not that many reviews to need it?
      • 2022-03-01 06040, 2022

      • alastairp
        reosarevok: right, the db dump is <10MB compressed...
      • 2022-03-01 06016, 2022

      • reosarevok
        hah
      • 2022-03-01 06017, 2022

      • reosarevok
        Fair enough
      • 2022-03-01 06030, 2022

      • alastairp
      • 2022-03-01 06049, 2022

      • lucifer
        lgtm 👍
      • 2022-03-01 06024, 2022

      • BrainzGit
        [brainzutils-python] 14alastair merged pull request #80 (03master…musicbrainzdb-docs): Clarify return values in musicbrainz_db documentation https://github.com/metabrainz/brainzutils-python/…
      • 2022-03-01 06030, 2022

      • Ansh
        alastairp: I had submitted a pr for removing Gravatar. Can you review that?
      • 2022-03-01 06026, 2022

      • alastairp
        Ansh: yes, I already looked at it! I have some comments to make on it, which I'll do when I've finished my other tasks for today
      • 2022-03-01 06000, 2022

      • Ansh
        Okay.
      • 2022-03-01 06019, 2022

      • alastairp
        there is a database field, `user.show_gravatar` where users can choose if they want a gravatar to show. We can also remove this (and the associated config item on the user's profile page)
      • 2022-03-01 06045, 2022

      • Ansh
        yes, I saw that. I think we can leave it as it is. If we want to integrate some other type of avatar in the future, it would be easier
      • 2022-03-01 06014, 2022

      • alastairp
        we removed gravatar completely from musicbrainz, so I think we can also remove it completely from CB
      • 2022-03-01 06008, 2022

      • Ansh
        I saw the PR for MBS-12137. It said that support of avatar is still kept in the codebase if we want to integrate in the future.
      • 2022-03-01 06009, 2022

      • BrainzBot
      • 2022-03-01 06011, 2022

      • alastairp
        we want to use user avatars from the community.metabrainz.org forums, when we work out how to do that we can integrate it, but because it's all controlled completely within our infrastructure we can probably just enable it by default
      • 2022-03-01 06019, 2022

      • alastairp
        ah, interesting :)
      • 2022-03-01 06054, 2022

      • yvanzo
        actually we replaced avatars with generic editor icon, not to change the page layout
      • 2022-03-01 06038, 2022

      • alastairp
        yvanzo: mmm, right. There are some users who do have specific gravatars: https://critiquebrainz.org/ (aerozol and jesus2099)
      • 2022-03-01 06010, 2022

      • alastairp
        but I'm not sure if we should also replace it with the generic MB user avatar, or remove it completely. I think it's OK to remove it completely
      • 2022-03-01 06032, 2022

      • lucifer
        +1 to remove completely
      • 2022-03-01 06046, 2022

      • alastairp
        I can't remember the steps that are necessary for adding discourse avatars
      • 2022-03-01 06002, 2022

      • lucifer
        if we need this code in future, its in git history.
      • 2022-03-01 06012, 2022

      • alastairp
        +1 absolutely
      • 2022-03-01 06028, 2022

      • alastairp
        I think that a single editor icon (like the MB one) would make the front page or browse page of CB quite repetitive
      • 2022-03-01 06004, 2022

      • Ansh
        Ok. I'll work on it.
      • 2022-03-01 06014, 2022

      • alastairp
        thanks Ansh!
      • 2022-03-01 06024, 2022

      • Ansh
        Where are the static files hosted?
      • 2022-03-01 06032, 2022

      • alastairp
        what static files?
      • 2022-03-01 06011, 2022

      • Ansh
        where will we host the avatar img file
      • 2022-03-01 06036, 2022

      • alastairp
        for now, we should remove all avatar files
      • 2022-03-01 06048, 2022

      • alastairp
        in the future we will load them from https://community.metabrainz.org automatically
      • 2022-03-01 06015, 2022

      • BrainzGit
        [bookbrainz-site] 14MonkeyDo opened pull request #789 (03master…monkey-fix-tooltip-position): fix(bs4): Place tooltips to the right in modal header https://github.com/metabrainz/bookbrainz-site/pul…
      • 2022-03-01 06003, 2022

      • Ansh
        Ok alastairp
      • 2022-03-01 06024, 2022

      • dseomn has quit
      • 2022-03-01 06048, 2022

      • alastairp
        Ansh: by the way, the work that you have already done on that PR is great. thanks!
      • 2022-03-01 06042, 2022

      • dseomn joined the channel
      • 2022-03-01 06045, 2022

      • bitmap
        zas: ping. thanks for checking. I don't think we're running the latest PG point release either, so we should upgrade that too just in case
      • 2022-03-01 06010, 2022

      • zas
        bitmap: can we move active db to pink in order to proceed? When this would be possible?
      • 2022-03-01 06007, 2022

      • bitmap
        yes, we could try this, but it'll require downtime so perhaps when there's less traffic
      • 2022-03-01 06039, 2022

      • bitmap
        the repmgr failover itself isn't instant, and also the service names need to be swapped so that pink is {postgres,pgbouncer}-master and floyd is -slave
      • 2022-03-01 06003, 2022

      • bitmap
        do you want to attempt this today or tomorrow?
      • 2022-03-01 06034, 2022

      • zas
        tomorrow
      • 2022-03-01 06015, 2022

      • zas
        we need to plan it carefully, I think we should upgrade the system as well while at it.
      • 2022-03-01 06058, 2022

      • zas
        but only if hardware checks report no issue
      • 2022-03-01 06011, 2022

      • bitmap
        okay, I can upgrade the postgres cluster repo and prepare some steps for the failover
      • 2022-03-01 06032, 2022

      • bitmap
        what would be a good time to start? I can make a blog post too
      • 2022-03-01 06033, 2022

      • zas
        at which time will you be available tomorrow?
      • 2022-03-01 06055, 2022

      • bitmap
        normally from around 16 UTC but I can appear earlier if needed
      • 2022-03-01 06000, 2022

      • bitmap
        reosarevok: wanna chat about the replication terminology?
      • 2022-03-01 06006, 2022

      • bitmap
        zas: somewhere between 14 and 24 UTC would be ideal for me I guess
      • 2022-03-01 06041, 2022

      • yvanzo
        bitmap: Do you have some time to discuss dbmirror2's implications w.r.t. sir?
      • 2022-03-01 06052, 2022

      • bitmap
        yvanzo: sure
      • 2022-03-01 06047, 2022

      • bitmap
        my initial idea there was that we'd mirror the dbmirror2.pending_data data for sir's usage
      • 2022-03-01 06020, 2022

      • yvanzo
        bitmap: do you mean that sir would query a pending_data table directly?
      • 2022-03-01 06035, 2022

      • bitmap
        yes, since it would contain all the information it needs, so there wouldn't be any need for triggers to push the changes into a queue
      • 2022-03-01 06003, 2022

      • bitmap
        the pending_data mirror would act as the queue
      • 2022-03-01 06034, 2022

      • yvanzo
        bitmap: There are theoretically some benefits with using rmq but I guess it's not really used as it should (no message deduplication atm for example).
      • 2022-03-01 06055, 2022

      • yvanzo
        bitmap: Querying a table instead of a queue is already working for mglubb, so there is some code available to make it work for sir. :)
      • 2022-03-01 06008, 2022

      • bitmap
        right, I don't mean rmq isn't useful, just not really in this context, there's not enough traffic or need to have complex routing
      • 2022-03-01 06008, 2022

      • bitmap
        oh yes that's convenient to have something pre written then :)
      • 2022-03-01 06035, 2022

      • bitmap
        (we can also deduplicate things on the sir side in the future - just pull in a chunk of changes at a time)
      • 2022-03-01 06055, 2022

      • reosarevok
        bitmap: sure, I can
      • 2022-03-01 06013, 2022

      • reosarevok
        (sorry, was watching news)
      • 2022-03-01 06014, 2022

      • bitmap
        hey
      • 2022-03-01 06020, 2022

      • bitmap
        np
      • 2022-03-01 06025, 2022

      • reosarevok
        So yeah, if the idea is to move from slave to mirror
      • 2022-03-01 06047, 2022

      • reosarevok
        We have a lot of triggers, RT_SLAVE on dbdefs, etc
      • 2022-03-01 06005, 2022

      • reosarevok
        Which of those require a schema change to rename, and should we do it during this schema change?
      • 2022-03-01 06037, 2022

      • bitmap
        for RT_MASTER and RT_SLAVE specifically I think we can introduce new constants like RT_PRIMARY and RT_MIRROR. but I'd keep the old ones around for compatibility -- at least for a while
      • 2022-03-01 06007, 2022

      • bitmap
        this shouldn't cause any breakage then but we can encourage people to change them in their settings
      • 2022-03-01 06016, 2022

      • bitmap
        in the upgrade notes
      • 2022-03-01 06013, 2022

      • reosarevok
        I wouldn't change MASTER yet, personally, but I guess we can if we must
      • 2022-03-01 06036, 2022

      • bitmap
        sure
      • 2022-03-01 06049, 2022

      • reosarevok
        Anyway, the basic principle would be the same
      • 2022-03-01 06023, 2022

      • reosarevok
        In the code we have a lot of references to slave but that is probably not a hard change
      • 2022-03-01 06046, 2022

      • reosarevok
        But that's why I was asking about the schema change stuff, which is harder
      • 2022-03-01 06054, 2022

      • bitmap
        well, renaming the PG functions/triggers with slave in the name can be done this schema change if we want, and shouldn't break anything
      • 2022-03-01 06010, 2022

      • yvanzo
        👍
      • 2022-03-01 06027, 2022

      • reosarevok
        Would that just involve "drop them with the old name, create them with the new"?
      • 2022-03-01 06055, 2022

      • bitmap
        yes
      • 2022-03-01 06009, 2022

      • bitmap
        iirc there is a way to rename a function, but
      • 2022-03-01 06024, 2022

      • bitmap
        any functions that call it also have to be redefined
      • 2022-03-01 06037, 2022

      • reosarevok
        Sure
      • 2022-03-01 06011, 2022

      • bitmap
      • 2022-03-01 06022, 2022

      • bitmap
        I guess you can rename triggers too - https://www.postgresql.org/docs/12/sql-altertrigg…
      • 2022-03-01 06051, 2022

      • bitmap
        that makes it a bit less tedious at least
      • 2022-03-01 06059, 2022

      • reosarevok
        Ok. I guess I can work on that tomorrow
      • 2022-03-01 06044, 2022

      • zas
        bitmap: ok for 16 UTC
      • 2022-03-01 06015, 2022

      • lucifer
        mayhem: huh weird, https://beta.listenbrainz.org/sitewide/reports/?r… the map is errily empty for many countries for all time.
      • 2022-03-01 06028, 2022

      • lucifer
        i'd expect at least a few listens :/
      • 2022-03-01 06040, 2022

      • lucifer
        (these are sitewide charts)
      • 2022-03-01 06045, 2022

      • BrainzGit
        [listenbrainz-server] 14amCap1712 opened pull request #1888 (03master…sitewide-stats-fe): Add sitewide stats frontend https://github.com/metabrainz/listenbrainz-server…
      • 2022-03-01 06052, 2022

      • atj
        zas: sorry, today has been a bit of a bust. I've got a role for telegraf pretty much completed though, and made some improvements to the exim setup.
      • 2022-03-01 06059, 2022

      • zas
        np, wanna work on that now? or tomorrow morning?
      • 2022-03-01 06002, 2022

      • atj
        are any firewall changes required on stats. to allow connections from aphex?
      • 2022-03-01 06019, 2022

      • zas
        not for telegraf, as it pushes metrics
      • 2022-03-01 06036, 2022

      • zas
        but node_exporter will require to open one port
      • 2022-03-01 06029, 2022

      • atj
        I need to eat so let's work on it tomorrow morning
      • 2022-03-01 06007, 2022

      • zas
        ok, enjoy ;)
      • 2022-03-01 06014, 2022

      • atj
        I think I've got enough done today so that we can get telegraf working pretty quickly tomorrow
      • 2022-03-01 06044, 2022

      • zas
        perfect