^^ this is something cb devs need to have a look imho
2023-02-03 03457, 2023
zas
bitmap: sir-prod isn't running it seems, I thought yvanzo will move it, but that's not the case, or something is broken in alerts
2023-02-03 03431, 2023
zas
I restarted sir-prod on floyd, container wasn't running
2023-02-03 03446, 2023
zas
ok, it seems to work again, all alerts cleared
2023-02-03 03451, 2023
bitmap
zas: sorry was eating, thanks for taking care of it
2023-02-03 03452, 2023
bitmap
btw sentry has been acting up for a while, the logs have some error related to redis
2023-02-03 03448, 2023
bitmap
wasn't sure how to deal with that. (tried restarting redis, but it looks like it might be stuck starting up, since the web container just returns "Redis is loading the dataset in memory" for everything)
65.108.55.50 (dunno what the hostname is, syswiki says sentry-cloud.metabrainz.org but that doesn't resolve for me)
2023-02-03 03431, 2023
bitmap
yeah, I've heard of Patroni
2023-02-03 03448, 2023
zas
no idea about sentry stuff, I guess alastairp can have a look in few hours
2023-02-03 03400, 2023
bitmap
it looked interesting but repmgr involved a lot less magic to get working iirc :) (and we were already using barman from 2ndquadrant)
2023-02-03 03438, 2023
zas
I'm off to bed (3AM soon)
2023-02-03 03449, 2023
bitmap
there is actually a haproxy container in the docker-postgres-cluster repo which tries to automatically route requests to the current primary, it's not tested but I'd like to try it out soon
2023-02-03 03413, 2023
bitmap
have a good night!
2023-02-03 03419, 2023
zas
thx
2023-02-03 03458, 2023
aerozol
zas: alastairp: are ‘CB devs’ just alastair? Maybe? Just making sure you see the error zas posted above alastair, sorry ;)
2023-02-03 03445, 2023
aerozol
monkey: for when you’re back, just thought this was interesting, and looks very LB already. Do we want to do something similar eventually? https://album.link/b/1646963467
2023-02-03 03436, 2023
aerozol
It also makes me think that we’ve had mockups where we try squish in genres/tags in LB, but we could just do them quite big instead. Not worry about really taking up space, being mobile friendly (but only letting one listen expand at a time?)
We've gotten a bunch of crypto scam edit notes that would be good to purge
2023-02-03 03423, 2023
yvanzo
There is a lot to say about this PR but I’m available to help with handling the current scam through database updates if needed.
2023-02-03 03412, 2023
reosarevok
For now, it's small enough we probably don't need to rush into it - but if it gets bigger before we have a proper way of doing this we can look into removing them via the DB :)
2023-02-03 03448, 2023
mayhem
moooin!
2023-02-03 03456, 2023
jivte_ has quit
2023-02-03 03451, 2023
monkey
aerozol: yes, definitely have been dreaming about a good album page on LB, if that's what you mean.
2023-02-03 03449, 2023
aerozol
monkey: even in the individual ‘listen’ drop down on the home page, but album page too!
2023-02-03 03456, 2023
monkey
aerozol: what part of this link page would you see in the listen card dropdown?
2023-02-03 03411, 2023
aerozol
Buy and listen but Copy URL/Embed/Share is nice too
2023-02-03 03455, 2023
aerozol
But I was mainly thinking about the design rather than the actual content
2023-02-03 03457, 2023
aerozol
Nice and spacious
2023-02-03 03432, 2023
aerozol
good night by the way :D
2023-02-03 03408, 2023
alastairp
hello, looking at chat history / CB stuff now, thanks for the ping
2023-02-03 03416, 2023
alastairp
lucifer: hi, I was talking to mayhem earlier in the week and we were thinking that it'd be a good idea to have a primary key column in the listen table (for when you're going to rewrite it). The reason is that we're probably going to start publishing other datasets of things like messybrainz id mappings, and it'd be useful to do things like indicate which listens a particular messybrainz id maps to
2023-02-03 03448, 2023
alastairp
while it's possible to have a unique id on user/name/date (or user/msid/date in the future), that'd make such a dump much larger if we had to include all fields to identify a unique key
are you able to confirm about what server 10.2.2.44 was at that time? Maybe there was a hiccup in consul, or maybe it hadn't restarted yet?
2023-02-03 03449, 2023
alastairp
regarding sentry-cloud, this was the DNS name that we gave the VM temporarily while sentry.meb.org was still pointing to gateways. I've updated syswiki to point to the proper hostname
2023-02-03 03457, 2023
atj
alastairp: floyd is 10.2.2.44
2023-02-03 03444, 2023
zas
10.2.2.44 didn't change, it was floyd, but pg master moved to pink, it was solved after I restarted the container, so I guess it didn't actually restarted on template change or smt like that
I'm not 100% certain of the sentry redis error that bitmap said, I see that sentry is up at the moment, but I do see this error that he mentioned in the webserver logs.
2023-02-03 03442, 2023
alastairp
redis DB is currently 800mb, that doesn't seem very big to me (VM has 16gb memory)
I wonder if we had 2 pgbouncer-master at the same time, may explain the issue
2023-02-03 03411, 2023
alastairp
https://develop.sentry.dev/self-hosted/troublesho… "Redis is used both as a transactional data store and a work queue of Celery in the self-hosted setup. For this reason, it may get overwhelmed during event spikes."
2023-02-03 03444, 2023
alastairp
it's possible that the db switchover caused many errors to be sent to sentry and redis got overwhelmed
2023-02-03 03400, 2023
zas
very likely
2023-02-03 03450, 2023
zas
alastairp: too bad we don't have logs anymore for cb web container, they were spammed and since they are limited in size we don't have lines for when the switch occured
2023-02-03 03438, 2023
jasje joined the channel
2023-02-03 03412, 2023
jasje
akshaaatt: please update sentry to v3.4.2 where the app crashing is fixed.
2023-02-03 03436, 2023
jasje
we can upgrade to room 2.5.0 after that
2023-02-03 03412, 2023
jasje has quit
2023-02-03 03428, 2023
Shelly joined the channel
2023-02-03 03422, 2023
Shelly has quit
2023-02-03 03413, 2023
alastairp
lucifer: from cron dumps log, "2023-02-03 00:00:02,800 listenbrainz.webserver ERROR Invalid dump ID 1191, could not find previous dump"
2023-02-03 03437, 2023
alastairp
is this possibly related to the postgres master switchover, causing a jump in the PK sequence?
alastairp: PK makes sense. about dumps, yeah its likely due to switchover.
2023-02-03 03431, 2023
}8]
not sure if this is the right place but i have a physical cd that i was going to rip to flac. the album is in musicbrainz but the discid wasnt associated with it so i submitted that. the problem is now that the artists real name is used rather than his performance/artist name. whats the best/correct way to change the artist name for an album?
2023-02-03 03443, 2023
}8]
specifically, the album is "The Second Coming" by "Roger Troutman II" (his artist/performer name), but in musicbrainz its listed at "The Second Coming" by "Roger Troutman Jr." (his real name)
2023-02-03 03427, 2023
alastairp
}8]: you should be able to get some feedback on this question in the #musicbrainz channel
2023-02-03 03412, 2023
}8]
ok thanks
2023-02-03 03413, 2023
alastairp
(while there is an overlap in the membership of both channels, this one tends to focus more on the software development of musicbrainz, while that one has more editors of the database)
2023-02-03 03426, 2023
}8]
ahh ok, sorry, i didnt realize that
2023-02-03 03432, 2023
alastairp
no problem at all!
2023-02-03 03419, 2023
jivte joined the channel
2023-02-03 03453, 2023
ArjunM joined the channel
2023-02-03 03402, 2023
Sum joined the channel
2023-02-03 03423, 2023
yvanzo
Does BrainzBot has backups? If not, the data volumes of the new dockerized instance could be collected by borg backup.
2023-02-03 03458, 2023
yvanzo
alastairp: By deploying on the current Postgres instance, it seems that you just addressed OTHER-319, right?
yvanzo: yes, this is in the middle of happening. By moving to MeB infra we have chatlogs (and botbot configuration) in the main PG cluster, which gives us automatic backups using the current backup system.
2023-02-03 03455, 2023
alastairp
There is a redis server, but I don't think it's useful to back up its volume using borg, it's just used as a transient message queue between the chatbot and plugins
2023-02-03 03437, 2023
yvanzo
👍
2023-02-03 03402, 2023
alastairp
in terms of python dependency upgrades, I've already applied Leo's changes to upgrade to django 1.11 and python 3, with a pending task to upgrade to django 2.2. I've also started chopping a lot of code out of the original chatlots webapp from the original upstream project that we forked from (no longer maintained)
2023-02-03 03428, 2023
alastairp
I think we've managed to remove approximately half of the dependencies, which makes ongoing maintenance much easier
2023-02-03 03404, 2023
alastairp
unfortunately there is one large dependency used to generate bootstrap html from python which isn't supported with the latest version of django, so we can't yet upgrade to that
2023-02-03 03419, 2023
alastairp
However we've been talking for years about how the web interface is terrible, and this may be a chance to rewrite the interface that generates the chat logs allowing us to make something simpler that doesn't have bad javascript, and at the same time reduce the number of external dependencies that the app has
2023-02-03 03407, 2023
reosarevok
bitmap, yvanzo: hi! :)
2023-02-03 03428, 2023
reosarevok
alastairp: oh, something that doesn't suck would be nice
2023-02-03 03452, 2023
alastairp
reosarevok: taking feedback for ideas
2023-02-03 03411, 2023
reosarevok
"something that doesn't suck" is a good starting point ;D
2023-02-03 03423, 2023
alastairp
but honestly, 1 URL per day with a prev/next button, and real working #anchors is probably 100x better than what we currently have
2023-02-03 03412, 2023
alastairp
or even just static html per month like what we used to have :-P (that being said, I do like the search)
2023-02-03 03413, 2023
reosarevok
+1 - only thing we'd need is the search
2023-02-03 03448, 2023
alastairp
yeah, search is the one thing that makes a fully static system more difficult
2023-02-03 03405, 2023
alastairp
personally I think the search works quite well - itd just be great if the results actually linked to the message :-P
2023-02-03 03422, 2023
petitminion joined the channel
2023-02-03 03433, 2023
yvanzo
alastairp: Thanks, I’ve assigned the ticket to you to reflect the current status. Feel free to create sub-task if needed.
2023-02-03 03444, 2023
alastairp
thanks yvanzo
2023-02-03 03402, 2023
bitmap
reosarevok, yvanzo hello
2023-02-03 03416, 2023
yvanzo
hi you both
2023-02-03 03409, 2023
yvanzo
The banner text is gone from the beta website, do you have it somewhere?
2023-02-03 03410, 2023
yvanzo
A tip is to comment banner text with <!-- --> for temporarily displaying a different message.
2023-02-03 03415, 2023
bitmap
yes, just added it back
2023-02-03 03428, 2023
yvanzo
Thank you
2023-02-03 03407, 2023
bitmap
yesterday I spent well over and hour investigating those "Two relationships with the same key" errors and didn't get anywhere, so I'm working on improving the error message to give more information
2023-02-03 03407, 2023
yvanzo
So there are 68 tickets addressed in beta, and 70 tickets reporting about beta.
2023-02-03 03436, 2023
reosarevok
Seems like a good plan
2023-02-03 03440, 2023
yvanzo
Which is really good to have that much feedback for beta :)
It’s getting wild, it would probably help to move it to tickets (existing or new ones)?
2023-02-03 03449, 2023
reosarevok
The discussion being there is good (not everyone can be bothered to comment on tickets) but we should probably create tickets for some of these
2023-02-03 03459, 2023
bitmap
tickets would help make tracking the issues a bit easier for me. I'm not sure how many of these will be fixed before monday, though
2023-02-03 03431, 2023
yvanzo
It’s good to have feedback for sure, it’s better to have it in a structured way issues can be tracked, for example with... an issue tracker?
2023-02-03 03438, 2023
bitmap
i'd prioritize the ones that crash the page or throw errors
2023-02-03 03416, 2023
reosarevok
If we need a bit of time to try to debug the clashing rels error, maybe we should keep this in beta one more week anyway?
2023-02-03 03432, 2023
reosarevok
I can enter some tickets based on the feedback, at least
2023-02-03 03407, 2023
bitmap
it's better to have an extra week than rush it, I'd say
2023-02-03 03417, 2023
yvanzo
bitmap: Do you want us to start with helping on a specific issue you are stuck with? reviewing the existing tickets? or referencing the issues mentioned in this thread as tickets?
2023-02-03 03453, 2023
yvanzo
(Or any other way we can help with getting a more clear idea of what is left to be done or to be postponed)
2023-02-03 03441, 2023
bitmap
I was mainly stuck on MBS-12860 and MBS-12861 yesterday but it seems like they are not easy to reproduce without more info. help with opening issues from comments in the thread would be helpful
we do have a captureExceptions call for any errors that occur
2023-02-03 03418, 2023
bitmap
so either sentry was down, or blocked by both users, or the error capturing didn't work
2023-02-03 03412, 2023
reosarevok
IIRC we've had the impression at least one of the users does block sentry before?
2023-02-03 03427, 2023
yvanzo
Ok, I’m starting with “opening issues from comments in the thread”. I will mention each post here before opening a ticket, and edit the post to link to the ticket after.
2023-02-03 03445, 2023
reosarevok
yvanzo: ok, you start from the top I start from the bottom? : )
2023-02-03 03451, 2023
yvanzo
👍
2023-02-03 03434, 2023
bitmap
one of them might but I've seen errors in sentry for the other before
2023-02-03 03408, 2023
reosarevok
Hmm, then maybe there's a second issue with the logging, ok
2023-02-03 03423, 2023
reosarevok
"The never ending “show more”. Would be much better to show how many more there actually are in the list. “Show more (10 items)” or similar." bitmap: do you know where we show "show more"? https://beta.musicbrainz.org/artist/dc8c2046-469f… already shows "See all 105 relationships", but maybe that's in the release rel editor?
