lucifer: can you also please read the last 3 pages for accuracy of the technical data?
reosarevok
"Removing deleted listens from statsOn the 2nd and 16th of each month"
"Stats may not update for the 2rd or 16th day of the month."
I'm confused :D
"It may take up to an hour for a listen to be deleted at the top of the next hour"
I guess either "It may take up to an hour for a listen to be deleted" or "Any listens marked for deletion will be deleted at the top of the next hour" ?
mayhem
the first 2 confusing issues are for lucifer to review.
monkey
Sorry kellnerd, I pushed it out of my head ! Fixed now with the help of mayhem
mayhem
the second option for the latter issue is good, reosarevok. in case you'd like to edit it straight out.
monkey
The project page should now reflect the correct dates, but please do check I didn't flop up again
reosarevok
Sure
mayhem
thx
kellnerd
Thanks monkey, I've just checked the project page and it shows the updated dates now.
monkey
👍
reosarevok
How come it's easier to add to spark than delete from it anyway? I haven't played with spark at all so it's probably obvious for those who have, it's just confusing as someone used to psql :D
kellnerd
First I had to logout and login again because Google was telling me that no GSoC account would we associated with that email 🙄
mayhem
reosarevok: its not spark's fault. we just havent implemented letting spark know when a listen is deleted.
reosarevok
kellnerd: the gsoc site is known to be the worst site on the internet
After Renfe's, maybe - mayhem might have an opinion on that choice
mayhem
renfe is for buying tickets. bahn.de is for finding trains.
so, yes, fucking terrible.
CatQuest
deichmans (norwegian library) website want's a word about that (worst website)
well it's not horrible, it's jsut really rally bad design
reosarevok
"It may take up to 4 hours for data entered into MusicBrainz to become available in the mapping system or for manual mapping."
How come it's 4 h rather than 1? Isn't it a normal mirror DB then?
lucifer
reosarevok, mayhem: Listens are deleted from Postgres Database every hour, but removed from spark hence the stats reports bi-weekly.
reosarevok
"in the mapping system for both automatic and manual mapping" maybe?
lucifer
reosarevok: not a mirror.
mayhem
reosarevok: there are more steps involved then just propagating data.
reosarevok
Postgres and Timescale both are used for playlists?
jivte has quit
lucifer
yes
timescale stores the playlist data. and we query metadata of each recording from MB postgres.
reosarevok
Maybe we should expand on that? Right now it's a bit confusing if it just says "Postgres is our primary workhorse that serves user data (accounts, followers, likes/hates, etc) and Playlists. Timescale (an extended version of Postgres) is used to store Listens and Playlists"
Seems like they're stored in both places, but I guess it's parts and parts then
mayhem
remove the first mention of playlists and we're good, reosarevok
reosarevok
Ok
mayhem
lucifer: dumps happen on the 1st and 15th. importing happens on the 2nd and 16th, which is why we might miss stats calculation on those dates. is that correct?
lucifer
mayhem: we usually don't miss stats calculations any day.
unless the dump generation or import failed.
reosarevok
Then I wouldn't mention the missing bit at all
Or at least, less gloomily than "Stats may not update for the 2rd or 16th day of the month." :)
mayhem
when was the last time we missed stats because of this?
lucifer
this 16th/17th because the postgres db was restarted during schema change that interrupted dumps
mayhem
should we just leave this out then?
lucifer
sounds good to leave out
mayhem
ok, great.
reosarevok
Otherwise it all looks good, if technical
mayhem
ok, thanks reo.
reosarevok
But I guess it's meant to be a "for technical people" read
mayhem
yes.
reosarevok
(you could have a TL;DR version, I'm sure aerozol can write one)
mayhem
the overview section is for that. and labeled as such.
lucifer: I'm not sure if this should be added as a wikidoc page on musicbrainz.org or if I should make a standard HTML page on lb.org? I'm leaning towards the latter.
lucifer
mayhem: listenbrainz.readthedocs.io ?
mayhem
sure, why not. good idea.
jivte joined the channel
CatQuest
🐟
bitmap
zas: I'll have a look at the query to see if it can be optimized at all. is it urgent to restart pink/floyd today or can it be done this week?
zas
bitmap: it can be done later this week, but the sooner the better as usual ;)
servers remaining are: aretha clash floyd gaga kiss pink quest
reosarevok
mayhem: sure, ok, I was just thinking in case you want a sentence about having many interconnected parts
Something like "This complicated schedule is caused by ListenBrainz having a lot of interconnected parts that work at different scales. For more details, read on!"
But with aerozol's "make it simple and not-stupid" wording maybe :p
v6lur has quit
mayhem
where would you put that?
reosarevok
At the end of the Overview under the list of update times I guess?
reads the JSON data from the html and creates a globalAppContext for it
jivte_
okk thanks!!
kellnerd
reosarevok: I might join the meeting a bit later today, so please don't call me first :)
reosarevok
kellnerd: go!
Oh, wait
Ok :)
yvanzo
lucifer: Would tomorrow 10 UTC work for you moving RMQ from/to clash?
lucifer
yvanzo: yes
reosarevok
<BANG>
yvanzo
Ok
reosarevok
Welcome to Modify your Musings Monday!
(aka, you can now edit recent edit notes in MB, mess up with confidence!)
yvanzo
(?)
reosarevok
After a break for last week's schema change, which went, as the song goes, surprisingly ok
We're back with a standard meeting
CatQuest
🐟 !
reosarevok
We have one mailed in review
mayhem
lolol.
as the song goes.
reosarevok
Our usual sleeper says:
'''
Atā mārie (good morning) from NZ.
The main thing I worked on was templates and systems re. making noise about news/updates/events - putting in the time now so we can quickly and confidently get news out in future.
Can you imagine quickly filling one of these out if you have done something cool (or see something cool)? If not, how could I tempt you or make it easier (including ideas that don’t involve a form)?
I also finally did the first round of mockups for the ‘proper’ rollout of the LB recommendations/jams feature!
Other than that, I did the usual, some guideline updates, and helped drag a Picard ticket to the seventh level of hell. Good times.
In nicer news, my other work deadline has passed, so I am able to do more on MeB again! Woooo