reosarevok: what's slow vs fast? I'm not too worried about specific order, I just see that we have a bug in BU that only sorts by release year (and therefore it's possible that SQL could just sort differently within all releases in a certain year if we're doing paging)
But if you only want titles and artists, yeah, you could probably manage with less tables :)
alastairp: slow uses normal queries for mirrors without the materialized tables
(so we check if the tables exist and have content, and if not, we use the slow mechanism)
I only query 8 tables, so I'd like to exclude all the rest
thanks reosarevok! this helps us significantly
(I mean, it now means that we need to do more work, but at least things will look better)
q3lont: that'd probably need some changes to dbmirror, so you might want bitmap, but he's mostly away for a few days at least, so maybe after that he can check? I haven't played with the replication much at all
I mean, I guess maybe you could drop all the tables you don't want, and then always run ImportReplicationChanges with --ignore-errors (if a table fails to import, continue anyway)
And see if that somehow doesn't fail terribly? :D
But I certainly cannot suggest that in good faith, it *might* fail terribly :D
alastairp: no problem, always glad to make y'all work more
btw alastairp, just wanted to give you a heads up, I have created the JIRA ticket and also updated the PR, although the test failed because of the sqlalchemy connection error, which is a false positive.do lmk if you want any more changes. I would now get started on the frontend :)
riksucks: yeah, dn
yeah, don't worry about the connection error. I'll take a look at the PR now and re-trigger the tests
mayhem: interesting, I was just thinking about the listen writer last week (in the context of "what happens if the listenstore goes away while it's doing stuff?")
so in the case of crash-only software this should just fail but if an operation didn't completely finish, when it starts up again it should pick up from where it stopped, potentially re-reading the same thing from the queue and trying to add it again
yes. while I would not call the ts writer crash-only, that is the general concept. architect so that things are idempotent and can without harm be re-done. only consider something done, when we've got confirmation for it.
I find this to be a good exercise in building resilient systems. it is pleasure to watch LB heal itself after PG or something else was down.
yeah, that last sentence is key
2-3 seconds and everything is back. great feeling.
that article gave me faith that people are perhaps making progress on truly distributed systems. I first delved into this stuff during 2001 when everyone was convinced everything would be decentralized going forward because we had napsten and limewire.
odnes has quit
I'm so glad I eventually gave up and just focused on building a working centralized version.
alastairp: do you happen to have some academic articles that describe how to build a playlist?
the one AB-like feature I wish we had was to analyze the begin and end of a track.
although I think that the main task for this project was for "playlist continuation", so given 10 tracks already in a playlist, recommend the next 5 to add
it would be really nice to know if a tracks starts/ends: quietly, suddenly, different types of ramp-up, etc...
not sure if anyone did "create a playlist from scratch given existing interests"
exactly. I think ben & paul's presentation is the closest I've seen.
yes right, I've thought about this kind of stuff too. from time to time I've wondered if my discover weekly had this kind of stuff going - the last track always seemed great. but that hasn't happened much recently
recently everything produced by spotify has been mediocre at best.
your recommended tracks already feels more promising, but it is missing variance.
I wonder if there is space for different moods of daily jams. daily energetic jams, daily chill out jams...
Got a couple of mailed in reviews, so let’s start off with those:
Last week I’ve been preparing for moving ay (which is today), but I did do a few things:
Continued the ListenBrainz metadata viewer and merged that first PR (currently deployed on beta.LB)
I did some outreach and debugging with third-party players and scrobblers that use the ListenBrainz API, mostly relating to better support for love/hate feedback.
I reviewed PRs for LB and BB, and did a bunch of repository admin and ticket triage.
There was a notice from Github that some OAuth tokens were leaked by Travis (which we were using for BB). According to Github we weren’t targeted, but I disconnected Travis app on Github and ensured no tokens or authorization remained for it.
That’s most of it for me! I expect to be partly AFK this week, between unpacking and no internet at the new house.
Take care y’all !
Last week, I continued updating documents on securing MeB infrastructure
with the 2nd meeting this past Tuesday.
Also updated my PR documenting our current instance of RabbitMQ (which has
to be eventually replaced with a cluster).
Freed tens of GBs of useless logs related to a bug that has been fixed in
Opened a pair of small pull requests to docker server configs and ansible.
Plus PR reviews and support.
Others up: lucifer, alastairp, atj, reosarevok, zas, akshaaatt, bitmap, mayhem, CatQuest, Freso – anyone else who wish to give review, let me know ASAP. :)
(No delivery yet btw, so sorry if I disappear for a minute.)
last week, i worked on adding last listened times to recommendations but data turned out to be too much to store in LB database so need to rework it to do more stuff on spark side.
I'm here too 😃
mayhem waves at aerozol
hackerman joined the channel
other than i worked on more recommendations tuning and refactoring LB recommendations to use more and newer built-in Spark APIs instead of hand rolling our own code.
hackerman has quit
aerozol: I assume that’s you saying you want to be added to the list?
hackerman joined the channel
finally investigated, some LB issues like broken emails, broken dumps and some ISEs due to bad user data submission.
that's it for me. alastairp: next?
Freso: yes plz
(Others up: atj, reosarevok, zas, akshaaatt, mayhem, aerozol, CatQuest, Freso – anyone else who wish to give review, let me know ASAP. :))
last week I fixed a bug in CB that prevented non-releasegroup reviews from appearing on the admin "reported reviews" page, and also fixed a bug that had been reported where a user could craft a malicious oauth redirect url on an oauth app
I also fixed some bugs in CB oauth around dealing with invalid client ids/secrets
I reviewed some of Ansh's open PRs about adding more data to Label and Place pages on CB
I restarted some old tasks about making data dumps for LB/MB datasets that we want to publish, and I fixed a bug in our config file generation for LB that prevented us from using consul to take down the listenstore the last time we had an LB downtime
that's me, atj next?
last week I spent some time rewriting the netplan ansible role to try and simplify network configuration
this is currently a WIP PR that will hopefully result in some further benefits to other configuration, such as managing /etc/hosts
I think that's it, aerozol?
has aerozol nodded off? akshaaatt?
Mōrena! Not much to report, but I did some guideline updates and want to make sure people still have it on their to-do list to have a look
The endless back and forth between long country lists for digital and worldwide has worn me down and I'm keen to tackle it finally - wondering if there's any devs that are interested or have ideas. Let me know!