alastairp: yes, I noticed, and that was the issue, if the binary wasn't built before running agent with the flag
2021-05-04 12400, 2021
zas
it creates an empty dir on the host (but doesn't complain)
2021-05-04 12426, 2021
zas
in those cases we should use --mount (it would fail instead)
2021-05-04 12433, 2021
ruaok
not better.
2021-05-04 12442, 2021
zas
that's because make wasn't executed
2021-05-04 12454, 2021
zas
ruaok: did you restart containers?
2021-05-04 12403, 2021
ruaok
ah, no. ok, hang on.
2021-05-04 12435, 2021
alastairp
web is up with valid TS config
2021-05-04 12449, 2021
alastairp
now it's restarting
2021-05-04 12405, 2021
ruaok
that was likely caused by me restarting the containers as per zas.
2021-05-04 12408, 2021
ruaok
should be back nowish
2021-05-04 12431, 2021
zas
this pg health script thing is messy
2021-05-04 12442, 2021
alastairp
it did just start back up, but without timescale service
2021-05-04 12400, 2021
alastairp
spoke to early again. just restared again with correct config
2021-05-04 12408, 2021
ruaok
timescale is up and running ok.
2021-05-04 12425, 2021
zas
and it should be registered in consul now
2021-05-04 12400, 2021
alastairp
web appears to be stable too
2021-05-04 12432, 2021
alastairp
starting ts writer again
2021-05-04 12432, 2021
ruaok
and listens from the last hour are there. good.
2021-05-04 12457, 2021
ruaok
typesense is unhappy. I'll look into it.
2021-05-04 12430, 2021
alastairp
timescale writer is up, has correct config, and has hit python code
2021-05-04 12451, 2021
alastairp
incoming queue just plummeted from 2.5k to 0 in 1 refresh cycle on stats
2021-05-04 12453, 2021
alastairp
phew
2021-05-04 12400, 2021
alastairp
thanks ruaok, thanks zas
2021-05-04 12424, 2021
_lucifer
!m alastairp, ruaok, zas
2021-05-04 12424, 2021
BrainzBot
You're doing good work, alastairp, ruaok, zas!
2021-05-04 12425, 2021
alastairp
looks like we need some docs on the postgres healthchecks (if they don't exist already). I didn't know about them
2021-05-04 12425, 2021
ruaok
thank you!
2021-05-04 12433, 2021
alastairp
ruaok: docker-server-configs on gaga is on a temporary typesense branch of yours, we should get this to master when convenient. zas is currently running consulagent from his home directory which we should fix
2021-05-04 12404, 2021
ruaok
ok, I'll commit once I get typesense up and running again.
2021-05-04 12456, 2021
alastairp
zas: do you want to discuss how we can improve this postgres health check script? changing from -v to --mount sounds like a good first step
2021-05-04 12420, 2021
ruaok
not sure why , but the typesense server refuses to start. I'm going to kill the volume and rebuild the index.
2021-05-04 12412, 2021
zas
alastairp: I see 2 possible improvements: ensure the binary exists (and if not, build it), and use --mount (so it doesn't create a dir on the host if we reach this point)
2021-05-04 12414, 2021
d4rkie has quit
2021-05-04 12404, 2021
d4rkie joined the channel
2021-05-04 12410, 2021
alastairp
zas: cool, I'll see how easy it is to make those two changes, otherwise open a ticket
2021-05-04 12404, 2021
ruaok
alastairp: docker server configs on gaga is clean now.
this came with our airbnb. not sure if I should open it. lol.
2021-05-04 12417, 2021
shivam-kapila
From the look of it looks nice
2021-05-04 12431, 2021
alastairp
brut
2021-05-04 12447, 2021
alastairp
I don't know that brand
2021-05-04 12439, 2021
shivam-kapila
I dont even know whats written on it lol
2021-05-04 12402, 2021
alastairp
shivam-kapila: Brut rose -> dry rosé (style of wine/grape)
2021-05-04 12423, 2021
alastairp
it's a champagne
2021-05-04 12443, 2021
ruaok
its not bad. organic even!
2021-05-04 12417, 2021
shivam-kapila
Thanks alastairp
2021-05-04 12430, 2021
shivam-kapila
ruaok: cheers. Enjoy the drink :)
2021-05-04 12429, 2021
ruaok
:)
2021-05-04 12418, 2021
BrainzGit
[listenbrainz-server] alastair opened pull request #1424 (master…add-git-hash-label-last): Don't add GIT_COMMIT_SHA build to the label at the beginning of the build https://github.com/metabrainz/listenbrainz-server…
2021-05-04 12450, 2021
_lucifer
alastairp: can you take a look at the youtube PR next? i think only the admin view needs to be fixed otherwise the pr is ready to merge.
2021-05-04 12413, 2021
alastairp
_lucifer: sure, I'll be in at the office tomorrow so will do some reviews first up
2021-05-04 12434, 2021
_lucifer
awesome, thanks!
2021-05-04 12400, 2021
ruaok
I hope to turn up tomorrow near noon. leaving calella in the morning, going straight to the office.
2021-05-04 12428, 2021
alastairp
I'll be noon-ish too. currently trying to negotiate starting time for a bike ride
there's a related ticket to try and unify the Listenstore class (or get rid of it if it doesn't make sense any more). Did I open one for that, or is that something that we just discussed?
2021-05-04 12442, 2021
_lucifer
i think we only discussed it.
2021-05-04 12459, 2021
alastairp
OK, let me open a ticket for it too
2021-05-04 12431, 2021
alastairp
this flows into another related issue that I wanted to tackle sooner rather than later - the randomly failing integration tests.
2021-05-04 12434, 2021
alastairp
I think that if we improve the service initialisation we can set up a cleaner test infrastructure that hopefully stops these errors from happening
2021-05-04 12417, 2021
alastairp
related to that, I'd also like to group together the integration tests and unit tests into a single test run, I don't think that the distinction is useful. There are a bunch of tests in the integration folder which imo don't fit there
2021-05-04 12448, 2021
_lucifer
there was some network issue on my end. back again.
2021-05-04 12437, 2021
alastairp
np
2021-05-04 12416, 2021
_lucifer
yeah, i think mostly what the api tests lie in the integration tests.
2021-05-04 12419, 2021
_lucifer
yup, i agree. i do not see much point of having separate unit and integration tests.
2021-05-04 12425, 2021
alastairp
integration tests are useful for when we want to check that items flow through webserver -> queue -> writer -> database, but if we're just checking that an endpoint returns the right data or directly hits the database, that's just a view test
2021-05-04 12453, 2021
alastairp
so we could have a small set of integration tests (maybe behind a pytest mark), but move everything else into views tests
2021-05-04 12402, 2021
_lucifer
makes sense.
2021-05-04 12425, 2021
alastairp
cool, but that's a bit further down the line. let me open some tickets for these items then we can start from the top
mmm, I thought that some of the integration tests do end0to-end
2021-05-04 12449, 2021
_lucifer
yes, the api tests are end to end. i mean the view tests.
2021-05-04 12400, 2021
alastairp
oh, right.
2021-05-04 12435, 2021
alastairp
yeah, there's an open question about how we want to deal with this. I'm mostly of the view that it's OK to put stuff in a database and then call a view and check that data is returned
2021-05-04 12448, 2021
alastairp
what we should do is be smart about how we reset the database between tests
2021-05-04 12409, 2021
_lucifer
how is that handled currently?
2021-05-04 12445, 2021
alastairp
we currently drop the tables (listenbrainz) or the database (timescale - integration) and recreate them again
2021-05-04 12446, 2021
_lucifer
in the current lb tests it seems each method has a clean database with only the setup method.
2021-05-04 12448, 2021
alastairp
django does it by starting a transaction before each test and then rolling it back afterwards. this saves the time of deleting all tables and creating them again
2021-05-04 12427, 2021
ruaok
zas: it looked so cheezy, but its actually a nice bottle of cava. not hard to do here in :spain:.
2021-05-04 12431, 2021
_lucifer
yeah, that is in line with my understanding. i remember you had shared a library the last year in context of BU db tests.
2021-05-04 12433, 2021
alastairp
ruaok made some work towards making integration tests work even if other data existed in the db, which means that we wouldn't even have to clear them after each test, but that involves a bit more work to ensure that tests are clearly isolated
2021-05-04 12445, 2021
ruaok
isolating individual tests isn't too hard, thankfully.
2021-05-04 12406, 2021
alastairp
it depends if we do things like "count all listens for a user" in some tests
yvanzo: reosarevok: do you have any thoughts on the best place to dump the new materialized tables, and think it'd be good to keep them optional so you can still use the website without them? at first I thought about -derived, but it'd increase the size of that quite a bit -- people who won't need these might appreciate them being in a seprate dump, like -denormalized
