i thought that we could store and retrieve as the values as integers. but i just tested with redis-cli and that is not the case, it returns integers as strings.
2021-05-07 12736, 2021
_lucifer
so just ignore that comment.
2021-05-07 12709, 2021
alastairp
the only gain perhaps is that logically the values are all joined together in a single key
2021-05-07 12738, 2021
alastairp
instead of having 2 keys per user, there are just 2 count keys in redis
2021-05-07 12744, 2021
_lucifer
ah, alastairp's motivation was different.
2021-05-07 12705, 2021
alastairp
_lucifer: a reminder, all counts in redis are always strings
2021-05-07 12718, 2021
alastairp
even if you use `INCR foo`, it'll be stored in a string
2021-05-07 12743, 2021
_lucifer
oh! i didn't know that.
2021-05-07 12711, 2021
ruaok
good enough for now
2021-05-07 12725, 2021
ruaok
353850 listens processed: exact 78138 high 800 med 2426 low 2697 no 23198 prev 246591 err 0 93.4%
2021-05-07 12757, 2021
ruaok
some really nice stats coming from the MBID writer working on live listens coming in.
2021-05-07 12721, 2021
alastairp
ruaok: listenstore review done
2021-05-07 12741, 2021
alastairp
-> lunch. back to answer questions soon
2021-05-07 12751, 2021
piti has quit
2021-05-07 12715, 2021
ruaok
all comments addressed, most fixed.
2021-05-07 12753, 2021
alastairp
I tried the select max(listened_at) query without a group by and it gave me a result
2021-05-07 12704, 2021
alastairp
not sure what's different in your one
2021-05-07 12710, 2021
ruaok
I took it out and it barfed on me.
2021-05-07 12753, 2021
ruaok
different timescale versions?
2021-05-07 12758, 2021
alastairp
you took out just group by, or group by and order by?
2021-05-07 12709, 2021
ruaok
just group by
2021-05-07 12710, 2021
alastairp
I can reproduce the error if I take out just group by
2021-05-07 12724, 2021
alastairp
but you don't need order by, because you're already using max/min?
2021-05-07 12755, 2021
ruaok
LIMIT 1
2021-05-07 12705, 2021
ruaok
🤦‍♂️
2021-05-07 12710, 2021
alastairp
but max(listened_by) only returns 1 row
2021-05-07 12750, 2021
ruaok
I wonder if this was confusing the query planner making things slow.
2021-05-07 12727, 2021
ruaok
but now tests are failing. huh
2021-05-07 12702, 2021
BenOckmore has quit
2021-05-07 12757, 2021
BrainzGit
[bookbrainz-site] MonkeyDo merged pull request #623 (master…dependabot/npm_and_yarn/underscore-1.13.1): chore(deps): bump underscore from 1.10.2 to 1.13.1 https://github.com/bookbrainz/bookbrainz-site/pul…
_lucifer: looks like this job may not have used any docker cache. any thoughts as to why? Either because it's the first build since we fixed the ARGs, or because it failed to store the cache the last time it ran?
2021-05-07 12732, 2021
_lucifer
the latter or because the cache got evicted.
2021-05-07 12722, 2021
_lucifer
the tests run often but the release is seldom so its probable that the release cache gets evicted.
2021-05-07 12712, 2021
alastairp
right. I'm not sure - 10 minutes for a build is quite a long time. I know that if I do a small incremental release locally (e.g a .0 then a .1) then the build time is going to be much smaller
2021-05-07 12714, 2021
alastairp
shorter
2021-05-07 12738, 2021
_lucifer
yeah, right.
2021-05-07 12700, 2021
ruaok
gah. refreshing the 30day continuous aggregate also takes up gobs of diskspace. :(
2021-05-07 12734, 2021
_lucifer
the cache got stored this time, alastairp.
2021-05-07 12707, 2021
_lucifer
ruaok, could this be related to WITH NO DATA option?
2021-05-07 12728, 2021
ruaok
it is.
2021-05-07 12744, 2021
ruaok
but then you would normally refresh the agg and it would run into the same issues.
2021-05-07 12708, 2021
ruaok
I need to see if I can have it refresh the agg in chunks (5 years at a time)
2021-05-07 12717, 2021
yvanzo
nelgin: There is a more general issue: These logs don’t provide enough details to identify the edit(s) that caused the failure.
SEARCH-577: Index limit exceeded on medium formats update
2021-05-07 12709, 2021
yvanzo
It’s doable to increase your 'index_limit' value but to the risk of PostgreSQL being less available for other tasks during such events.
2021-05-07 12751, 2021
yvanzo
As a workaround... ^
2021-05-07 12731, 2021
yvanzo
I will resume addressing indexer’s issues (and prerequisites to port to python 3) after the next schema change release.
2021-05-07 12729, 2021
ruaok
wow, gaga. 1210M/s write speed. zoooooom!
2021-05-07 12727, 2021
ruaok
still, hurry up, eh gaga?
2021-05-07 12716, 2021
alastairp
what's going on? You can generate the new aggregate before we deploy, then when we deploy it'll start using it, and then we can delete the old one?
2021-05-07 12724, 2021
ruaok
that is what I am doing.
2021-05-07 12735, 2021
ruaok
its stopped consuming disk, so now just need to wait.
2021-05-07 12744, 2021
_lucifer
alastairp: if you are idle currently, can you take a look at the youtube PR?
2021-05-07 12745, 2021
alastairp
I approved them
2021-05-07 12723, 2021
alastairp
we're going to have to have a talk about this workflow, I can see that I'm a bottleneck for reviews and I'm not sure how we can set things up to get them processed quicker
2021-05-07 12724, 2021
_lucifer
thanks!
2021-05-07 12725, 2021
_lucifer
one issue regarding the time ranges and youtube PR has been that these are huge PRs. if we could do small incremental PRs, merges and deployments. it would have been easier.
2021-05-07 12746, 2021
_lucifer
but i don't think it was possible in these 2 cases.
2021-05-07 12713, 2021
_lucifer
although we could probably setup a weekly meeting sort of thing like the MB team have.
2021-05-07 12705, 2021
alastairp
yeah, iliekcomputers has commented about the value of small PRs before, and I agree there
2021-05-07 12719, 2021
alastairp
but sometimes a change has to be big (time ranges, this one, etc)
2021-05-07 12743, 2021
_lucifer
yeah, sometimes there's no option but otherwise we should aim for small PRs.
2021-05-07 12751, 2021
alastairp
a few times for SoC I've tried to do these incremental PRs like you did here, but in my experience that's always gone badly too
2021-05-07 12715, 2021
_lucifer
yeah, rebasing so on becomes a mess.
2021-05-07 12734, 2021
ruaok
i don't rebase anymore -- it always turns into a painful mess. merging master in works much better for me.
2021-05-07 12753, 2021
ruaok
ok, aggregate built, finally.
2021-05-07 12757, 2021
ruaok
starting deploy.
2021-05-07 12710, 2021
ruaok
taking taking web-test for the time being.
2021-05-07 12734, 2021
_lucifer
yeah, i gave up on rebasing last week too.
2021-05-07 12721, 2021
_lucifer
alastairp, ruaok, Mr_Monkey: so how about we try to aim for a weekly release, we can set a day in advance that works for all of us. that day we try to review and get merged as many PRs as possible and release at the end of day. just like the PR smashing days we had some time ago, but only every week.
so the 'prod' image on docker hub is the one we want
2021-05-07 12739, 2021
_lucifer
yes right
2021-05-07 12723, 2021
alastairp
_lucifer: it's worth a try setting a release day, though I don't want the day to turn into a panic to try and merge as many PRs as possible before we release
2021-05-07 12746, 2021
alastairp
I think our periodic "fix prs" day that we do at the office is working well, though the last time we did that we definitely picked up the easy ones
alastairp: right, i agree. we should aim for a reasonable amount of changes.
2021-05-07 12715, 2021
alastairp
_lucifer: the job failed to pull cache layers again
2021-05-07 12723, 2021
ruaok
though if .1 is done and ready, I'll use that.
2021-05-07 12738, 2021
alastairp
nah, it'll be another 10 minutes, because for some reason the build job isn't using a cache
2021-05-07 12743, 2021
ruaok
k
2021-05-07 12702, 2021
ruaok
these build times are getting redonkulous
2021-05-07 12743, 2021
alastairp
yeah, this 10 minutes is for a full LB prod image. we added caching in the github actions precicesly to avoid this, and when building locally on your machine it should reuse caches
2021-05-07 12752, 2021
alastairp
but even then, it's a pretty long process from 0
2021-05-07 12709, 2021
ruaok
yeah
2021-05-07 12716, 2021
alastairp
growing up in a country at the far end of a single cable to the rest of the internet still makes me cringe at downloading the same packages again and again every time you want to make a new deployment. it feels like such a waste
2021-05-07 12731, 2021
ruaok
huh? still not found???
2021-05-07 12747, 2021
ruaok
is push.sh busted?
2021-05-07 12754, 2021
ruaok
do I need to merge another branch?
2021-05-07 12724, 2021
_lucifer
the image is not pushed yet, a couple of minutes more.
2021-05-07 12732, 2021
alastairp
how did you try and push it yourself?
2021-05-07 12733, 2021
ruaok
MY image didn't appear.
2021-05-07 12746, 2021
ruaok
docker/push.sh prod v-2021-05-07.0
2021-05-07 12752, 2021
alastairp
sorry - I thought you pulled :prod and renamed it to v-2021-05-07.0
2021-05-07 12756, 2021
_lucifer
you do not need prod now.
2021-05-07 12705, 2021
alastairp
ok. we made this change and didn't inform you of it - sorry about that
2021-05-07 12717, 2021
_lucifer
docker/push.sh v-2021-05-07.0
2021-05-07 12724, 2021
alastairp
the syntax is now ^
2021-05-07 12749, 2021
_lucifer
the action has also pushed now, so you can use that for now.
2021-05-07 12723, 2021
ruaok
finally found it.
2021-05-07 12745, 2021
_lucifer
alastairp, i'll enable DEBUG logging on the actions and see if I can find the issue.
2021-05-07 12715, 2021
alastairp
_lucifer:
2021-05-07 12717, 2021
alastairp
> Stored root cache, key: docker-layer-caching-Build image and publish to Docker Hub-61ad5188f09e6d2abe065a8e450c87c8238c1c1a0f06684921281fbebe0e9d70-root, id: 50510
