I received my GSoC acceptance, thank you metabrainz team! I'm really excited to work on listenbrainz this summer (and hopefully beyond), thanks for the great opportunity :)
RetroPunk has quit
RetroPunk joined the channel
thomasross has quit
agatzk joined the channel
yyoung
I have received my GSoc acceptance email, thanks to Metabrainz team. It is my honor to be part of this community, and I'm glad to work with my mentor yvanzo, hope we'll have a good time coding this summer :)
D4RK-PH0ENiX has quit
_lucifer
akshaaatt[m], we can continue most of the talks and work in the channel. but feel free to PM me anytime you need. regarding a call, let's see what others decide and we can do accordingly and yes I am from India as well.
!m bitmap, yvanzo and reosarevok
BrainzBot
You're doing good work, bitmap, yvanzo and reosarevok!
_lucifer
congrats jasondk_ and yyoung! :D
ruaok, just checked the logs on leader, full import succeeded.
D4RK-PH0ENiX joined the channel
akshaaatt[m]
<_lucifer "akshaaatt, we can continue most "> Sounds great! :)
Kevin12 joined the channel
Kevin12 has quit
akashgp09 joined the channel
mglubb joined the channel
sumedh joined the channel
akashgp09 has quit
ruaok
moooin!
!m _lucifer
BrainzBot
You're doing good work, _lucifer!
ruaok
have you checked to see if any of the stats ran? if not, we should request a full run. I plan to do nothing with the cluster today, so lets run everything.
_lucifer
yes some of the requests had run when i last checked.
sure lets run all.
mglubb has quit
ruaok
_lucifer: ohhh sorry. I didn't realize that leader was actually use in the spark configuration. that was never the intent; had I known I wouldn't have changed it on you. sorry!
(it was always meant to be a shortcut to michael, nothing more)
_lucifer
ruaok, ah no its not used in spark configuration. you are talking about docker-server-configs PR right?
ruaok
yes. either one wasn't intended to be using it, but I never communicated that to you, sorry.
the same things keep popping up in the setup at different places so its a bit confusing. that's why i am putting together a document with general notes and observations on the setup.
ruaok
very good
shivam-kapila
Congrats GSoC students.
shivam-kapila goes to see who all are accepted
sumedh has quit
Congrats mentor _lucifer :p
_lucifer
:)
shivam-kapila
Ah the propsals sound really good. Exciting things coming. All the best fellas
I think we should compact both of these and move them into one crontab -- having two is harder to manage.
_lucifer
oh! makes sense.
i'll add a new command to request all kinds of stats at once.
ruaok
makes sense.
thanks
ruaok goes to look after diskspace on lemmy
_lucifer
all stats, similar users, recommendations requested.
ruaok
sweet.
_lucifer
are we going to upgrade lemmy today?
ruaok
54% disk space used.
we upgrade when hetzner says we upgrade. there is no ETA, no insight, just beer lacking bavarians being grumpy.
_lucifer
:(
ruaok
does anyone of the gsoc mentors feel like writing a "welcome gsoc students" email?
I think tomorrow might need to be PR merge day...
_lucifer
alastairp, i think that command issue might be coming from weird interactions between the default docker builder and buildkit. I just noticed that docker's default builder fills the `ContainerConfig` section seen in docker inspect but BuildKit doesn't.
ruaok
questionable bug ticket name of the day: "Further dump hardening"
_lucifer
while we haven't started using BuildKit in actions, I have locally started using it so it might be possible that one of my pushed images was built using BuildKit but actions' is using default docker builder so the error happened.
lol 🤣
ruaok
_lucifer: the dumps logs have gone stale since may 11.
_lucifer
how so?
ruaok
`2021-05-12 13:00:05,335 INFO Created channel=1`
is the last entry.
nor am I getting mails about dumps being out of date.
if you get a minute to have a look, please do. if not, I'll pick it up this afternoon.
but I gotta go now and make my way towards getting a vaccine...
shivam-kapila
Niceee
_lucifer
noice :D
i'll look into dumps issue.
zas
yvanzo, bitmap : we just got an alert for high number of WALs
_lucifer
ruaok, we have had two incremental dumps since 12th, one on 13 and one on 16, it looks like both were manually triggered. was that so?
ruaok
that sounds about right.
well, not the 16th, no.
_lucifer
my theory was that dump-crontab is not being run due to some reason. but if crontab did run on 16th then that's wrong.
ruaok
well, we've been starting/restarting so many containers, it is hard to know.
Mr_Monkey
ruaok: I'm taking a stab at writing a welcome email. Will send you a draft
ruaok
Mr_Monkey: thank you!
_lucifer
ruaok: i am not familiar with how cron works but a lot of this coming up the fact each crontab is user specific so adding entries from different users to same crontab might be causing issues.
alastairp
morning
_lucifer: we don't (or at least we shouldn't) install things into user-specific crontabs. we use global files
_lucifer
ah ok!
also, i was looking for logs from refresh aggregates job to see if that ran but it seems that job doesn't log anything.
_lucifer: let's open a PR for 1) adding this username to the dumps crontab, 2) merging all stats commands into a single item (which we can then move into a single crontab with dumps)
_lucifer
sure, on it.
alastairp
I think you understand the spark manage stuff better than me
ruaok
thanks alastairp, _lucifer
alastairp
do you want to make a new command? I'll open PR with the username
ruaok
maybe I'll just take the rest of the day off and... chill a bit.
alastairp
especially after a jab, I'd recommend it
_lucifer
alastairp, i was thinking to add a new command and then use context.invoke to invoke all the other ones.
ruaok
well, ping me if ya really need me. otherwise, I'll be fucking off. :)
alastairp
_lucifer: sounds fine
ruaok: see you tomorrow, then
_lucifer
have a nice day, ruaok :)
alastairp
fingers crossed lemmy doesn't fall over, lol
_lucifer
lol
alastairp
ruaok: should we run continuous agg update manually?
ruaok
sure, do it in tmux. takes a while.
alastairp
will do.
_lucifer
there's also a logging error in the command to refresh aggregates. will fix in the PR.
it seems that |1 denoted the build arg's in the dockerfile, but no idea how that ended up in the CMD
unable to reproduce the issue currently.
alastairp
yeah, it seems like this is something that docker does to keep track of how it built an image, but somehow it jumped from a RUN to a CMD, which is weird
_lucifer: I'm running refresh_continuous_aggregates in a shell in cron now.