I received my GSoC acceptance, thank you metabrainz team! I'm really excited to work on listenbrainz this summer (and hopefully beyond), thanks for the great opportunity :)
2021-05-18 13838, 2021
RetroPunk has quit
2021-05-18 13831, 2021
RetroPunk joined the channel
2021-05-18 13833, 2021
thomasross has quit
2021-05-18 13836, 2021
agatzk joined the channel
2021-05-18 13837, 2021
yyoung
I have received my GSoc acceptance email, thanks to Metabrainz team. It is my honor to be part of this community, and I'm glad to work with my mentor yvanzo, hope we'll have a good time coding this summer :)
2021-05-18 13810, 2021
D4RK-PH0ENiX has quit
2021-05-18 13804, 2021
_lucifer
akshaaatt[m], we can continue most of the talks and work in the channel. but feel free to PM me anytime you need. regarding a call, let's see what others decide and we can do accordingly and yes I am from India as well.
2021-05-18 13817, 2021
_lucifer
!m bitmap, yvanzo and reosarevok
2021-05-18 13817, 2021
BrainzBot
You're doing good work, bitmap, yvanzo and reosarevok!
2021-05-18 13835, 2021
_lucifer
congrats jasondk_ and yyoung! :D
2021-05-18 13822, 2021
_lucifer
ruaok, just checked the logs on leader, full import succeeded.
2021-05-18 13855, 2021
D4RK-PH0ENiX joined the channel
2021-05-18 13821, 2021
akshaaatt[m]
<_lucifer "akshaaatt, we can continue most "> Sounds great! :)
2021-05-18 13830, 2021
Kevin12 joined the channel
2021-05-18 13814, 2021
Kevin12 has quit
2021-05-18 13853, 2021
akashgp09 joined the channel
2021-05-18 13801, 2021
mglubb joined the channel
2021-05-18 13811, 2021
sumedh joined the channel
2021-05-18 13810, 2021
akashgp09 has quit
2021-05-18 13837, 2021
ruaok
moooin!
2021-05-18 13839, 2021
ruaok
!m _lucifer
2021-05-18 13840, 2021
BrainzBot
You're doing good work, _lucifer!
2021-05-18 13812, 2021
ruaok
have you checked to see if any of the stats ran? if not, we should request a full run. I plan to do nothing with the cluster today, so lets run everything.
2021-05-18 13843, 2021
_lucifer
yes some of the requests had run when i last checked.
2021-05-18 13848, 2021
_lucifer
sure lets run all.
2021-05-18 13820, 2021
mglubb has quit
2021-05-18 13833, 2021
ruaok
_lucifer: ohhh sorry. I didn't realize that leader was actually use in the spark configuration. that was never the intent; had I known I wouldn't have changed it on you. sorry!
2021-05-18 13851, 2021
ruaok
(it was always meant to be a shortcut to michael, nothing more)
2021-05-18 13852, 2021
_lucifer
ruaok, ah no its not used in spark configuration. you are talking about docker-server-configs PR right?
2021-05-18 13813, 2021
ruaok
yes. either one wasn't intended to be using it, but I never communicated that to you, sorry.
the same things keep popping up in the setup at different places so its a bit confusing. that's why i am putting together a document with general notes and observations on the setup.
2021-05-18 13831, 2021
ruaok
very good
2021-05-18 13808, 2021
shivam-kapila
Congrats GSoC students.
2021-05-18 13818, 2021
shivam-kapila goes to see who all are accepted
2021-05-18 13857, 2021
sumedh has quit
2021-05-18 13801, 2021
shivam-kapila
Congrats mentor _lucifer :p
2021-05-18 13814, 2021
_lucifer
:)
2021-05-18 13829, 2021
shivam-kapila
Ah the propsals sound really good. Exciting things coming. All the best fellas
I think we should compact both of these and move them into one crontab -- having two is harder to manage.
2021-05-18 13838, 2021
_lucifer
oh! makes sense.
2021-05-18 13804, 2021
_lucifer
i'll add a new command to request all kinds of stats at once.
2021-05-18 13816, 2021
ruaok
makes sense.
2021-05-18 13820, 2021
ruaok
thanks
2021-05-18 13828, 2021
ruaok goes to look after diskspace on lemmy
2021-05-18 13812, 2021
_lucifer
all stats, similar users, recommendations requested.
2021-05-18 13820, 2021
ruaok
sweet.
2021-05-18 13826, 2021
_lucifer
are we going to upgrade lemmy today?
2021-05-18 13827, 2021
ruaok
54% disk space used.
2021-05-18 13854, 2021
ruaok
we upgrade when hetzner says we upgrade. there is no ETA, no insight, just beer lacking bavarians being grumpy.
2021-05-18 13825, 2021
_lucifer
:(
2021-05-18 13859, 2021
ruaok
does anyone of the gsoc mentors feel like writing a "welcome gsoc students" email?
2021-05-18 13808, 2021
ruaok
I think tomorrow might need to be PR merge day...
2021-05-18 13810, 2021
_lucifer
alastairp, i think that command issue might be coming from weird interactions between the default docker builder and buildkit. I just noticed that docker's default builder fills the `ContainerConfig` section seen in docker inspect but BuildKit doesn't.
2021-05-18 13813, 2021
ruaok
questionable bug ticket name of the day: "Further dump hardening"
2021-05-18 13828, 2021
_lucifer
while we haven't started using BuildKit in actions, I have locally started using it so it might be possible that one of my pushed images was built using BuildKit but actions' is using default docker builder so the error happened.
2021-05-18 13841, 2021
_lucifer
lol 🤣
2021-05-18 13855, 2021
ruaok
_lucifer: the dumps logs have gone stale since may 11.
2021-05-18 13823, 2021
_lucifer
how so?
2021-05-18 13808, 2021
ruaok
`2021-05-12 13:00:05,335 INFO Created channel=1`
2021-05-18 13817, 2021
ruaok
is the last entry.
2021-05-18 13828, 2021
ruaok
nor am I getting mails about dumps being out of date.
2021-05-18 13838, 2021
ruaok
if you get a minute to have a look, please do. if not, I'll pick it up this afternoon.
2021-05-18 13856, 2021
ruaok
but I gotta go now and make my way towards getting a vaccine...
2021-05-18 13817, 2021
shivam-kapila
Niceee
2021-05-18 13814, 2021
_lucifer
noice :D
2021-05-18 13823, 2021
_lucifer
i'll look into dumps issue.
2021-05-18 13834, 2021
zas
yvanzo, bitmap : we just got an alert for high number of WALs
2021-05-18 13858, 2021
_lucifer
ruaok, we have had two incremental dumps since 12th, one on 13 and one on 16, it looks like both were manually triggered. was that so?
2021-05-18 13814, 2021
ruaok
that sounds about right.
2021-05-18 13823, 2021
ruaok
well, not the 16th, no.
2021-05-18 13830, 2021
_lucifer
my theory was that dump-crontab is not being run due to some reason. but if crontab did run on 16th then that's wrong.
2021-05-18 13850, 2021
ruaok
well, we've been starting/restarting so many containers, it is hard to know.
2021-05-18 13820, 2021
Mr_Monkey
ruaok: I'm taking a stab at writing a welcome email. Will send you a draft
2021-05-18 13828, 2021
ruaok
Mr_Monkey: thank you!
2021-05-18 13808, 2021
_lucifer
ruaok: i am not familiar with how cron works but a lot of this coming up the fact each crontab is user specific so adding entries from different users to same crontab might be causing issues.
2021-05-18 13844, 2021
alastairp
morning
2021-05-18 13806, 2021
alastairp
_lucifer: we don't (or at least we shouldn't) install things into user-specific crontabs. we use global files
2021-05-18 13838, 2021
_lucifer
ah ok!
2021-05-18 13849, 2021
_lucifer
also, i was looking for logs from refresh aggregates job to see if that ran but it seems that job doesn't log anything.
_lucifer: let's open a PR for 1) adding this username to the dumps crontab, 2) merging all stats commands into a single item (which we can then move into a single crontab with dumps)
2021-05-18 13830, 2021
_lucifer
sure, on it.
2021-05-18 13838, 2021
alastairp
I think you understand the spark manage stuff better than me
2021-05-18 13844, 2021
ruaok
thanks alastairp, _lucifer
2021-05-18 13851, 2021
alastairp
do you want to make a new command? I'll open PR with the username
2021-05-18 13807, 2021
ruaok
maybe I'll just take the rest of the day off and... chill a bit.
2021-05-18 13821, 2021
alastairp
especially after a jab, I'd recommend it
2021-05-18 13854, 2021
_lucifer
alastairp, i was thinking to add a new command and then use context.invoke to invoke all the other ones.
2021-05-18 13803, 2021
ruaok
well, ping me if ya really need me. otherwise, I'll be fucking off. :)
2021-05-18 13809, 2021
alastairp
_lucifer: sounds fine
2021-05-18 13817, 2021
alastairp
ruaok: see you tomorrow, then
2021-05-18 13819, 2021
_lucifer
have a nice day, ruaok :)
2021-05-18 13823, 2021
alastairp
fingers crossed lemmy doesn't fall over, lol
2021-05-18 13827, 2021
_lucifer
lol
2021-05-18 13835, 2021
alastairp
ruaok: should we run continuous agg update manually?
2021-05-18 13856, 2021
ruaok
sure, do it in tmux. takes a while.
2021-05-18 13802, 2021
alastairp
will do.
2021-05-18 13816, 2021
_lucifer
there's also a logging error in the command to refresh aggregates. will fix in the PR.
it seems that |1 denoted the build arg's in the dockerfile, but no idea how that ended up in the CMD
2021-05-18 13827, 2021
_lucifer
unable to reproduce the issue currently.
2021-05-18 13831, 2021
alastairp
yeah, it seems like this is something that docker does to keep track of how it built an image, but somehow it jumped from a RUN to a CMD, which is weird
2021-05-18 13845, 2021
alastairp
_lucifer: I'm running refresh_continuous_aggregates in a shell in cron now.
