lucifer: zas: what is the status of aretha / willaims?
2021-07-19 20001, 2021
yvanzo
mo’’in’
2021-07-19 20059, 2021
lucifer
morning
2021-07-19 20005, 2021
lucifer
jasondk: cool, thanks!
2021-07-19 20032, 2021
lucifer
ruaok: from what i see, barman has been moved but everything else is on williams.
2021-07-19 20041, 2021
ruaok
zas: can we work on this today, please>
2021-07-19 20043, 2021
ruaok
lucifer: what is the status with the dumps and importing data to spark from the 15th?
2021-07-19 20039, 2021
lucifer
sentry's clear and the dumps also imported fine yesterday. so i think it was a one time connectivity issue.
2021-07-19 20038, 2021
lucifer
let's generate the parquet dumps for testing?
2021-07-19 20049, 2021
ruaok
yes, let's. I'll get that moving.
2021-07-19 20058, 2021
ruaok
we're still running old cron right?
2021-07-19 20001, 2021
lucifer
yes
2021-07-19 20003, 2021
ruaok
k
2021-07-19 20005, 2021
lucifer
we should do one full dump and 2 inc dumps. so that we can test all 3 scenarios. full dump import, first inc dump import and subsequent inc dump imports.
2021-07-19 20026, 2021
ruaok
ok
2021-07-19 20038, 2021
lucifer
ruaok: also, how do continous aggregates work? do they refresh counts for current year or only older than 1 year?
2021-07-19 20041, 2021
ruaok
let me adjust the dummy dump code.
2021-07-19 20049, 2021
lucifer
👍
2021-07-19 20009, 2021
Etua joined the channel
2021-07-19 20041, 2021
ruaok
the cont aggs are currently set to refresh only 1 year automatically. then the cron job wakes up and processes 2 or 3 years in one go, until all time is done.
2021-07-19 20050, 2021
ruaok
this keeps the disk space usage low.
2021-07-19 20038, 2021
lucifer
makes sense, due to some reason counts on my profile have been broken for ~3 days now. i was hoping the cron job would fix it but it didn't.
2021-07-19 20030, 2021
ruaok
ok, I'll look into that in a bit once I get the spark stuff going
ruaok: moooin, yes, I'm transferring ftp data atm, we can start to move services, machine looks stable
2021-07-19 20044, 2021
ruaok
zas: cool and you can take care of the ssh keys? do we need to install new ones or reuse the existing ones?
2021-07-19 20049, 2021
zas
ssh keys for ? I guess we can copy them over
2021-07-19 20047, 2021
ruaok
for LB and other servivces to copy things to FTP.
2021-07-19 20053, 2021
ruaok
would be ideal if they could be copied over.
2021-07-19 20003, 2021
ruaok
lucifer: dumps started.
2021-07-19 20040, 2021
ruaok
lucifer: do you know if the listen counts are broken for only you or for all people?
2021-07-19 20006, 2021
lucifer
ruaok: they are broken for at least me. i checked a few other profiles and it looked okay for them. for me the counts were 0 (now ~100) so i noticed.
2021-07-19 20045, 2021
ruaok
have you deleted your listen count key in redis and then gone back to your profile page to force a recount?
2021-07-19 20002, 2021
ruaok
redis should be the first place to look at this.
2021-07-19 20031, 2021
lucifer
no i haven't done that yet. because i thought it might be useful to know why this happened in the first place.
2021-07-19 20057, 2021
lucifer
i'll try forcing the recount.
2021-07-19 20053, 2021
ruaok
its going to be hard to figure out why -- we should really replace the redis stuff with the solution that I suggested -- keeping the counts in TS and be done with it.
2021-07-19 20005, 2021
ruaok
(without relying on cont agg)
2021-07-19 20044, 2021
lucifer
agreed. +1
2021-07-19 20023, 2021
ruaok
ok, to get back into the swing of things and since i like pain, I'll add the listen_mbid_mapping table to the dumps.
2021-07-19 20039, 2021
ruaok
I suppose that ought to be a new dump file...
2021-07-19 20017, 2021
lucifer
yeah, new file makes sense.
2021-07-19 20027, 2021
lucifer
forcing recount fixed the issue.
2021-07-19 20006, 2021
ruaok
ok, keep your eye on it and see if it gets borked again.
> Distance of the Race in 2021 is around 2450km which makes it possible to finish it in around a week.
2021-07-19 20020, 2021
lucifer
ruaok: yes, the listen counts increment for me as well. the issue is my listens were around 8000 last week, then those reset to 0 due to some reason. from then they kept on incrementing as i listened to ~100 today.
2021-07-19 20058, 2021
ruaok
yeah, sounds like your redis key was lost. for whatever reason.
2021-07-19 20016, 2021
lucifer
possibly.
2021-07-19 20030, 2021
ruaok
alastairp: those rules. ouch.
2021-07-19 20036, 2021
alastairp
winner rolled in to arc de triomf 2 days ago
2021-07-19 20042, 2021
alastairp
after 6 days of riding
2021-07-19 20015, 2021
ruaok
sheeesh. thats freaking insane.
2021-07-19 20035, 2021
ruaok
~400km a day? over mountains? insane.
2021-07-19 20017, 2021
alastairp
I guess some people like doing that to themselves...
question for react experts: as a matter of style do you always have exactly 1 component per file, or if you have small components that are only used as a helper in a larger one will you include them in the same file?
2021-07-19 20027, 2021
tandy[m]
im trying to submit this listen, but im getting an error 500, any ideas?
i think its because the json has escape characters
2021-07-19 20026, 2021
ruaok
I doubt it -- then you'd get a 400 error.
2021-07-19 20034, 2021
ruaok
did you retry submitting it?
2021-07-19 20041, 2021
ruaok
it may be a transient error.
2021-07-19 20000, 2021
tandy[m]
no i didnt retry, but i have tried sending the submission a couple times
2021-07-19 20013, 2021
tandy[m]
```Error: unhandled exception: ERROR 500 Data requested not available [HttpRequestError]```
2021-07-19 20016, 2021
tandy[m]
thats the error
2021-07-19 20013, 2021
ruaok
please submit it again, so I can look for the error in the logs. tell me right after you submit.
2021-07-19 20023, 2021
peterhil joined the channel
2021-07-19 20024, 2021
tandy[m]
done
2021-07-19 20038, 2021
tandy[m]
the user id is tandy1000
2021-07-19 20045, 2021
tandy[m]
if that helps
2021-07-19 20024, 2021
monkey
alastairp: I'm not a purist, so if the helper component is small and not used anywhere else I think it's acceptable to keep it in the same file as the main component.
2021-07-19 20024, 2021
monkey
If there are more helper components I usually create a separate helpers file for them.
It's anoying that we can't see the whole data in the sentry issue
2021-07-19 20023, 2021
lucifer
sorry had fallen asleep. 😅
2021-07-19 20007, 2021
lucifer
i have seen that error before tandy[m]. my hunch is on an encoding issue. let me look deeper.
2021-07-19 20002, 2021
monkey
lucifer: Hi! I have another one for your plate :) There are some uncommitted changes in docker-server-configs on kiss that I don't want to mess up
2021-07-19 20030, 2021
lucifer
monkey: hi! just took a look, you can safely discard that. those are probably from ruaok updating cron earlier today for testing.
2021-07-19 20003, 2021
monkey
OK, thanks :)
2021-07-19 20032, 2021
lucifer
tandy[m]: your entire json is wrapped inside extra `''`. so its not a correct json payload. i am not sure of the source of those but place would be to look into your listen submission code.
2021-07-19 20012, 2021
lucifer
*first place
2021-07-19 20016, 2021
akshaaatt[m]
Hi lucifer ! Thanks for reviewing the PR. I am available in case any discussions are to be made.
2021-07-19 20051, 2021
akshaaatt[m]
I even got some good feedback from the community regarding the design, which I intend to incorporate soon.
2021-07-19 20029, 2021
lucifer
akshaaatt[m]: hi! sounds nice. i am trying to test your picard addition PR again and it does not seem to work this time. can you try again and see if it works for you.
2021-07-19 20048, 2021
lucifer
ruaok: the alert you set up seems to working nicely ;)
2021-07-19 20003, 2021
akshaaatt[m]
Sure!
2021-07-19 20027, 2021
kepstin has quit
2021-07-19 20027, 2021
JuniorJPDJ has quit
2021-07-19 20027, 2021
elomatreb[m] has quit
2021-07-19 20030, 2021
akshaaatt[m] has quit
2021-07-19 20030, 2021
yyoung[m] has quit
2021-07-19 20030, 2021
tandy[m] has quit
2021-07-19 20033, 2021
MagnusSvensson[m has quit
2021-07-19 20049, 2021
ruaok
oh good, I hadn't had a chance to test it yet.
2021-07-19 20014, 2021
ruaok
did you stop the metric writer to test it? or did something else happen?
2021-07-19 20041, 2021
lucifer
its hung up itself. i just came to know of it from the alerts.
