Also helped a little bit with some of the smaller things with the move, but nothing compared to the devs.
And I upgraded Discourse.
fin, I think.
reosarevok: Go.
LordSputnik
Freso: Don't think Leftmost is here today - his review would probably be something like "Started reviewing large entity editing pull request, asked LordSputnik some questions, possibly did some sketches for new entity display UI"
Freso
LordSputnik: If he's not here, he's not here, and that's fine too.
reosarevok
I spent half of the week in Germany. The rest of the week was spent on support plus reviewing all the edits and whatnot that I didn't review in Germany
Btw, jira is not sending me style tickets anymore, or any mail really, which is bad. So we should update or fix
ruaok
as soon as we put out the other fires, sure.
chirlu is glad not to get mail, and had JIRA configured accordingly. ;-)
Freso
reosarevok: Wiki isn't either, AFAICT, so may be a wider problem.
reosarevok
Fun, anyway
Er. Fin
chirlu
I followed the NewHost migration and bossed people around to add banners, tweet things, change banners, remove n.* testing sites, remove banners, …
Freso
:p
!m chirlu
BrainzBot
You're doing good work, chirlu!
chirlu
In total, I was surprised how well the migration went. I had expected more serious issues with the new site.
Freso
Too bad about the replication packet(s). :(
chirlu
I also answered some things on Discourse and IRC.
Furthermore, I closed a number of JIRA issues that were fixed or invalidated by the migration.
darwin
is there any data on before/after response times?
Leo_Verto did actually get some work done since the last meeting
zas
darwin: just very bad to very good... most slow things are now due to slow requests, or lack of software optimization
chirlu
I also finally removed the old defunct FreeDB gateway VM and set up an MB testing server again, which I hadn’t had since rika disappeared.
Speaking of rika, I retrieved the reportstool data from it and plan to make that available again at some point.
Freso
🙌
chirlu
Finally, I changed the account creation CAPTCHA to make use of reCAPTCHA v2, making it more difficult for spammers and others.
Fin.
Freso also fetched i18n and will try to set up a new i18n site when/if we get something similar to rika again
now i'm trying to get all metrics we need to proceed to optimizations
also testing everything bits to detect issues, and trying to fix them
i detected many small issues that will be addressed this week
overall, the performance is very good, even though we have to dig why web servers are so cpu-bound, why some pages are still slow
etc...
i'm pretty happy with new infrastructure, even if many stuff is a bit hacky (we gave up on docker swarm for example)
more to come, but we finally did it, in time, and apart the lost replication packet, emails problems, and plenty of minor things, it was pretty smooth.
fin.
Freso
(People still up: ruaok alastairp? Leo_Verto?)
zas
ruaok ,
ruaok
k
first off, I want to thank everyone who had a hand in the migration.
THANK YOU!
I know few of us got enough sleep and we all worked far too much.
zas
so true ;)
ruaok
I personally was so stressed in California that I never got more than about 5 hours of sleep the whole time I was there.
I'm trying to muster the energy to carry on with the replication packet recovery nonsense. it is already hairy and complicated and getting worse by the minute.
oh well.
for most people restarting replication isn't a big deal -- I think the BBC will be most affected.
still ongoing are the recovery efforts for the drive in cali.
currently the disk is being cloned to a working disk
once that process is done, my friend is going to run fsck on the drive and see what comes out the other end.
we may get packets, we may not.
in total there are two lost packets.
bitmap
your friend is a hero for doing all this
ruaok
we won't know for a few more hours since cloning a failing drive is a slow process.
srsly.
we'll have to sort out an appropriate thank you for him.
once we push past the myriad of issues we need to address this week, we should stand on much more solid ground.
hibiscuskazeneko has quit
darwin
dd_rescue?
Freso
A metric ton of chocolate?
chirlu
darwin: Yes.
ruaok
we've gone through a massive revamp of the org and our infrastructure. I'm really looking forward to more normal day-to-day activities soon.
darwin
dd_rescue saved my ass once when I had two drives fail at once in a md raid5
ruaok
so, thanks team. thanks for kicking ass zas. thanks for rocking it Gentlecat. thanks for making it happen bitmap. thanks for being so bossy, chirlu.
and thanks for everyone else who was helping and or at least making smart ass comments from the sidelines.
fin.
Freso: who is left, I forget.
bitmap
:)
Leo_Verto
✋
ruaok picks the first one on the list
ruaok
Leo_Verto:
Freso
Leo_Verto :)
zas
And thanks to ruaok :)
Leo_Verto
I redesigned https://chatlogs.metabrainz.org/ to better suit our use case (non-public logging) and added some fancy MeB branding. Fin.
Freso
🙌
"non-public logging"?
How is it non-public?
Leo_Verto
Well, botbotme is designed to allow anyone to add channels to be logged
they kinda removed most of their branding and the frontend for those features but their public repo is not a very nice state since that
Freso
Sure, but there must be a better name for that. The logs are still public. :)
Anyway. Carry on.
(And thank you!)
hibiscuskazeneko joined the channel
Leo_Verto
And working with botbot is always kind of a pain but nowhere close to migrating the entire meb infrastructure :P
Freso
:)
Leo_Verto
that's everything from me. fin (for real this time).
Freso
I don't think anyone else has spoken up, so...
Freso: GCI
Woo!
We got accepted again this year!
gcilou
🙌
Freso
(For anyone who didn't get the memo last week. :))
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | MeB meeting [20 UTC] agenda: Google Code-in (Freso), spam editor accounts (Freso), replication trouble (chirlu), DR: MBS-8830 (Gentlecat)
I've slowly been getting up to speed, but LordSputnik's already started adding tasks.
I added a handful of mentors to the GCI site this weekend, so if anyone's who's talked about being a mentor this year that I forgot about, please poke me.
And if we didn't talk but you still want to be a mentor, talk to me and we can likely make it happen. We can't really have too many mentors. :)
yvanz pokes Freso
More mentors who might want to mentor MBS coding tasks in particular would be amazing.
yvanz: :D Let's talk after the meeting. :)
Anyway. Just a heads up that it's on. If you think of something that might be a good GCI task, poke me, we can always use more tasks. :)
bitmap
when does gci start?
gcilou
Nov 28
Freso
Nov. 28th.
Leo_Verto
Two weeks from now
Freso
^ any of those three. :)
I feel like there was one more thing I wanted to talk about, but I forgot. So eh.
Freso: Spam editor accounts.
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | MeB meeting [20 UTC] agenda: spam editor accounts (Freso), replication trouble (chirlu), DR: MBS-8830 (Gentlecat)
There are *a lot* of obvious passive spam profiles.
chirlu's updated the registration captcha now, so hopefully the influx of spam accounts will be somewhat decreased now, but there's still 1000s of accounts (I'm guesstimating) from 2013 and to now.
chirlu
Hundreds of thousands.
Freso
Can anyone remember what we did in 2013 to clean up the accounts? A query or whatever?
Because it would be nice to do a massive clean-up again.
ruaok
yes. the accounts had a certain pattern in their usernames.
bitmap
didn't they all have very similar email addresses or something?
ruaok
we identified and nuked all the ones that met that criteria and were created within a certain timeframe.
Freso
Every once in a while the user admin editors will get flooded with someone getting zealous about reporting spam profiles.
bitmap: I've been keeping an eye on the mail addresses of the ones I've manually deleted. There's no pattern. Plenty of hotmail, gmail, yahoo, mail.ru, and plenty others. And some with no mail at all.
G0re joined the channel
ruaok
once things calm down a bit we should do an analysis of dormant accounts.
Freso
I doubt anyone is free to look into this for the next couple of weeks though.
Yeah.
bitmap
about gci, I have a responsibility to help with mbs stuff though I don't know if I can do 100% effort
Freso
I'll make a mental note to bring it up in December/January.
chirlu
Last time we discussed this, some ideas were collected in MBS-8735.
ruaok
figure out which ones we can nuke or finally implement a "no activity for 6 months on a new account = delete"
policy
chirlu
BTW, reCAPTCHA can be tuned to be even more difficult. Needs someone with Google Webmaster Tools access for mb.o.
Freso
chirlu: Right. I had thought about that, but forgotten it again.
Anyway. Let's look at it in a few weeks/months again and move on for now.
Gore|home has quit
(I think I have webmaster tools access, maybe. I can check at least.)
chirlu: Replication trouble.
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | MeB meeting [20 UTC] agenda: replication trouble (chirlu), DR: MBS-8830 (Gentlecat)
chirlu
This was mentioned before already.
On the migration, the newer replication packets got lost.
This means that data users are in many cases stuck at a pre-migration state and can’t get their slave database updated.
But I guess there is not much to discuss here now. We looked into various points the whole day today already.
In the end, it depends on whether the missing files can be restored from the broken HD (via dd_rescue).
