We have to think about those details before the switch, since we'll do it in steps
2016-04-29 12007, 2016
zas
Preparation will include updates to DNS (TTL reduction etc...) and to GMail (whitelist new IPs)
2016-04-29 12035, 2016
ruaok
can CAA move later?
2016-04-29 12039, 2016
ruaok
or does it have to move with MB?
2016-04-29 12047, 2016
ruaok
I'm afraid it has to move with MB.
2016-04-29 12058, 2016
zas
I think too, bitmap could tell more
2016-04-29 12040, 2016
zas
FTP should move too at the same time imho
2016-04-29 12048, 2016
ruaok
maybe start adding a preparation section?
2016-04-29 12054, 2016
ruaok
yeah, FTP is on the first to move list now.
2016-04-29 12033, 2016
ruaok
I guess anything that is running via musicbrainz.org without a subdomain must first be split or moved first.
2016-04-29 12047, 2016
ruaok
migrating postgres will be fun. :)
2016-04-29 12056, 2016
ruaok
the process as I understand it:
2016-04-29 12019, 2016
ruaok
1. turn on WAL replication with a generous amount of buffer space.
2016-04-29 12024, 2016
ruaok
2. dump the DB
2016-04-29 12029, 2016
ruaok
3. Copy across the atlantic.
2016-04-29 12039, 2016
ruaok
4. import into new postgres
2016-04-29 12051, 2016
ruaok
5. turn on replication, and hope that the WAL was big enough for the replicant to catch on.
2016-04-29 12001, 2016
ruaok
6. run new infrastructure in read-only mode and test.
2016-04-29 12009, 2016
ruaok
7. flip master from CA to EU
2016-04-29 12047, 2016
ruaok
In theory there should be almost no visible downtime for users. a few minutes of DNS uncertainty for sure.
2016-04-29 12001, 2016
ruaok
and then of course... search.
2016-04-29 12005, 2016
ruaok starts crying
2016-04-29 12040, 2016
zas
Well, search servers can be installed fast (Chef), but they'll suck on new servers as they do on old ones ;)
2016-04-29 12055, 2016
zas
And i fear they can't handle much more requests ...
2016-04-29 12045, 2016
reosarevok
Well we have GSoC for that at least now...
2016-04-29 12051, 2016
ruaok
reosarevok: well, sorta.
2016-04-29 12003, 2016
ruaok
weeksio is waffling on that.
2016-04-29 12041, 2016
ruaok
he wants to basiacally do two internships over the summer.
2016-04-29 12058, 2016
ruaok
I fear that I may need to just do this myself just so it gets fucking done.
2016-04-29 12019, 2016
ruaok
but I was planning going to lots of festivals this summer. not sure how that would work.
2016-04-29 12001, 2016
zas
ruaok: about new servers, i think web servers don't need RAID at all, if one fails we take it out of rotation, NewHost fixes/replaces it, we take it back in rotation
2016-04-29 12009, 2016
ruaok
agreed/
2016-04-29 12017, 2016
ruaok
and we dont need to keep logs on them either.
2016-04-29 12028, 2016
ruaok
since log collection can and should happen on gateways.
2016-04-29 12051, 2016
zas
In fact, this can be an issue
2016-04-29 12053, 2016
ruaok
and I think we should ship logs to Google Cloud storage, since we have a budget for that.
2016-04-29 12013, 2016
ruaok
because of failover and such?
2016-04-29 12028, 2016
zas
And because of number of requests we may want to handle
2016-04-29 12036, 2016
ruaok
I think we need to investigate a log collection service.
2016-04-29 12045, 2016
ruaok
ship logs to a dedicated log collection server.
2016-04-29 12001, 2016
ruaok
and get them off the gateways asap.
2016-04-29 12016, 2016
zas
Currently 30 Mbps / 800 reqs -> if we x10 the bandwidth, it would be 8000 reqs
2016-04-29 12058, 2016
zas
So logging will be an issue in disk spaces and performance, this is something which is not that an issue for now
2016-04-29 12004, 2016
ruaok
I really want analysitics for all of our traffic.
2016-04-29 12006, 2016
ruaok
but that can come later.
2016-04-29 12011, 2016
ruaok
agreed.
2016-04-29 12016, 2016
D4RK-PH0ENiX joined the channel
2016-04-29 12050, 2016
ruaok
!m chrisskye
2016-04-29 12050, 2016
BrainzBot
You're doing good work, chrisskye!
2016-04-29 12004, 2016
KodeStar
ruaok / zas OVH vs Hetzner, what are the most important considerations for you?
under 70 euro a month you are going to be limited to an average of 100mbps with hetzner as they have a transfer limit of 30TB
2016-04-29 12018, 2016
ruaok
we're looking at 10-20 servers a month. :)
2016-04-29 12041, 2016
ruaok
so figuring out how to network them, RAID support, service support and such things are key for us.
2016-04-29 12003, 2016
ruaok
and given that our current limit is 30mbs, going to 100mbs is a great improvement.
2016-04-29 12034, 2016
ruaok
but I think our installation is going to be big enough and us being a non profit that we'll have to have a detailed conversation with however our new provider will be.
2016-04-29 12037, 2016
KodeStar
well if you have multiple servers ovh has a vrack, and if you go with the hosting range they have 10gbps vrack options, so all your internal traffic could go over that
We should make a HostingBrainz to be able to compare hosts...
2016-04-29 12054, 2016
KodeStar
the only problem with online.net is business support starts to get expensive, but as you are a non profit they may do something for you
2016-04-29 12000, 2016
Freso
ServerBrainz
2016-04-29 12001, 2016
ruaok
KodeStar: thanks, I've added it the doc.
2016-04-29 12019, 2016
KodeStar
how many ips do you need?
2016-04-29 12020, 2016
ruaok smacks Freso
2016-04-29 12025, 2016
ruaok
one
2016-04-29 12030, 2016
ruaok
actually no.
2016-04-29 12031, 2016
ruaok
three.
2016-04-29 12049, 2016
ruaok
depending on how bandwidth is charged, we may use more.
2016-04-29 12020, 2016
ruaok
if all of our traffic is routed via a gateway, but we get bandwidth on a per server basis, then we'll want our FTP to run over a different IP, thus a different bandwidth budget.
2016-04-29 12020, 2016
zas
We need to be able to add IPs on needs, but basically < 16 will be enough
2016-04-29 12026, 2016
KodeStar
then any will do, the only difference is ovh you pay a one off setup fee (around £3 iirc), online.net you pay a setup fee and monthly fee, I think it's the same with hetzner as well
2016-04-29 12030, 2016
ruaok
zas: good call.
2016-04-29 12049, 2016
ruaok
hetzner's setup is one month fee, which is pretty high.
2016-04-29 12014, 2016
KodeStar
ovh don't have bandwidth caps, nor online.net only hetzner, ovh guarantee 250mbps with burst up to 1gbps
2016-04-29 12040, 2016
ruaok
leaning more towards OVH by the minute. :)
2016-04-29 12048, 2016
zas
KodeStar: OVH also offer 500Mbps guarantee, depending on offers
2016-04-29 12000, 2016
KodeStar
zas, true, depends on which one you go to
2016-04-29 12008, 2016
ruaok
I don't think we have the demand to go past 250mbit.
"For example, Krafft said, descriptions of items in Cornell University Library’s Hip Hop Collection, such as annotated LP recordings from hip-hop’s earliest days, could now be linked to MusicBrainz.org"
2016-04-29 12055, 2016
reosarevok
Not bad I guess :)
2016-04-29 12033, 2016
ruaok
reosarevok: chrisskye already hit them up for money. prviate uni and all that. :)
2016-04-29 12040, 2016
reosarevok
haha
2016-04-29 12047, 2016
reosarevok
And $1.5 million sounds like a lot of money :p
2016-04-29 12002, 2016
reosarevok
Not sure why anyone would need that much to "use linked data"
