(And thanks to jesus2099’ mega userscript, it’s very easy to switch between the sandbox and the main server.)
2015-12-23 35714, 2015
zas
ruaok's changes on the search server code did miracles, they didnt crash yet after 4 restarts, indexer is running normally, let's see how it does when load will increase
2015-12-23 35747, 2015
Freso
zas: Did he do more changes? Or is this those changes from 1-2 week(s) ago?
BB: Add /creator/ tests || 3 days || Code, QA || MetaBrainz Foundation || Currently claimed || Instances: 2/2
2015-12-23 35713, 2015
reosarevok
Heh, 22 minutes
2015-12-23 35715, 2015
reosarevok
Ok, 2 days now
2015-12-23 35725, 2015
stanislas
reosarevok: I know, thanks :)
2015-12-23 35744, 2015
stanislas
reosarevok: I've been doing /creator/id for most of the time, when i've this task claimed
2015-12-23 35720, 2015
reosarevok is happy that you're doing stuff, whatever it is :) We can use every person!
2015-12-23 35748, 2015
stanislas
stanislas is happy that he can contribute and learn a lot :)
2015-12-23 35749, 2015
Freso
stanislas: "/me is happy that he can contribute and learn a lot :)" ;)
2015-12-23 35714, 2015
Freso
stanislas: Also, remember to always link your task instance when you're requesting stuff like that. Even if no mentor is currently around, it will be much easier for us to act on it when we see it (esp. in case, you've gone off to eat or sleep or some other silly RL stuff like that ;)).
zas, Can you copy the logs to the VM you set up so I can look at them>?
2015-12-23 35744, 2015
zas
logs don't have any useful infos
2015-12-23 35754, 2015
zas
but we can get stack traces
2015-12-23 35702, 2015
zas
i'll do it on next issue
2015-12-23 35720, 2015
zas
i have older ones, but since code changed last night, better work on newer ones
2015-12-23 35728, 2015
zas
the main issue is that both servers eat 1500% cpu, and there is no way to simply restart them to clear the issue, we also need to reduce drastically the number of queries they receive, you can check the behavior on http://stats.musicbrainz.org/dashboard/db/search-…
2015-12-23 35734, 2015
zas
anonymous/anonymous
2015-12-23 35758, 2015
zas
this time: indexer pushed new indexes, restarted dora search server, it went to 1500% and never went down, 10 minutes later indexer pushed to roobarb, restarted it, and it went also to 1500%, both servers dead
2015-12-23 35711, 2015
zas
note it doesn't happen all the times
2015-12-23 35745, 2015
zas
6 index push / restart went well
2015-12-23 35714, 2015
zas
it is related to load, it is more likely to happen on peak hours
2015-12-23 35705, 2015
CJ_
zas, Ooo. I like stack traces.
2015-12-23 35725, 2015
zas
i'll provide you one on next failure
2015-12-23 35737, 2015
CJ_
Great!
2015-12-23 35723, 2015
zas
we currently use lucene-4.10.4 branch
2015-12-23 35739, 2015
zas
this is important, since it contains updated lucene and icu4j, which solved a part of the issues (ie. a huge memleak with older lucene)
2015-12-23 35755, 2015
CJ_
Understood.
2015-12-23 35702, 2015
zas
indexer runs in a loop, during the day it pushes indexes every hour atm, but when building recordings (once per day) or freedb indexes (once per month)
2015-12-23 35712, 2015
CJ_
I have not made it that far into things. I am still working on setting up the server to poke around.
2015-12-23 35741, 2015
CJ_
Why does the indexer loop instead of pushing updates as they happen?
2015-12-23 35726, 2015
zas
what do you mean ? there was an attempt to index using replicating packets, but it was never finished, so it re-indexes all entities over and over
2015-12-23 35746, 2015
zas
ruaok could tell you much more about that i think
2015-12-23 35750, 2015
CJ_
That seems a waste. You should only need to index what changes.
2015-12-23 35711, 2015
zas
yes, i agree, but for some reason it was never made
2015-12-23 35734, 2015
CJ_
Understood.
2015-12-23 35744, 2015
CJ_
I will see what I can come up with once I get into the code.
2015-12-23 35744, 2015
gcilou joined the channel
2015-12-23 35752, 2015
zas
ok
2015-12-23 35759, 2015
weeksio
CJ_ and zas: this is main objective with the solr project
2015-12-23 35720, 2015
zas
weeksio: yes, i was told that
2015-12-23 35730, 2015
CJ_
weeksio, Once I understand the existing setup I will start helping with the solr project.
2015-12-23 35741, 2015
Leo_Verto has quit
2015-12-23 35710, 2015
weeksio
I've been in a bit over my head since I started it
2015-12-23 35733, 2015
CJ_
weeksio, How so?
2015-12-23 35705, 2015
weeksio
in every way....not much real world learnin in CS school
2015-12-23 35721, 2015
Mineo has quit
2015-12-23 35723, 2015
CJ_
Understood. Are you still a student?
2015-12-23 35735, 2015
weeksio
yes
2015-12-23 35705, 2015
gcilou
Submitted BB: Redesign landing page for review
2015-12-23 35705, 2015
CJ_
Well, if you have specific questions I will be more than happy to answer them as well as I can.
2015-12-23 35727, 2015
CJ_
But I am still learning the overall setup, hence the starting with existing search before working on the solr project.
