I wondered because the ratelimiter access is the only part where the search server directly deals with sockets.
2015-03-05 06412, 2015
chirlu`
The normal requests are handled by the framework.
2015-03-05 06417, 2015
ruaok
yeah, it was a good consideration
2015-03-05 06442, 2015
ruaok
is there a way from a java program to invoke the GC?
2015-03-05 06414, 2015
chirlu`
System.GC() apparently.
2015-03-05 06450, 2015
Mineo
the openjdk management console also has a button for that, if you're using that
2015-03-05 06450, 2015
chirlu`
Or System.gc(), rather.
2015-03-05 06453, 2015
ruaok
well, does anyone think that invoking it every 60 seconds is a bad idea?
2015-03-05 06410, 2015
chirlu`
Perhaps? :)
2015-03-05 06412, 2015
ruaok
Mineo: using sun's jdk atm
2015-03-05 06428, 2015
Mineo
ah
2015-03-05 06453, 2015
chirlu`
I’d like to know if running GC in a stuck state removes the CLOSE_WAIT sockets.
2015-03-05 06408, 2015
chirlu`
(Via destructors, or what they are called in Java.)
2015-03-05 06436, 2015
ruaok
ijabz2: can you please do me a favor?
2015-03-05 06439, 2015
ijabz2
chirlu thats a good point because I forgot how that code works and lookong at it Im not closing the Datagram socket
2015-03-05 06447, 2015
ruaok
can you create a new URL endpoint that runs the GC?
2015-03-05 06407, 2015
ruaok
/run-gc-now-you-piece-of-shit
2015-03-05 06410, 2015
ruaok
:)
2015-03-05 06422, 2015
ruaok
then we can hit that URL whenever we want the GC to run
2015-03-05 06428, 2015
ruaok
then we can see clearly what happens.
2015-03-05 06430, 2015
chirlu`
ijabz2: Yes, I saw that, but if it connects to a different IP address than what we see, it can’t be that.
2015-03-05 06438, 2015
ijabz2
ok, I;ll do that
2015-03-05 06449, 2015
ruaok
thanks
2015-03-05 06455, 2015
chirlu`
What we see in the netstat output, I mean.
2015-03-05 06456, 2015
ruaok
CPU usage is climing steadily.
2015-03-05 06412, 2015
ruaok
roobarb is fine (restarted recently)
2015-03-05 06454, 2015
anjali2906 joined the channel
2015-03-05 06432, 2015
anjali2906 has left the channel
2015-03-05 06449, 2015
ijabz2
Gc may release memory used by lucene for the memory mapping of indexes but not sure if will as they are not on the heap
2015-03-05 06426, 2015
ijabz2
either way even if it does will just slow searches a little rather breaking anyhting so woth a try
2015-03-05 06410, 2015
ruaok nods
2015-03-05 06421, 2015
ruaok
it sure seems like the GC isn't doing its job
2015-03-05 06431, 2015
ruaok
this may not help if the GC gets stuck.
2015-03-05 06436, 2015
ruaok
but if the GC doesn't fire, it might
2015-03-05 06458, 2015
ruaok
though running the GC on a regular basis may it getting stuck in the first place
2015-03-05 06421, 2015
s0014 joined the channel
2015-03-05 06416, 2015
nikki
Gentlecat: hm, the metabrainz sign up thing asks for an email address, couldn't it actually request that info from the mb account when doing oauth?
2015-03-05 06404, 2015
Gentlecat
yeah, I can add that
2015-03-05 06431, 2015
ruaok
I'd say auto-fill the form, but allow users to change it.
2015-03-05 06442, 2015
nikki
(I imagine quite a few people wanting to sign up won't actually be active editors, so they'll have to create an mb account, then sign up "again" (from their point of view))
2015-03-05 06403, 2015
ijabz2
ruaok ok ive done it, invoked using paramter gc=true
2015-03-05 06405, 2015
nikki
yeah, I was thinking that
2015-03-05 06426, 2015
ruaok
on any request or a specifc endpoint, ijabz2?
2015-03-05 06450, 2015
MBJenkins
Paul Taylor: Close datagram socket after contacting rate limiter
only from local machine, like we do frpo reloading indexes etc, is that ok
2015-03-05 06424, 2015
ruaok
perfect
2015-03-05 06431, 2015
ruaok
compiling now
2015-03-05 06412, 2015
s0014
\nick siddharth
2015-03-05 06433, 2015
chirlu`
s0014: Forward slash. :)
2015-03-05 06405, 2015
s0014
Yeah, by mistake. Sorry to spam the list.
2015-03-05 06437, 2015
MBJenkins
Paul Taylor: Added gc option to force garbage collection
2015-03-05 06419, 2015
ruaok
heh. ijabz2 my build succeeded. how should I proceed?
2015-03-05 06434, 2015
ijabz2
What do yo mean, same as usual I guess
2015-03-05 06448, 2015
ruaok
your build failed.
2015-03-05 06403, 2015
ijabz2
oh, I see
2015-03-05 06447, 2015
ijabz2
erm, I cant see any errors details on Jenkins ?
2015-03-05 06408, 2015
ijabz2
ruaok I thought there was a way to force Jenkins but cant see it, instead done a tiny bit of refactoring and done a new checkin
2015-03-05 06437, 2015
ruaok
doing a new pull
2015-03-05 06437, 2015
s0014
Hey, I am Siddharth Gupta, a final year BTech student. In GSOC'14, I did a Data Visualization project for CiviCRM. I was going through the Idea page, documentation as we as How to contribute. After reading some of the code, I am starting on setting up the server. I would like to know is there anything you are expecting us to do before we start talking about the project.
invoking gc seems to have no effect on memory usage.
2015-03-05 06410, 2015
ruaok
which is what I would roughly expect.
2015-03-05 06422, 2015
chirlu`
But we don’t have any CLOSE_WAIT sockets now, do we?
2015-03-05 06434, 2015
ruaok
correct, we have none
2015-03-05 06443, 2015
ruaok
I may need to wait a few hours for that to occur.
2015-03-05 06450, 2015
ruaok
because a watched server never crashes
2015-03-05 06438, 2015
kepstin-laptop joined the channel
2015-03-05 06413, 2015
ijabz2
ruaok, classic bad timing Ive just had some feedback on the other bug (TreeMap Null) causing me to commit another change which will get us the neccessary debeugging if/when it happens again
2015-03-05 06438, 2015
ijabz2
Wil need another build and deploy, but Ill leave it you to decide if yo want to do it now or some other time
Right, we didnt use to log these we just returned the error to user
2015-03-05 06443, 2015
ijabz2
Does the line before the satck trace actually show the query ?
2015-03-05 06454, 2015
Leo_Verto
hmm, looks like an empty request to me
2015-03-05 06405, 2015
ruaok
05-Mar-2015 16:04:38.311 WARNING [http-nio-8080-exec-16] org.musicbrainz.search.servlet.SearchServerServlet.doGet 9044705516.7897105009 AND artist:sri sai telicom maupakar/prakhar yamraaj:Cannot parse '9044705516.7897105009 AND artist:sri sai telicom maupakar/prakhar yamraaj': Lexical error at line 1, column 74. Encountered: <EOF> after : "/prakhar yamraaj"
Paul Taylor: Log query whatever error occurs instead of just if get a parseexception in case it is the query causing the error (which I think it is in this case of SEARCH-411)
2015-03-05 06409, 2015
Siddharth__ joined the channel
2015-03-05 06412, 2015
UmkaDK joined the channel
2015-03-05 06411, 2015
ijabz2
Leo_Verto yes it doesnt like the slash does it
2015-03-05 06450, 2015
UmkaDK_ joined the channel
2015-03-05 06403, 2015
ruaok
just a bad client then?
2015-03-05 06411, 2015
ijabz2
is that a http restriction not a lucene one
2015-03-05 06420, 2015
ruaok
no clue
2015-03-05 06444, 2015
chirlu`
I think /.../ is a regex to Lucene.
2015-03-05 06435, 2015
ijabz2
ah yeah I was looking at lucene 3, thanks chirlu
2015-03-05 06409, 2015
ruaok
maybe we don't want to log the queryparter errors.
ruaok can you get the preious line that shows the query like you did before please
2015-03-05 06428, 2015
ruaok
ops sorry
2015-03-05 06436, 2015
JesseW joined the channel
2015-03-05 06452, 2015
ruaok
05-Mar-2015 16:50:40.726 WARNING [http-nio-8080-exec-41] org.musicbrainz.search.servlet.SearchServerServlet.doGet artist:pandora /reyli barba recording:Solo el y yo/Alguien llena mi lugar:null
<s0014> Hey, I am Siddharth Gupta, a final year BTech student. In GSOC'14, I did a Data Visualization project for CiviCRM. I was going through the Idea page, documentation as we as How to contribute. After reading some of the code, I am starting on setting up the server. I would like to know is there anything you are expecting us to do before we start talking about the project.
2015-03-05 06413, 2015
derwin
hi s0014 :)
2015-03-05 06447, 2015
s0014
Hi derwin, I have been looking at the issue tracker, and would now to understand what all I should do before discussing the ideas I have for the project.
