0:13 AM
kori has quit
2019-08-02 21402, 2019
1:21 AM
ayerhart joined the channel
2019-08-02 21459, 2019
1:26 AM
ayerhart has quit
2019-08-02 21414, 2019
1:31 AM
d4rkie joined the channel
2019-08-02 21420, 2019
1:33 AM
D4RK-PH0ENiX has quit
2019-08-02 21435, 2019
1:39 AM
ayerhart joined the channel
2019-08-02 21417, 2019
1:47 AM
kori joined the channel
2019-08-02 21417, 2019
1:47 AM
kori has quit
2019-08-02 21417, 2019
1:47 AM
kori joined the channel
2019-08-02 21402, 2019
1:50 AM
ayerhart has quit
2019-08-02 21401, 2019
2:45 AM
d4rkie has quit
2019-08-02 21440, 2019
2:45 AM
D4RK-PH0ENiX joined the channel
2019-08-02 21439, 2019
4:55 AM
D4RK-PH0ENiX has quit
2019-08-02 21409, 2019
5:12 AM
D4RK-PH0ENiX joined the channel
2019-08-02 21443, 2019
5:25 AM
pristine__
ruaok: hey. Can you review #40, #41 and #42?
2019-08-02 21421, 2019
5:33 AM
iliekcomputers
alastairp: dump is done, if you add me to bono, i'll copy it over.
2019-08-02 21437, 2019
5:33 AM
iliekcomputers
2019-08-02 21434, 2019
7:01 AM
d4rkie joined the channel
2019-08-02 21447, 2019
7:04 AM
D4RK-PH0ENiX has quit
2019-08-02 21432, 2019
7:33 AM
d4rkie has quit
2019-08-02 21416, 2019
7:34 AM
D4RK-PH0ENiX joined the channel
2019-08-02 21430, 2019
7:34 AM
reosarevok
2019-08-02 21431, 2019
7:34 AM
BrainzBot
MBS-10258: CD stub search fails with error on prod & beta
2019-08-02 21405, 2019
7:35 AM
reosarevok
(I copied the config lines from serge)
2019-08-02 21433, 2019
8:14 AM
antlarr2 is now known as antlarr
2019-08-02 21420, 2019
8:19 AM
Cyna
reosarevok: what is `comma_only_list`
2019-08-02 21437, 2019
8:19 AM
Cyna
I dont see a macro for it
2019-08-02 21420, 2019
8:20 AM
Cyna
nor a utility
2019-08-02 21404, 2019
8:23 AM
yvanzo
reosarevok: The new server is not publicly accessible on purpose.
2019-08-02 21439, 2019
8:24 AM
yvanzo
reosarevok: However, you might be able to access it from your local development server through ssh port forwarding.
2019-08-02 21404, 2019
8:25 AM
reosarevok
I know, thought there would be some key/token I can use
2019-08-02 21444, 2019
8:25 AM
reosarevok
Cyna: it's already in JS as CommaOnlyList or whatnot, search for uses
2019-08-02 21451, 2019
8:26 AM
Cyna
I'll have a look, thanks :)
2019-08-02 21401, 2019
8:27 AM
yvanzo
git grep 'sub *comma_only_list'
2019-08-02 21435, 2019
8:27 AM
Cyna
I saw the `sub comma only list`
2019-08-02 21440, 2019
8:27 AM
Cyna
What do they mean ?
2019-08-02 21456, 2019
8:27 AM
yvanzo
it is a definition for a subroutine in Perl
2019-08-02 21414, 2019
8:28 AM
Cyna
Any idea how do I get it in React
2019-08-02 21432, 2019
8:28 AM
Cyna
reosarevok: Couldnt find CommaOnlyList using git grep
2019-08-02 21436, 2019
8:28 AM
yvanzo
it is defined in module Translation, and exported for use in templates in View::Default
2019-08-02 21419, 2019
8:30 AM
Cyna
didnt get it 😅
2019-08-02 21433, 2019
8:30 AM
yvanzo
(that was about comma_only_list)
2019-08-02 21459, 2019
8:30 AM
Cyna
ohh yea
2019-08-02 21405, 2019
8:31 AM
Cyna
I see that in the grep
2019-08-02 21420, 2019
8:31 AM
yvanzo
Cyna: there should be many occurrences of CommaOnlyList using git grep
2019-08-02 21423, 2019
8:31 AM
reosarevok
Maybe it's a function so lowercase comma
2019-08-02 21427, 2019
8:31 AM
reosarevok
I don't remember
2019-08-02 21429, 2019
8:31 AM
iliekcomputers
ruaok: I like how airports are one of the few places that don't judge you if you get drunk at 10:30 in the morning :)
2019-08-02 21444, 2019
8:31 AM
Cyna
ohh sorry
2019-08-02 21448, 2019
8:31 AM
Cyna
Its a lower case comma
2019-08-02 21453, 2019
8:32 AM
yvanzo
Cyna: git grep 'const *CommaOnlyList'
2019-08-02 21436, 2019
8:33 AM
reosarevok
yvanzo: so there's no key or way to use the solr server, is it hardcoded on the solr side?
2019-08-02 21438, 2019
8:33 AM
yvanzo
Oops, git grep -i 'const *CommaOnlyList'
2019-08-02 21431, 2019
8:34 AM
yvanzo
reosarevok: there is just no port open to the outside world from the solr server.
2019-08-02 21450, 2019
8:34 AM
yvanzo
(if I understood it correctly)
2019-08-02 21452, 2019
8:36 AM
Cyna
```[%- FOR attribute=type.value %]
2019-08-02 21452, 2019
8:36 AM
Cyna
<li>[% attribute.l_value %]</li>
2019-08-02 21452, 2019
8:36 AM
Cyna
[%- END %]```
2019-08-02 21400, 2019
8:37 AM
reosarevok
Oh no
2019-08-02 21413, 2019
8:37 AM
Cyna
Can this be ported to type.value.l_value ?
2019-08-02 21400, 2019
8:38 AM
yvanzo
Cyna: what reosarevok just answered above ;)
2019-08-02 21415, 2019
8:38 AM
reosarevok
Cyna: l_attributes(value) is probably what's needed there, but maybe it's slightly different
2019-08-02 21442, 2019
8:38 AM
reosarevok
We don't usually pass l_stuff to JS (l_ there is a translated value)
2019-08-02 21456, 2019
8:38 AM
reosarevok
But yvanzo has touched attributes more than I have, so he might have something else to say
2019-08-02 21403, 2019
8:39 AM
reosarevok
yvanzo: hmm, I see
2019-08-02 21426, 2019
8:39 AM
reosarevok
But test.mb is on the same place and could be changed to access solr, right?
2019-08-02 21433, 2019
8:39 AM
reosarevok
So I could test the code there?
2019-08-02 21440, 2019
8:39 AM
yvanzo
reosarevok: that might be both/either a security issue and/or networking performance issue.
2019-08-02 21445, 2019
8:39 AM
reosarevok
(I know it currently also uses lucene)
2019-08-02 21412, 2019
8:41 AM
yvanzo
reosarevok: probably but it would then access search indexes unrelated to its database, though that would probably be ok for your tests.
2019-08-02 21439, 2019
8:41 AM
reosarevok
Oh, fair
2019-08-02 21450, 2019
8:41 AM
reosarevok
So change its dbdefs, test, change back
2019-08-02 21400, 2019
8:43 AM
yvanzo
yup, you’ll probably have to restart the container each time after changing DBDefs
2019-08-02 21427, 2019
8:43 AM
yvanzo
(or not)
2019-08-02 21443, 2019
8:49 AM
ruaok
iliekcomputers: where are you? On CEST?
2019-08-02 21451, 2019
8:58 AM
nav2002_ joined the channel
2019-08-02 21401, 2019
9:05 AM
Gazooo has quit
2019-08-02 21453, 2019
9:06 AM
Gazooo joined the channel
2019-08-02 21405, 2019
9:19 AM
ruaok
pristine__: reviewing it on top of my list of things to do.
2019-08-02 21426, 2019
9:19 AM
ruaok
I escaped the time consuming and fattterning family interactions in the UK.
2019-08-02 21453, 2019
9:19 AM
ruaok
now on to a chosen family gathering in the netherlands.
2019-08-02 21422, 2019
9:20 AM
ruaok
yvanzo: ping me when you're around, I'd like to hear what you need python help with.
2019-08-02 21415, 2019
9:23 AM
zas
moooin
2019-08-02 21414, 2019
9:28 AM
alastairp
hello
2019-08-02 21421, 2019
9:31 AM
yvanzo
ruaok: I may just have fixed the bug, thanks to the hints alastairp gave me yesterday.
2019-08-02 21427, 2019
9:31 AM
alastairp
woo!
2019-08-02 21443, 2019
9:31 AM
yvanzo
At least, last reindex didn’t log any error.
2019-08-02 21457, 2019
9:31 AM
alastairp
there's still the open question of why there are entities in the database with control characters in their name
2019-08-02 21405, 2019
9:33 AM
yvanzo
Oops, sorry, I did not :(
2019-08-02 21433, 2019
9:34 AM
yvanzo
Despite removing control chars in query_result_to_dict, it still throws the same error.
2019-08-02 21415, 2019
9:35 AM
yvanzo
alastairp: I could not confirm there is such entity.
2019-08-02 21455, 2019
9:35 AM
reosarevok
bitmap, yvanzo: I'm at a festival until Sunday, away from my laptop until Monday
2019-08-02 21408, 2019
9:36 AM
reosarevok
Will be online for questions / chat if needed, but no access to my code :
2019-08-02 21409, 2019
9:36 AM
reosarevok
* :)
2019-08-02 21414, 2019
9:37 AM
yvanzo
ruaok: before anything else, I probably need a way to better trace the error being thrown.
2019-08-02 21403, 2019
9:38 AM
ruaok
pristine__: #40 reviewed, we should chat a bit.
2019-08-02 21416, 2019
9:38 AM
alastairp
yvanzo: do you know the entity?
2019-08-02 21434, 2019
9:40 AM
ruaok
yvanzo: how can I help?
2019-08-02 21431, 2019
9:42 AM
yvanzo
alastairp: I have not been able to reproduce the error with a limited dataset so far.
2019-08-02 21443, 2019
9:43 AM
ruaok
would a bigger VM help?
2019-08-02 21414, 2019
9:49 AM
yvanzo
not for now, just trying to isolate the bug first.
2019-08-02 21447, 2019
9:53 AM
ruaok
can you give me some background on it? what happens, what dataset? links to code involved?
2019-08-02 21425, 2019
9:55 AM
yvanzo
2019-08-02 21404, 2019
9:56 AM
yvanzo
it only occurs while reindexing annotation, recording, release, and url
2019-08-02 21440, 2019
9:56 AM
alastairp
it'd be a bit of work, but my recommendation is to implement an import method that doesn't use multiprocessing, so that you can get the full stacktrace
2019-08-02 21440, 2019
9:58 AM
yvanzo
I tried to bisect by deleting some database entries that were correctly reindexed => the error was not thrown anymore
2019-08-02 21440, 2019
9:58 AM
ruaok
yvanzo: have you added a detailed print statement that prints out the contents of the offending XML in sir/indexing.py ?
2019-08-02 21407, 2019
9:59 AM
ruaok
and then running it on whatever data set that exhibits the problem?
2019-08-02 21408, 2019
9:59 AM
alastairp
ruaok: my understanding of the error message is that it's thrown by lxml, which means it's while it's trying to construct the xml
2019-08-02 21444, 2019
9:59 AM
alastairp
so yeah, the same but actually print the objects/ids that it's trying to serialise
2019-08-02 21454, 2019
9:59 AM
ruaok
yep, that.
2019-08-02 21412, 2019
10:00 AM
ruaok
might be ton of output, but redirect to a file and look a that tail after failure, no?
2019-08-02 21431, 2019
10:00 AM
ruaok
it sure seems that there is an encoding like issue that isn't being handled by the serializer.
2019-08-02 21408, 2019
10:01 AM
alastairp
which from my understanding is weird, because the solr library that's being used strips control characters from values before putting them into the xml
2019-08-02 21417, 2019
10:01 AM
ruaok
the curious thing is, why does it happen only on reindex and not on the original data set?
2019-08-02 21441, 2019
10:01 AM
ruaok
or is it that new data comes in that is faulty and it follows a different code path?
2019-08-02 21453, 2019
10:01 AM
alastairp
2019-08-02 21400, 2019
10:02 AM
alastairp
depending on the value of `live`
2019-08-02 21449, 2019
10:02 AM
alastairp
but that only seems to affect the WHERE clause
2019-08-02 21454, 2019
10:02 AM
yvanzo
I added some details to logs to try to find out which database entries where causing this, but it did not help so far
2019-08-02 21406, 2019
10:03 AM
yvanzo
alastairp: 'live' is 'false' during reindex
2019-08-02 21414, 2019
10:03 AM
alastairp
right
2019-08-02 21438, 2019
10:05 AM
alastairp
how difficult is sir/solr to set up?
2019-08-02 21459, 2019
10:06 AM
ruaok
is there some processing that happens to the data that comes in when "live" but does happen when the data is being fetched from the DB?
2019-08-02 21432, 2019
10:07 AM
yvanzo
2019-08-02 21451, 2019
10:07 AM
yvanzo
but it takes some resources
2019-08-02 21445, 2019
10:08 AM
yvanzo
ruaok: I don't know if this error would happen during live reindex because these entries may just not have been through live reindex yet.
2019-08-02 21404, 2019
10:09 AM
ruaok
what resources do you need?
2019-08-02 21459, 2019
10:09 AM
yvanzo
that was for alastairp if he wanted to deploy it
2019-08-02 21410, 2019
10:10 AM
alastairp
yvanzo: indexer doesn't build..
2019-08-02 21419, 2019
10:10 AM
alastairp
indexer: build: indexer-dockerfile
2019-08-02 21432, 2019
10:10 AM
alastairp
but no such folder in the checkout, just sir-dockerfile and solr-dockerfile
2019-08-02 21448, 2019
10:10 AM
Gazooo has quit
2019-08-02 21449, 2019
10:10 AM
alastairp
but the build tries to use airdock/oracle-jdk:jdk-8u74, says that it doesn't exist
2019-08-02 21456, 2019
10:10 AM
yvanzo
ruaok: is it possible to give alastairp access to the current test vm?
2019-08-02 21409, 2019
10:11 AM
ruaok
yes.
2019-08-02 21410, 2019
10:11 AM
Gazooo joined the channel
2019-08-02 21415, 2019
10:11 AM
ruaok
hang on.
2019-08-02 21423, 2019
10:11 AM
ruaok
which key alastairp? one on github?
2019-08-02 21418, 2019
10:12 AM
alastairp
2019-08-02 21446, 2019
10:12 AM
yvanzo
alastairp: what is commit HEAD id on your musicbrainz-docker clone?
2019-08-02 21406, 2019
10:13 AM
alastairp
yvanzo: oh, sorry. of course
2019-08-02 21409, 2019
10:13 AM
alastairp
that'd be master, not your branch
2019-08-02 21405, 2019
10:14 AM
ruaok
alastairp: ssh alastair@104.197.183.152
2019-08-02 21439, 2019
10:14 AM
alastairp
yeah, I'm in
2019-08-02 21442, 2019
10:14 AM
ruaok
now with sudo love
2019-08-02 21401, 2019
10:15 AM
yvanzo
sudo su - yvanzo
2019-08-02 21426, 2019
10:15 AM
yvanzo
see *annotations*log
2019-08-02 21432, 2019
10:18 AM
alastairp
yeah, there's not a lot of debugging there
2019-08-02 21445, 2019
10:18 AM
alastairp
how do you run the indexer?
