-
darwin
yea we run kafka at work
2015-09-13 25627, 2015
-
ruaok
2015-09-13 25633, 2015
-
darwin
I don't have a huge amount of hands on, unfortunately
2015-09-13 25641, 2015
-
ruaok
some of the answer below might be interesting to try too.
2015-09-13 25651, 2015
-
ruaok
darwin: bummer.
2015-09-13 25657, 2015
-
ruaok
only one VM experiences this.
2015-09-13 25606, 2015
-
Gentlecat
no, I haven't tried removing directories
2015-09-13 25623, 2015
-
ruaok
Gentlecat: try it. and then maybe try the lesser answers below.
2015-09-13 25610, 2015
-
Gentlecat
looks like our data dir for kafka is /var/lib/kafka/kafka-logs
2015-09-13 25636, 2015
-
Gentlecat
not sure about zookeeper though
2015-09-13 25646, 2015
-
Gentlecat
it's not /tmp/zookeeper
2015-09-13 25639, 2015
-
ruaok
/var/lib/zookeeper ?
2015-09-13 25608, 2015
-
Gentlecat
yeah, this one has things
2015-09-13 25642, 2015
-
Gentlecat
Created topic "listens".
2015-09-13 25645, 2015
-
Gentlecat
let's try again
2015-09-13 25612, 2015
-
Gentlecat
"success"
2015-09-13 25615, 2015
-
Gentlecat
yay?
2015-09-13 25627, 2015
-
ruaok
odd.
2015-09-13 25635, 2015
-
ruaok
at least we have a workaround for this case.
2015-09-13 25649, 2015
-
Gentlecat
still nothing in my profile though
2015-09-13 25605, 2015
-
ruaok
is the writer script working?
2015-09-13 25607, 2015
-
ruaok
running...
2015-09-13 25613, 2015
-
Gentlecat
which one is that?
2015-09-13 25640, 2015
-
Gentlecat
listenstore-kafka-to-cassandra-writer.py?
2015-09-13 25651, 2015
-
ruaok
yes
2015-09-13 25652, 2015
-
Gentlecat
no, it's not
2015-09-13 25659, 2015
-
ruaok
did you run python setup.py develop in the listenstore dir?
2015-09-13 25613, 2015
-
ruaok
cassandra_consumer.py in the root is the old one.
2015-09-13 25623, 2015
-
Gentlecat
I think I did
2015-09-13 25637, 2015
-
ruaok
does the script run?
2015-09-13 25644, 2015
-
Gentlecat
running listenstore-kafka-to-cassandra-writer.py solved the issue with missing listens on my profile page
2015-09-13 25650, 2015
-
Gentlecat
I guess it's all working now
2015-09-13 25602, 2015
-
ruaok
\ø/
2015-09-13 25604, 2015
-
Gentlecat
time to update README.md
2015-09-13 25607, 2015
-
Gentlecat
lel
2015-09-13 25622, 2015
-
ruaok
phew.
2015-09-13 25658, 2015
-
Gentlecat
so, uh, how do we fix this?
2015-09-13 25621, 2015
-
ruaok
fix what?
2015-09-13 25628, 2015
-
ruaok
it happens only for you.
2015-09-13 25637, 2015
-
ruaok
I wonder if it has to do that you're not on a mac.
2015-09-13 25645, 2015
-
ruaok
make a note in README and let's move on
2015-09-13 25646, 2015
-
Gentlecat
ok
2015-09-13 25638, 2015
-
Gentlecat
is JIRA broken? I'm getting 502s
2015-09-13 25615, 2015
-
ruaok
rebooting the server, which hosts mail as well.
2015-09-13 25619, 2015
-
ruaok
should be back in a sec.
2015-09-13 25621, 2015
-
alastairp
oh cool. good to see you got it going
2015-09-13 25633, 2015
-
ruaok
Gentlecat: ERROR: 58P01: could not open extension control file "/usr/share/postgresql/9.3/extension/uuid-ossp.control": No such file or directory
2015-09-13 25645, 2015
-
Gentlecat
9.3?
2015-09-13 25651, 2015
-
ruaok
yes.
2015-09-13 25653, 2015
-
ruaok
do I need 9.4?
2015-09-13 25604, 2015
-
Gentlecat
I think we run 9.4
2015-09-13 25613, 2015
-
ruaok
k
2015-09-13 25614, 2015
-
Gentlecat
at least that's what I've been using all this time
2015-09-13 25631, 2015
-
Gentlecat
I'm putting that into readme
2015-09-13 25633, 2015
-
ruaok
I'll uprade
2015-09-13 25657, 2015
-
alastairp
yes, we’re using 9.4, it’s not available in the version of ubuntu we’re using
2015-09-13 25612, 2015
-
Gentlecat
huh?
2015-09-13 25632, 2015
-
Gentlecat
there's postgreql-9.4 package
2015-09-13 25640, 2015
-
ruaok
I coudln't find 9.4 earlier.
2015-09-13 25651, 2015
-
alastairp
what does apt-cache policy say?
2015-09-13 25635, 2015
-
LordSputnik has quit
2015-09-13 25659, 2015
-
Gentlecat
2015-09-13 25642, 2015
-
alastairp
apt-cache policy postgresql-9.4
2015-09-13 25642, 2015
-
alastairp
sorry
2015-09-13 25655, 2015
-
ruaok
I see it now.
2015-09-13 25600, 2015
-
ruaok
but I did an apt-get update
2015-09-13 25639, 2015
-
Gentlecat
2015-09-13 25654, 2015
-
alastairp
2015-09-13 25605, 2015
-
alastairp
ruaok: fyi, I got the leader error as well
2015-09-13 25619, 2015
-
ruaok
persistent or only once?
2015-09-13 25622, 2015
-
ruaok
everyone gets it once.
2015-09-13 25604, 2015
-
alastairp
I poked it a few times, did the delete thing, used the kafka producer cli to put data in it
2015-09-13 25607, 2015
-
alastairp
and it works now
2015-09-13 25613, 2015
-
alastairp
I’m not sure which of the things fixed it
2015-09-13 25630, 2015
-
alastairp
yeah, i think i understand how it gets it once, if the topic doesn’t exist
2015-09-13 25632, 2015
-
ruaok
delete thing worked for Gentlecat
2015-09-13 25653, 2015
-
alastairp
I wonder if we could disable automatic topic creation, and run the create topic command during setup
2015-09-13 25604, 2015
-
alastairp
perhaps that would stop the error from happening
2015-09-13 25605, 2015
-
Gentlecat
2015-09-13 25606, 2015
-
ruaok
might be more robust, yes
2015-09-13 25620, 2015
-
ruaok
apparently it is a timeout issue since creating a new topic takes a bit.
2015-09-13 25657, 2015
-
alastairp
ahhh
2015-09-13 25625, 2015
-
Gentlecat
so it keeps trying to create it again and again with every request that comes in?
2015-09-13 25644, 2015
-
ruaok
usually the second request succeeds, just not in your case.
2015-09-13 25608, 2015
-
ruaok
the easiest fix is to use the kafka-topics.sh script to create the topic upon installation.
2015-09-13 25621, 2015
-
alastairp
cool, that’s a good piece of knowledge to have
2015-09-13 25637, 2015
-
ruaok
ok, 9.4 succeeds in installing that extension but fails on something else
2015-09-13 25652, 2015
-
alastairp
ruaok: do we have a plan of attack for this week? is it written down somewhere?
2015-09-13 25653, 2015
-
JesseW has left the channel
2015-09-13 25659, 2015
-
ruaok
2015-09-13 25620, 2015
-
Gentlecat
sqlite3?
2015-09-13 25624, 2015
-
alastairp
why is it using sqlite?
2015-09-13 25626, 2015
-
alastairp
yeah, that
2015-09-13 25635, 2015
-
ruaok
I have a plan of attack, but that is for another important project that I'm working on this week.
2015-09-13 25636, 2015
-
Gentlecat
is your connection string defined?
2015-09-13 25638, 2015
-
ruaok
none for LB.
2015-09-13 25650, 2015
-
Gentlecat
we switched to sqlalchemy, so you need to update config file
2015-09-13 25659, 2015
-
Gentlecat
don't know if you've done that already
2015-09-13 25659, 2015
-
ruaok
PG_CONNECT = "dbname=listenbrainz user=listenbrainz password= host=localhost port=5432"
2015-09-13 25606, 2015
-
LordSputnik joined the channel
2015-09-13 25610, 2015
-
Gentlecat
no, that's the old one for psycopg2
2015-09-13 25621, 2015
-
ruaok
ok
2015-09-13 25626, 2015
-
Gentlecat
see sample config
2015-09-13 25630, 2015
-
Gentlecat
or just copy SQLALCHEMY_DATABASE_URI = "postgresql://listenbrainz@/listenbrainz"
2015-09-13 25619, 2015
-
ruaok
thanks.
2015-09-13 25624, 2015
-
ruaok
working, server started.
2015-09-13 25612, 2015
-
ariscop has quit
2015-09-13 25617, 2015
-
LordSputnik has quit
2015-09-13 25648, 2015
-
Gentlecat
2015-09-13 25609, 2015
-
Gentlecat
there's a special space for custom stuff
2015-09-13 25618, 2015
-
LordSputnik joined the channel
2015-09-13 25629, 2015
-
ruaok
no
2015-09-13 25653, 2015
-
Gentlecat
"add this line at the beginning" :)
2015-09-13 25604, 2015
-
ruaok
oh, details. :)
2015-09-13 25606, 2015
-
Gentlecat
I think we had this issue with alastairp before
2015-09-13 25610, 2015
-
ruaok
ok, +1 then.
2015-09-13 25613, 2015
-
Gentlecat
yup
2015-09-13 25621, 2015
-
alastairp
“I think I had this problem before and alastairp fixed it for me"
2015-09-13 25651, 2015
-
ruaok
alastairp, our problem child.
2015-09-13 25607, 2015
-
Gentlecat
right
2015-09-13 25617, 2015
-
alastairp
you talk about me because I know you care
2015-09-13 25644, 2015
-
ruaok
I'm only retaliating. :-p
2015-09-13 25632, 2015
-
Gentlecat
returning None
2015-09-13 25651, 2015
-
Gentlecat
alastairp is not going to like that :)
2015-09-13 25658, 2015
-
ruaok
did you already read my commit?
2015-09-13 25605, 2015
-
Gentlecat
heh
2015-09-13 25627, 2015
-
Gentlecat
I would probably agree
2015-09-13 25637, 2015
-
ruaok
2015-09-13 25627, 2015
-
Gentlecat
exceptions are the jam :)
2015-09-13 25643, 2015
-
ruaok
?
2015-09-13 25629, 2015
-
ruaok
feh, time to re-merge the schema branch again.
2015-09-13 25635, 2015
-
ruaok
we should really merge that soon.
2015-09-13 25649, 2015
-
Gentlecat
if not messybrainz_resp: raise MessyBrainzUnavailableException
2015-09-13 25658, 2015
-
Gentlecat
and in catches below
2015-09-13 25601, 2015
-
Gentlecat
something like that
2015-09-13 25645, 2015
-
ruaok
ok, at this point there are much bigger fish I would like to fry.
2015-09-13 25659, 2015
-
ruaok
like getting it going. my solution is "good enough".
2015-09-13 25659, 2015
-
Gentlecat
yeah, I'm just saying :)
2015-09-13 25602, 2015
-
alastairp
make mr fussypants happy
2015-09-13 25632, 2015
-
alastairp
Gentlecat: I won’t tell him that “return” implicitly is the same as “return None” if you don't
2015-09-13 25646, 2015
-
Gentlecat
:P
2015-09-13 25622, 2015
-
alastairp
ok, I have to head out again. I’ll be back in Barcelona about 7:30 tomorrow afternoon, so will take a look at all the new code then
2015-09-13 25643, 2015
-
ruaok
k
2015-09-13 25655, 2015
-
ruaok
I may merge Russs new schema branch before then.
2015-09-13 25659, 2015
-
alastairp
ok
2015-09-13 25600, 2015
-
alastairp
cool
2015-09-13 25605, 2015
-
ruaok
I've run it several times and understand what it does.
2015-09-13 25609, 2015
-
alastairp
I wanted to fix msb as well
2015-09-13 25621, 2015
-
ruaok
let me update it to resolve the conflicts with sqlalchemy
2015-09-13 25621, 2015
-
alastairp
add in rj’s recommendations about adding artist and album ids
