ruaok: iliekcomputers: Got an exception on acousticbrainz regarding redis, it looks like it might not be configured to allow as much memory usage? Is this something that you can look at?
2019-02-01 03207, 2019
bukwurm
Yeah, I a couple of them
2019-02-01 03222, 2019
bukwurm
*see a
2019-02-01 03238, 2019
ruaok
do you know how much you need?
2019-02-01 03239, 2019
Mr_Monkey
If you want a fresh start, find the ones associated with the BB containers, and delete them.
2019-02-01 03213, 2019
bukwurm
I am running compose up, if everything goes well - I think I'll continue
2019-02-01 03217, 2019
alastairp
no idea sorry. All I can say is "it worked fine when we were using LB's!"
the only thing we store in the cache at the moment is the recent list of recordings on the homepage, and some data about stats
2019-02-01 03241, 2019
alastairp
I don't know how much that is
2019-02-01 03259, 2019
ruaok
admittedly 8MB is rather paltry.
2019-02-01 03221, 2019
alastairp
e.g. if we could upgrade it to ~500 MB I think that would be fine, but that means we should fix all our cache code to consider the possibility of the cache being full
It's behaving on my machine, but I know some users have had issues with ElasticSearch. This might be a similar issue, with ES being to greedy
2019-02-01 03236, 2019
Mr_Monkey
bukwurm: If you want, you can try updating the ES image version in docker-compose, and see if that makes a difference for resources. I've seen better performance from more recent versions.
2019-02-01 03251, 2019
pristine_ joined the channel
2019-02-01 03252, 2019
dolina joined the channel
2019-02-01 03232, 2019
Mr_Monkey
It's bound to break some code, but just to test resource-wise it might give you a better idea if that's the issue
2019-02-01 03230, 2019
pristine_
ruaok , iliekcomputers , Mr_Monkey , alastairp : how about making the last-fm imports dynamic? Once they give permission, should we import listens as and when they come without the user having to do it explicitly.
2019-02-01 03231, 2019
ruaok
is that possible?
2019-02-01 03255, 2019
iliekcomputers
Could be done like we do with Spotify probably
2019-02-01 03215, 2019
bukwurm
Mr_Monkey: I increased memory from 2GB to 4GB and build happened in 2-4 seconds with already running containers
2019-02-01 03229, 2019
ruaok
oh yeah, duh. the api is open and we've handled all the bad parts about it.
2019-02-01 03234, 2019
iliekcomputers
Things to think about are rate limits and possible duplicates
2019-02-01 03237, 2019
bukwurm
But now docker is taking 6GB ram
2019-02-01 03245, 2019
ruaok
duplicates. that's the main problem.
2019-02-01 03248, 2019
bukwurm
which is fine I guess
2019-02-01 03218, 2019
ruaok
I think we may need better duplicate support and removal of duplicates before we add the feature.
2019-02-01 03232, 2019
Mr_Monkey
:/ hungry hungry BB.
2019-02-01 03232, 2019
Mr_Monkey
So it's all working now, bukwurm ? Would you mind documenting your findings and solution in the repo?
2019-02-01 03201, 2019
bukwurm
Mr_Monkey: It's pretty fast now, CPU usage is also down around 100% though RAM usage is high
2019-02-01 03215, 2019
bukwurm
But I am still not getting any search results :/
2019-02-01 03259, 2019
Mr_Monkey
Did you try slamming a heavy object on the computer?
2019-02-01 03259, 2019
bukwurm
The logs give 200 Ok though
2019-02-01 03205, 2019
Mr_Monkey
Hm.
2019-02-01 03210, 2019
bukwurm
Lol
2019-02-01 03249, 2019
bukwurm
I'll try checking if indexes have been created or not in my case
I was going to say, try localhost:9099/search/reindex .
2019-02-01 03248, 2019
Mr_Monkey
You'll need to have a look at src/server/routes/search.js L129 before that
2019-02-01 03256, 2019
Mr_Monkey
bukwurm: ^
2019-02-01 03245, 2019
pristine_
ruaok: Lol.
2019-02-01 03235, 2019
ruaok
pristine_: I think focusing on apple music first is a good idea. we have *some* form on import for last.fm. let's do something we have no support for all...
2019-02-01 03202, 2019
iliekcomputers
>I managed to bring down 13 CIA offices, all FBI offices east of the Mississippi, and the entire Southeastern university Research Network.
2019-02-01 03210, 2019
iliekcomputers
Noiice
2019-02-01 03208, 2019
dolina has quit
2019-02-01 03218, 2019
Mr_Monkey
bukwurm: I meant Line 133
2019-02-01 03232, 2019
pristine_
ruaok: yeah. Sounds good.
2019-02-01 03236, 2019
bukwurm
*132
2019-02-01 03256, 2019
reosarevok
iliekcomputers: see, you only cost money. Not so bad! :p
2019-02-01 03210, 2019
dolina joined the channel
2019-02-01 03236, 2019
pristine_
ruaok: I did some testing last night. Tried consuming data from rabbitMQ on lb-server side and then pushed them to db.
2019-02-01 03213, 2019
pristine_
We need to create a producer at lb-playground
2019-02-01 03243, 2019
bukwurm
Mr_Monkey: Reindexing works
2019-02-01 03251, 2019
bukwurm
\o/
2019-02-01 03208, 2019
Mr_Monkey
bukwurm: OK, good.
2019-02-01 03216, 2019
Mr_Monkey
Everything working?
2019-02-01 03220, 2019
bukwurm
Looks like it
2019-02-01 03231, 2019
Mr_Monkey
\o/
2019-02-01 03206, 2019
bukwurm
One more thing: we're not using nodemon in docker run?
2019-02-01 03234, 2019
bukwurm
Mr_Monkey: /\
2019-02-01 03257, 2019
Mr_Monkey
bukwurm: No. Suposedly stopping and rerunning ./develops.hs should be fast enough. But you can definitely change the run command in your docker-compose file if you prefer
2019-02-01 03227, 2019
code_master5 joined the channel
2019-02-01 03251, 2019
code_master5
reosarevok: mind if I pm?
2019-02-01 03202, 2019
bukwurm
Mr_Monkey: Cool, and thank you!
2019-02-01 03222, 2019
Mr_Monkey
My pleasure, nice to see you around ! :)
2019-02-01 03241, 2019
bukwurm
> Would you mind documenting your findings and solution in the repo? 5:35 PM <bukwurm>
2019-02-01 03212, 2019
bukwurm
Should I add it in the README?
2019-02-01 03212, 2019
bukwurm
Yeah, I have been totally out of the loop for a couple of months now, hopefully I'll be able to contribute more :)
2019-02-01 03211, 2019
alastairp
pristine_: ruaok: remember the reason to do last.fm in the browser is so that it was the user who did the requests to l.fm, not us
2019-02-01 03252, 2019
Freso
amCap1712: Log out of forums and back, that should carry it over IIRC.
2019-02-01 03234, 2019
ruaok
alastairp: oh yes. Good point.
2019-02-01 03209, 2019
ruaok
pristine_: yeah, I need to start working on that very soon.
I had a question. Its been half a month that I am working on the app. Should I post an update in the GSoC proposal about what has been materialized in these days?
2019-02-01 03204, 2019
pristine_
ruaok: yeah. Would love to assist you in that 😆
2019-02-01 03214, 2019
ruaok
Well, if you want to start, we can make it your task and I'll mentor you...
2019-02-01 03244, 2019
Mr_Monkey
ruaok: I've made some good UI improvements on the listens page on branch spotify-player-styling. When you get a chance, can you try it out and give me your feedback?
2019-02-01 03255, 2019
Mr_Monkey
Same for iliekcomputers :)
2019-02-01 03235, 2019
pristine_
ruaok: sounds wow :)
2019-02-01 03246, 2019
iliekcomputers
ruaok: Note that we're talking about the stats movement from the cluster to hetzner and not listen streaming spark writer. :)
2019-02-01 03217, 2019
iliekcomputers
I think pristine_ has a consumer all ready for that.
2019-02-01 03249, 2019
iliekcomputers
I can take the spark writer stuff in the meanwhile if needed.
2019-02-01 03216, 2019
pristine_
ruoak: I made a PR yesterday. It calculates some stats for users. It is very basic as of now. I was thinking of using the same script to calculate stats of users such that we build a json doc for every user and then push it into the queue. That ways it would be easier to retrieve data from the other end and fill the db. What do you think?
2019-02-01 03226, 2019
alastairp
iliekcomputers: hi, I've finished all the docker python PRs, just got one small one to finish regarding the push.sh script
2019-02-01 03213, 2019
alastairp
I wanted to chat with zas and/or bitmap one final time about what we discussed yesterday regarding metabrainz-specific version tags in dockerhub (e.g. :3.6-2 or :3.6.8)
