I'm just bringing up the containers now, it seems to be installing cpan packages. will report back when it's done. I'll probably install the sample db too, because otherwise I'd have to index my full db
ruaok: oh ok, can you give me a link. when i search influx, I get results about influxdb instead
2020-09-10 25458, 2020
pristine___
iliekcomputers: oh! I will try. Thanks
2020-09-10 25403, 2020
alastairp
yvanzo: I've almost finished my beer and I don't yet have everything set up. is that a problem?
2020-09-10 25411, 2020
reosarevok
alastairp: depends, is it your last beer?
2020-09-10 25419, 2020
alastairp
there are plenty more
2020-09-10 25440, 2020
alastairp
it just depends if this is a 2-beer meeting or a 5-beer meeting
2020-09-10 25418, 2020
ruaok
_lucifer: link for what?
2020-09-10 25434, 2020
_lucifer
ruaok: i think i got confused. to be clear, timescale is used for storing listens but influxdb is used for monitoring/metrics
2020-09-10 25445, 2020
ruaok
correct.
2020-09-10 25409, 2020
_lucifer
👍
2020-09-10 25428, 2020
alastairp
yvanzo: I really like the docker-compose configuration/override system, especially this way of hooking up external folders as volumes in development setup. I think I'll copy this for some other projects that I have that have a similar setup
2020-09-10 25425, 2020
CatQuest
lFuXCRkz :D
2020-09-10 25433, 2020
iliekcomputers
how many beer meetings is the summit
2020-09-10 25459, 2020
iliekcomputers
pristine___: i'll deploy the cron PR
2020-09-10 25459, 2020
CatQuest
[17:12] <Lotheric> @listenbrainz devs (not sure who works on it). Think it'd be possible to adapt the world map to show where a user is making edits on musicbrainz user profile page ? edits on releases from xyz countries etc.
2020-09-10 25459, 2020
CatQuest agrees such a thing would be awesome
2020-09-10 25438, 2020
pristine___
iliekcomputers: thanks
2020-09-10 25440, 2020
alastairp
OK, I have indexer d-c setup, and the MB server running. installing sample now
2020-09-10 25415, 2020
iliekcomputers
that would be more MusicBrainz devs forte, the react component we used for the map is pretty extensible
yvanzo: the sir development setup that you gave me has pip install -r requirements commented out
2020-09-10 25404, 2020
reosarevok
Ok, recording finally done
2020-09-10 25410, 2020
reosarevok
I assume everything else is fast
2020-09-10 25411, 2020
yvanzo
alastairp: that's alrighty
2020-09-10 25441, 2020
alastairp
but when I tried to run `python -m sir reindex` it said that dependency raven was missing
2020-09-10 25455, 2020
alastairp
I modified the local Dockerfile to install the dependencies, and then mount the git checkout as a volume. now it's importing
2020-09-10 25436, 2020
yvanzo
I think I used pip at run time, but I cannot find it back.
2020-09-10 25402, 2020
alastairp
yes, that would probably have fixed it
2020-09-10 25424, 2020
alastairp
I wonder how reosarevok got it to work
2020-09-10 25449, 2020
reosarevok
Indexing first, then his code, is what he said
2020-09-10 25418, 2020
alastairp
reosarevok: ah, that might have done it
2020-09-10 25454, 2020
reosarevok
I'm still waiting because my laptop is unhappy with all this hard work
2020-09-10 25442, 2020
reosarevok
Terminal is not responding, that's a first
2020-09-10 25419, 2020
ruaok
not even your laptop does real work??
2020-09-10 25441, 2020
yvanzo
alastairp: yes, I just open a shell in indexer and install requirements by hand
2020-09-10 25445, 2020
shivam-kapila
I feel you reo
2020-09-10 25429, 2020
yvanzo
we could make it easier to use with an entrypoint and a start script like for musicbrainz, but it should work as-is.
2020-09-10 25429, 2020
reosarevok
ruaok: well it is a gaming laptop...
2020-09-10 25450, 2020
_lucifer
reosarevok: which one?
2020-09-10 25454, 2020
_lucifer
i am considering to buy one myself
2020-09-10 25400, 2020
reosarevok
Helios 300
2020-09-10 25419, 2020
shivam-kapila
ah the predator
2020-09-10 25423, 2020
reosarevok
It's... fine?
2020-09-10 25453, 2020
alastairp
yvanzo: this kind of setup is very similar to how we do AB/LB setup, etc. The Dockerfile ADDs the requirements file (could also get it from a git clone) and does pip install, and then we mount the working directory as a volume on top of /code
2020-09-10 25414, 2020
alastairp
I think that this would work fine for the dev setup for sir as well
2020-09-10 25450, 2020
yvanzo
alastairp: adding the requirements in Dockerfile is not possible because we want to use a mounted working copy of sir, which cannot be accessed during build.
2020-09-10 25403, 2020
yvanzo
or you assume requirements won't change too often.
2020-09-10 25423, 2020
reosarevok
My terminal still says importing work - is there a big step using a lot of memory just after that?
2020-09-10 25430, 2020
reosarevok
Wondering what got it so stuck
2020-09-10 25449, 2020
yvanzo
for MBS, requirements are installed at run time (by start script) in mounted directories from MBS working copy
2020-09-10 25457, 2020
yvanzo
alastairp: in our case, requirements will change since we are working on upgrading dependencies *<:@)
2020-09-10 25450, 2020
alastairp
I guess that's an option as well. to me it seems like a bit too much magic, I like the idea of explicitly installing dependencies with docker-compose build when I change them, so that I know that dependencies only change when I tell them to
2020-09-10 25448, 2020
alastairp
however with this setup (git clone to get requirements file in Dockerfile) I don't know the best way to do that. perhaps you're correct and that for this situation an entrypoint is a good idea
2020-09-10 25457, 2020
yvanzo
reosarevok: is your DockerRootDir on an external drive? that might explain unexpected freeze?
2020-09-10 25427, 2020
reosarevok
Nah
2020-09-10 25436, 2020
reosarevok
HDD not SSD, but internal
2020-09-10 25455, 2020
yvanzo
alastairp: it doesn't matter really, as long as you can do stuff, pip can be run from within the container in both cases
2020-09-10 25439, 2020
yvanzo
so local dev setup is done for everyone?
2020-09-10 25426, 2020
alastairp
solr http isn't exposed by default, right?
2020-09-10 25451, 2020
alastairp
I'm still indexing, but because I can't get a solr console I don't know how much longer it will take
which card? Migrate SIR to Python 3? or solr8 or java 11, or...? :)
2020-09-10 25450, 2020
shivam-kapila
MAybe I can catchi t
2020-09-10 25450, 2020
pristine___
but some other test files fail with this message
2020-09-10 25454, 2020
pristine___
oooooooo
2020-09-10 25459, 2020
pristine___
thankssssssss
2020-09-10 25429, 2020
yvanzo
alastairp: I would like to let you draw a card but the py3 seems to be the most blocking one;
2020-09-10 25447, 2020
alastairp
pristine___: if _redis is None, it seems like the problem here is that the init method wasn't called
2020-09-10 25458, 2020
alastairp
in what context is this?
2020-09-10 25404, 2020
alastairp
yvanzo: yeah, sure.
2020-09-10 25455, 2020
alastairp
what's the list of done/todo? syntax for python 3? unicode/str checking? dependencies are all py3 versions? (sqlalchemy causes problem here?) anything else?
2020-09-10 25407, 2020
pristine___
alastairp: I have no idea, I just wrote a few tests and they are working but some other scripts fail with this error
2020-09-10 25434, 2020
yvanzo
alastairp: see the checklist, it's iterative, not complete yet
2020-09-10 25452, 2020
alastairp
what do you mean by scripts? tests? or you have some things that you want to run in the context of a server? how are you starting the scripts?
2020-09-10 25455, 2020
yvanzo
sqlalchemy causes problems because sir doesn't use just the api, it uses internals too
2020-09-10 25423, 2020
pristine___
test_files/test_functions
2020-09-10 25440, 2020
alastairp
which is your Readiness list? 1.1, 1.2, 1.3?
2020-09-10 25408, 2020
yvanzo
right
2020-09-10 25459, 2020
yvanzo
anyone a bit familiar with sqlalchemy would probably rocket progress here
2020-09-10 25411, 2020
alastairp
is that a "hint hint"? :)
2020-09-10 25434, 2020
reosarevok
Cancelled the process - or trying to anyway
2020-09-10 25401, 2020
yvanzo
reosarevok: docker-compose stop search
2020-09-10 25411, 2020
alastairp
yvanzo: do you think that doing each version in order is the best way of trying the upgrade? (I don't really know myself if that's best or not. I've seen people upgrade many versions both ways)
2020-09-10 25442, 2020
alastairp
yvanzo: is the workflow through sir mostly the same with live indexing and the indexing command?
2020-09-10 25401, 2020
reosarevok
Do we *need* sqlalchemy?
2020-09-10 25407, 2020
alastairp
is there any specific process which is causing problems, or do both of them have a problem?
2020-09-10 25409, 2020
reosarevok
Why can't we use direct sql like in MB?
2020-09-10 25417, 2020
alastairp
reosarevok: unless you want to rewrite sir?
2020-09-10 25427, 2020
yvanzo
alastairp: migration guides explain known issues for each version at least, so I thought that would help to address a subset at a time.
