zas: not urgent but you can remove musicbrainz-redis-store-beta from the redis stats whenever you get a chance, it's been merged into musicbrainz-redis-store
2020-05-26 14721, 2020
bitmap
btw, if anyone was logged out of beta let me know, out of curiosity (my session was preserved but there might've been race conditions during the move)
2020-05-26 14748, 2020
yvanzo
bitmap: can I help you with beta or redis?
2020-05-26 14757, 2020
bitmap
yvanzo: I finished merging the beta redis store into the prod one, so we can deploy #1529 to beta now if you're up for that
iliekcomputers: so, I've gotten very little to no feedback on the dups. everyone seems to be fixated on listen counts, rather than doing some digging to find dups.
2020-05-26 14750, 2020
ruaok
I think the listen counts in production are wrong too and I dont want to spend time fixing them.
2020-05-26 14718, 2020
iliekcomputers
The listen counts in production as in the user listen counts or the global listen count?
2020-05-26 14719, 2020
ruaok
I think the think to do is actually start a migration that could make its way to production.
2020-05-26 14731, 2020
ruaok
both probably
2020-05-26 14754, 2020
iliekcomputers
The global listen count is definitely incorrect
2020-05-26 14703, 2020
iliekcomputers
I remember opening an issue about it.
2020-05-26 14723, 2020
ruaok
so, by getting a production ready setup, we can create a more realistic comparison, that people can then ignore.
2020-05-26 14702, 2020
ruaok
but I wonder if "this is being ignored" is more like "I can't find problem, but I can't say for sure, so I won't say anything."
2020-05-26 14729, 2020
iliekcomputers
I think it's the second.
2020-05-26 14739, 2020
ruaok
likely.
2020-05-26 14709, 2020
ruaok
so, starting a production ready conversion.... I suppose first step might be for you to review the import script.
2020-05-26 14739, 2020
ruaok
then to do an actual migration -- which will be a bit tricky.
2020-05-26 14746, 2020
iliekcomputers
Yeah, that sounds good. Happy to review.
2020-05-26 14756, 2020
iliekcomputers
Would a doc detailing the steps be helpful?
2020-05-26 14703, 2020
ruaok
I would need to start a new exchange and connect it to the incoming stream.
2020-05-26 14710, 2020
ruaok
yes!
2020-05-26 14723, 2020
ruaok
I'll setup for the review and the doc after this chat.
2020-05-26 14716, 2020
ruaok
once the next change starts receiving listens (and the queue will grow significantly) we will need to trigger a new dump.
2020-05-26 14741, 2020
ruaok
and then once the dump is done, then it will take some 12-24 hours to prepare and import the dump.
2020-05-26 14757, 2020
ruaok
then we can connect it live and in theory we should have a clean and consistent database.
2020-05-26 14720, 2020
ruaok
I'm somewhat concerned about the number of listens growing, but I think we should be ok.
2020-05-26 14750, 2020
ruaok
so, right now my migration code is in a separate repo.
2020-05-26 14759, 2020
iliekcomputers
Could we just keep writing the listens in the queue and import simultaneously?
2020-05-26 14714, 2020
ruaok
that is exactly what I want to do.
2020-05-26 14721, 2020
iliekcomputers
Writing the listens to timescale
2020-05-26 14726, 2020
iliekcomputers
I meant to say
2020-05-26 14736, 2020
ruaok
ah, no, not ideal.
2020-05-26 14700, 2020
ruaok
the insert will be most performant when you have insert in sequential time order from oldest to newest.
2020-05-26 14717, 2020
iliekcomputers
Okay, that makes sense.
2020-05-26 14718, 2020
ruaok
having listens stream in will break that.
2020-05-26 14750, 2020
ruaok
what I suppose can try is to stream the listens to timescale *until* the import. then stop it, let the queue grow, do the import, then catch up.
2020-05-26 14709, 2020
ruaok
I *think* that wont impact the import so badly.
2020-05-26 14730, 2020
iliekcomputers
If the rabbitmq queue falls over, we'll just start the process over again with something else I guess. Sounds reasonable to me.
2020-05-26 14707, 2020
ruaok
given that the import code is going to run once, by me, I'm not too keen on importing the code into lb-server and tidying it up to our strict standards.
2020-05-26 14723, 2020
ruaok
can you review two scripts on their own and mainly look for logical errors?
is the second script that does the actual importing.
2020-05-26 14721, 2020
iliekcomputers
Okay, will read through when I get the time.
2020-05-26 14725, 2020
ruaok
ok, great.
2020-05-26 14730, 2020
ruaok
I'll work up a migration doc next.
2020-05-26 14738, 2020
iliekcomputers
Great, thanks!
2020-05-26 14751, 2020
ruaok
the importer is a bit tricky, since it uses threads to run 5 insertions at the same time.
2020-05-26 14704, 2020
ruaok
tuned to make the import move at a reasonable speed.
2020-05-26 14759, 2020
ruaok
the two most critical functions are check_for_duplicates and import_dump_file
2020-05-26 14709, 2020
iliekcomputers
Thanks, I'll take a look.
2020-05-26 14739, 2020
zas
yvanzo: we can stop search-server (old one) containers right?
2020-05-26 14755, 2020
yvanzo
zas: it is still available for mirrors using LUCENE search, not sure it is still used.
2020-05-26 14726, 2020
zas
we need to sort this out, because those are using quite a lot resources we may use for something else. What about stopping them for a while and see if anyone complains (if they do, we'll see what to do, but unlikely starting those again, rather pointing them at solr), I remember we had some hacks in preparation to ease the move, but they were never finished, and lead to a lot of complexity.
so I guess current instances aren't updated since a while
2020-05-26 14723, 2020
yvanzo
bitmap: I changed staticbrainz project on jenkins to remove previous temporary container, but the actual issue is that this container is not run and thus doesn't have a build/ directory anyway. Doesn’t seem to be in use currently anyway.
2020-05-26 14732, 2020
ishaanshah
iliekcomputers: Hi, please ping me when you get time
