10:41 AM
CatQuest
oh. ok
2017-06-26 17723, 2017
10:41 AM
ruaok
we'll see if we can improve on that.
2017-06-26 17730, 2017
10:41 AM
CatQuest
👍
2017-06-26 17709, 2017
10:42 AM
CatQuest
I think the diff is smaller this time though---
2017-06-26 17745, 2017
10:42 AM
ruaok
was 2k-3k last time, yes?
2017-06-26 17756, 2017
10:42 AM
CatQuest
last time when I tried importing the last 12-16 hours left of muiscl play ontop of it, it duplicated my enitre thing
2017-06-26 17719, 2017
10:43 AM
CatQuest
hmm I don't remember.. look in the chatlog
2017-06-26 17734, 2017
10:43 AM
CatQuest
or wait, the bug i made for it
2017-06-26 17724, 2017
10:44 AM
CatQuest
is i safe to reimport the newest listens from last.fm over my old import?
2017-06-26 17733, 2017
10:44 AM
ruaok
yes.
2017-06-26 17746, 2017
10:44 AM
ruaok
and now it does an incremental import that is much faster.
2017-06-26 17758, 2017
10:44 AM
CatQuest
👍
2017-06-26 17758, 2017
10:44 AM
CatQuest does it
2017-06-26 17704, 2017
10:45 AM
alastairp
(though that won't help for CatQuest's 1000 missing imports unless we delete the db again)
2017-06-26 17704, 2017
10:45 AM
CatQuest
uh, yes :O
2017-06-26 17753, 2017
10:45 AM
CatQuest
diff is Listen Count: 1,158,007 - Scrobbles 1,158,902
2017-06-26 17712, 2017
10:46 AM
CatQuest
so 1000>
2017-06-26 17719, 2017
10:46 AM
CatQuest
improvement!
2017-06-26 17745, 2017
10:46 AM
CatQuest
the "uh yes" was to ruaok's msg
2017-06-26 17723, 2017
10:54 AM
ruaok
Why not, alastairp ?
2017-06-26 17744, 2017
10:54 AM
ruaok
We could force full imports...
2017-06-26 17758, 2017
10:54 AM
alastairp
oh, sure. does that functionality exist yet? :)
2017-06-26 17710, 2017
10:56 AM
iliekcomputers
nope, it doesn't :)
2017-06-26 17736, 2017
10:57 AM
github joined the channel
2017-06-26 17736, 2017
10:57 AM
github
[critiquebrainz] ferbncode opened pull request #114: Add tests for functions users.tokens and users.clients (master...user-tests)
https://git.io/vQsIo
2017-06-26 17736, 2017
10:57 AM
github has left the channel
2017-06-26 17752, 2017
11:00 AM
ruaok
It is one single postgres query, we can manage.
2017-06-26 17733, 2017
11:01 AM
ruaok
For all users, anyway. Might be good to allow the user to do this from their profile.
2017-06-26 17750, 2017
11:03 AM
iliekcomputers
tbh, I think it would make the import page even more confusing. Plus ideally it shouldn't be needed ever
2017-06-26 17752, 2017
11:03 AM
zas
2017-06-26 17737, 2017
11:18 AM
samj1912 joined the channel
2017-06-26 17741, 2017
11:19 AM
samj1912
zas when you have time can you take a look at the ws PR?
2017-06-26 17706, 2017
11:44 AM
drsaunder has quit
2017-06-26 17710, 2017
11:47 AM
arbenina_ has quit
2017-06-26 17710, 2017
12:03 PM
github joined the channel
2017-06-26 17710, 2017
12:03 PM
github
[listenbrainz-server] paramsingh opened pull request #212: Add artist and recording MSID checks to dedup (master...add-check-dedup)
https://git.io/vQssy
2017-06-26 17710, 2017
12:03 PM
github has left the channel
2017-06-26 17735, 2017
12:18 PM
Freso
2017-06-26 17745, 2017
12:18 PM
Freso
(What's psolanki's MusicBrainz/forum user name?)
2017-06-26 17752, 2017
12:18 PM
D4RK-PH0ENiX has quit
2017-06-26 17720, 2017
12:20 PM
iliekcomputers
2017-06-26 17726, 2017
12:20 PM
iliekcomputers
Pinank_Solanki
2017-06-26 17735, 2017
12:20 PM
Freso
Cheers.
2017-06-26 17742, 2017
12:20 PM
iliekcomputers
:)
2017-06-26 17752, 2017
12:22 PM
Freso fixed notes
2017-06-26 17718, 2017
12:23 PM
arbenina_ joined the channel
2017-06-26 17756, 2017
12:25 PM
marc2k3
alastairp: since i was bored and your scrobble history isn't that big, I found your duplicates. 2 timestamps duplicated once each, all sequential one after the other...
https://pastebin.com/raKqkw0D
2017-06-26 17744, 2017
12:29 PM
marc2k3
i guess it's irrelevant once a fix is put in place but i was something to do for a couple of minutes!
2017-06-26 17753, 2017
12:30 PM
iliekcomputers
marc2k3: thanks! that is really helpful :)
2017-06-26 17720, 2017
12:31 PM
iliekcomputers
it kinda confirms our suspicions a bit :)
2017-06-26 17723, 2017
12:33 PM
D4RK-PH0ENiX joined the channel
2017-06-26 17726, 2017
12:33 PM
D4RK-PH0ENiX has quit
2017-06-26 17733, 2017
12:33 PM
D4RK-PH0ENiX joined the channel
2017-06-26 17747, 2017
12:34 PM
Freso
iliekcomputers ruaok: I just tried to re-import from Last.FM and it seems like the import is stuck. («Please do not close this page while the process is running» for 10+ minutes by now.)
2017-06-26 17755, 2017
12:34 PM
Freso
(On beta.lb.o.)
2017-06-26 17759, 2017
12:35 PM
Freso
(I also submitted some cached scrobbles from my phone of which some probably predate the last Last.FM beta.LB import, so maybe the timeline check in LB gets confused by that?)
2017-06-26 17728, 2017
12:36 PM
iliekcomputers
2017-06-26 17748, 2017
12:36 PM
iliekcomputers
Freso: could you see if there are any errors in the console?
2017-06-26 17723, 2017
12:38 PM
D4RK-PH0ENiX has quit
2017-06-26 17707, 2017
12:39 PM
Freso
2017-06-26 17729, 2017
12:39 PM
Freso
Let me try reloading the page with the console open and re-try the import.
2017-06-26 17751, 2017
12:39 PM
Freso
Oh, now I get 502 too. (I did manage to load some other pages though.)
2017-06-26 17731, 2017
12:42 PM
D4RK-PH0ENiX joined the channel
2017-06-26 17716, 2017
12:46 PM
ruaok
Beta is down.
2017-06-26 17736, 2017
12:46 PM
ruaok
Zas: I'm at lunch, you around d?
2017-06-26 17758, 2017
12:46 PM
zas
yes, let me check
2017-06-26 17734, 2017
12:47 PM
drsaunder joined the channel
2017-06-26 17747, 2017
12:50 PM
ruaok
Back up.
2017-06-26 17700, 2017
12:51 PM
ruaok
I needed to stop docker again.
2017-06-26 17731, 2017
12:51 PM
zas
you handle the issue ?
2017-06-26 17756, 2017
12:51 PM
ruaok
Mitigated, perhaps.
2017-06-26 17726, 2017
12:52 PM
ruaok
Certain it's not fixed.
2017-06-26 17737, 2017
12:52 PM
ruaok
No idea why it fails. :-(
2017-06-26 17704, 2017
12:57 PM
ruaok
2017-06-26 17727, 2017
12:57 PM
colbydray joined the channel
2017-06-26 17716, 2017
13:07 PM
Freso
Listen count: 39,111 // Scrobbles: 39,157
2017-06-26 17732, 2017
13:07 PM
Freso
("Freso" everywhere, for anyone wanting to go digging.)
2017-06-26 17737, 2017
13:19 PM
ruaok
Not yet, no. After the next fix, perhaps.
2017-06-26 17754, 2017
13:19 PM
ruaok
Not all discrepancies are mistakes, Freso
2017-06-26 17735, 2017
13:30 PM
samj1912 has quit
2017-06-26 17746, 2017
13:45 PM
agentsim joined the channel
2017-06-26 17709, 2017
13:47 PM
drsaunder has quit
2017-06-26 17726, 2017
13:55 PM
spuniun has quit
2017-06-26 17701, 2017
13:58 PM
spuniun joined the channel
2017-06-26 17722, 2017
14:02 PM
alastairp
marc2k3: thanks!
2017-06-26 17750, 2017
14:24 PM
marc2k3 has quit
2017-06-26 17749, 2017
14:26 PM
marc2k3 joined the channel
2017-06-26 17728, 2017
14:34 PM
Gentlecat
2017-06-26 17733, 2017
14:34 PM
Gentlecat
this is a bit confusing
2017-06-26 17725, 2017
14:36 PM
ferbncode
Gentlecat: any one of them. In the first case, dumps need to be downloaded separately and mounted as the volume. In second case, script automatically downloads the two dumps and does the import
2017-06-26 17746, 2017
14:36 PM
Gentlecat
I ran the second one and nothing was happening
2017-06-26 17754, 2017
14:36 PM
Gentlecat
at least that's the impression I got
2017-06-26 17727, 2017
14:37 PM
Gentlecat
2017-06-26 17744, 2017
14:38 PM
ferbncode
Gentlecat: maybe you have a directory called mbdata in critiquebrainz/data directory?
2017-06-26 17733, 2017
14:39 PM
Gentlecat
there shouldn't be one?
2017-06-26 17709, 2017
14:41 PM
Gentlecat
2017-06-26 17752, 2017
14:43 PM
alastairp
ferbncode: ferbncode wants to merge 2 commits into metabrainz:master from ferbncode:cb-231-entity/plate
2017-06-26 17757, 2017
14:43 PM
alastairp
plate?? :)
2017-06-26 17750, 2017
14:44 PM
alastairp
Gentlecat: hmm, you're right, that's not completely clear
2017-06-26 17700, 2017
14:45 PM
ferbncode
alastairp: I tried to change it later but then it became another one . typo :/
2017-06-26 17716, 2017
14:45 PM
alastairp
I believe that if you download the archive files you need to specify a -v in the docker-compose run line
2017-06-26 17718, 2017
14:45 PM
ferbncode
Gentlecat: I will try deleting my container and try again
2017-06-26 17722, 2017
14:45 PM
alastairp
let me see what I ran
2017-06-26 17735, 2017
14:45 PM
alastairp
docker-compose -f docker/docker-compose.dev.yml run -v `pwd`/mbdump/:/home/musicbrainz/dumps musicbrainz_db
2017-06-26 17719, 2017
14:47 PM
alastairp
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/create_test_db.sh
2017-06-26 17730, 2017
14:47 PM
alastairp
that looks wrong,
2017-06-26 17745, 2017
14:47 PM
alastairp
2017-06-26 17728, 2017
14:51 PM
arbenina_ has quit
2017-06-26 17744, 2017
14:55 PM
Gentlecat
and I'd prefer to have only one way of setting up the server described
2017-06-26 17716, 2017
14:56 PM
Gentlecat
perhaps note alternatives in some way that doesn't distract from the "main" path
2017-06-26 17709, 2017
14:57 PM
Gentlecat
2017-06-26 17730, 2017
14:57 PM
alastairp
do you think that download data + run, or run + automatically load data is best?
2017-06-26 17703, 2017
14:58 PM
drsaunder joined the channel
2017-06-26 17719, 2017
14:58 PM
Gentlecat
I'd prefer something that involves less manual steps, so second
2017-06-26 17716, 2017
14:59 PM
ferbncode
2017-06-26 17718, 2017
14:59 PM
alastairp
(that didn't actually work for me - something happened with the download operation and it disconnected halfway through, broken archive means that the import failed)
2017-06-26 17731, 2017
14:59 PM
alastairp
perhaps better to try and make it retry the downloads
2017-06-26 17739, 2017
15:02 PM
alastairp
Gentlecat: what image do you have listed in docker-compose.dev.yml for the musicbranz_db service?
2017-06-26 17707, 2017
15:03 PM
Gentlecat
also, if by default it downloads the data, why does it have to be a separate step and not just a part of `docker-compose ... up`?
2017-06-26 17721, 2017
15:03 PM
Gentlecat
alastairp: metabrainz/musicbrainz-test-database:schema-change-2017-q2
2017-06-26 17755, 2017
15:03 PM
alastairp
true, though it's a full import of musicbrainz. perhaps "run docker-compose up and wait for 4 hours" is a bit of a funny setup instructions
2017-06-26 17707, 2017
15:04 PM
Gentlecat
I deleted the container with the MB database and I still get the same error related to switching
2017-06-26 17721, 2017
15:04 PM
alastairp
yeah, weird. works here, with that image
2017-06-26 17725, 2017
15:04 PM
alastairp
are you on mac?
2017-06-26 17729, 2017
15:04 PM
Gentlecat
well, can note that it will take some time on the first run
2017-06-26 17731, 2017
15:04 PM
alastairp
shouldn't be an issue.... I hope??
2017-06-26 17745, 2017
15:04 PM
Gentlecat
there's no way around that now, so
2017-06-26 17755, 2017
15:04 PM
Gentlecat
yes, on mac
2017-06-26 17739, 2017
15:05 PM
Sophist-UK has quit
2017-06-26 17745, 2017
15:05 PM
Gentlecat
ok, I'll try to remove everything and start over
2017-06-26 17705, 2017
15:06 PM
Sophist-UK joined the channel
2017-06-26 17705, 2017
15:06 PM
Gentlecat
but I'm not sure why that would help, unless the image got updated
2017-06-26 17757, 2017
15:06 PM
alastairp
and the image should be tagged with the new version... I agree, it's kind of strange
2017-06-26 17758, 2017
15:07 PM
Gentlecat
another thing that I'm not sure about is `docker-compose -f docker/docker-compose.dev.yml build` being a separate step
2017-06-26 17752, 2017
15:08 PM
Gentlecat
the MB test database image has been updated 17 days ago, so I might have an old version
2017-06-26 17734, 2017
15:15 PM
arbenina_ joined the channel
2017-06-26 17710, 2017
15:20 PM
muesli has quit
2017-06-26 17710, 2017
15:20 PM
muesli joined the channel
2017-06-26 17745, 2017
15:20 PM
arbenina_ has quit
2017-06-26 17710, 2017
15:28 PM
drsaunder has quit
2017-06-26 17710, 2017
15:39 PM
Mineo joined the channel
2017-06-26 17753, 2017
15:40 PM
Gentlecat
2017-06-26 17754, 2017
15:43 PM
ferbncode
Gentlecat: the script runs only when there is no existing database "musicbrainz_db", so the script doesn't run. deleting the mbdata directory again would start things again.
2017-06-26 17701, 2017
15:44 PM
ferbncode notes that he should add another option to delete db and start import again
2017-06-26 17715, 2017
15:46 PM
Gentlecat
right, missed that part again
2017-06-26 17732, 2017
15:46 PM
Gentlecat
now it's downloading. I suspect that the problem was in the outdated image
2017-06-26 17710, 2017
15:50 PM
ferbncode
woo! :)
2017-06-26 17711, 2017
15:50 PM
ferbncode goes and fixes the script and the docs doing the noted changes
2017-06-26 17737, 2017
15:52 PM
ed___ joined the channel
2017-06-26 17754, 2017
15:52 PM
ed___ has quit
