II'm setting up local server using docker and I ran below command to build the search index. The command is downloading files with name likes replication*.tar.bz2 and running since last 30 hours and still not completed. so just wondering how much data it is supposed to download?
2020-10-05 27953, 2020
apiuser
sudo docker-compose run --rm musicbrainz fetch-dump.sh search
2020-10-05 27920, 2020
apiuser
the documentation says that it would fetch around 28 GB of data. is it?
2020-10-05 27928, 2020
yvanzo
Hi apiuser: yes, it is that large.
2020-10-05 27924, 2020
yvanzo
These dumps are made available in case you cannot build search indexes from your own server.
apiuser: it depends on how long ago you downloaded the data dump, there is a replication packet per hour, it takes about 1min to get and apply an hourly packet.
2020-10-05 27951, 2020
apiuser
I just downloaded the database 2 days back.
2020-10-05 27927, 2020
yvanzo
so that makes at most 48 packets
2020-10-05 27929, 2020
yvanzo
it should not be running for 30 hours
2020-10-05 27900, 2020
v6lur joined the channel
2020-10-05 27938, 2020
livingsilver94 joined the channel
2020-10-05 27943, 2020
livingsilver94_ has quit
2020-10-05 27918, 2020
ruaok
pristine___: I hope bitmap likes steely dan.
2020-10-05 27931, 2020
ruaok
but hey, it worked!
2020-10-05 27935, 2020
ruaok
also, moooin!
2020-10-05 27902, 2020
Gazooo794 has quit
2020-10-05 27945, 2020
Gazooo794 joined the channel
2020-10-05 27914, 2020
ishaanshah
pristine___: hi
2020-10-05 27900, 2020
jmp_music__
Morning!
2020-10-05 27954, 2020
supersandro2000 has quit
2020-10-05 27914, 2020
supersandro2000 joined the channel
2020-10-05 27957, 2020
kori has quit
2020-10-05 27926, 2020
pristine___
ishaanshah: recs were generated fir bitmap for last week, but the stat endpoint for him returns empty response, can you have a look?
2020-10-05 27949, 2020
pristine___
ruaok: steely dan is his top artist so I guess he likes steely dan :)
2020-10-05 27924, 2020
apiuser
yvanzo, thank for the clue. I need to check for any issue at my end.
2020-10-05 27950, 2020
kieto joined the channel
2020-10-05 27935, 2020
d4rkie joined the channel
2020-10-05 27918, 2020
ishaanshah
pristine___: The full dump import has been failing because of some issue with server space for past two weeks
2020-10-05 27953, 2020
ishaanshah
so the data in spark for last two weeks is wrong, ig something to do with that
problem with that model is you end up buying lossy music
2020-10-05 27921, 2020
Lotheric
if I'm going to buy music, I want lossless
2020-10-05 27905, 2020
c1e0 has quit
2020-10-05 27949, 2020
MajorLurker has quit
2020-10-05 27910, 2020
_lucifer
CatQuest: what's the difference between Bokmål, Norwegian and Norwegian ?
2020-10-05 27913, 2020
c1e0 joined the channel
2020-10-05 27948, 2020
c1e0 has quit
2020-10-05 27926, 2020
c1e0 joined the channel
2020-10-05 27913, 2020
bitmap
pristine___: cool, thanks for the reminder!
2020-10-05 27903, 2020
bitmap
I guess the similar_artist one is more useful to me since I don't listen to a lot of different artists in the span of a week
2020-10-05 27910, 2020
apiuser has quit
2020-10-05 27916, 2020
pristine___
bitmap: yeah, rn top artist have like 200 recs, so if you have about only 5-6 tracks of your last week's top artist, will the top_artist playlist be useful for you?
2020-10-05 27917, 2020
supersandro2000 has quit
2020-10-05 27947, 2020
supersandro2000 joined the channel
2020-10-05 27922, 2020
pristine___
I think you were overwhelmed by the tracks of steely dan
2020-10-05 27930, 2020
pristine___
alone
2020-10-05 27931, 2020
_lucifer
alastairp: can you check what is the latest sql schema on beta?
2020-10-05 27918, 2020
bitmap
that might help though I'd mostly use the recs to find new music or music I've forgotten about, and top_artist has a lot of songs I've listened to in the past few weeks
2020-10-05 27944, 2020
bitmap
steely dan also wasn't my top artist last week though I guess that's 'cause the stats were broken
2020-10-05 27919, 2020
bitmap
last.fm says they were #3
2020-10-05 27942, 2020
pristine___
Bitmap: hmm.... Rn you won't have any tracks in your recs which you have listened to in the last week. I think 7 days is a very small window for people to not have the taste of music they have listened to. A few other users also have the same concern, I think if we increase this window, top artist recs might make more sense.
2020-10-05 27914, 2020
bitmap
yeah, hard to say what's best for all users. a larger window would be better for me, but that's 'cause I mostly rotate the same 2-3 albums for a few weeks and then move onto new ones
2020-10-05 27956, 2020
ruaok
bitmap: agreed. these two recommended tracks were originally intended for a "daily mix" sort of playlist that was based on what you've recently listened to.
2020-10-05 27918, 2020
ruaok
for the "jump back in" or the "we think you might like" we'll need to train more models...
2020-10-05 27925, 2020
ruaok
in due time.
2020-10-05 27908, 2020
bitmap nods
2020-10-05 27927, 2020
pristine___
feedback will help in training models :)
2020-10-05 27928, 2020
bitmap
it seems like a good list of songs for the 'daily mix' use case, so nice work there
2020-10-05 27946, 2020
ruaok
:)
2020-10-05 27901, 2020
ruaok
case in point, we need to improve how we present these algorithms[]
2020-10-05 27957, 2020
pristine___
Any doc of the summit?
2020-10-05 27922, 2020
c1e0 has quit
2020-10-05 27918, 2020
c1e0 joined the channel
2020-10-05 27955, 2020
livingsilver94 has quit
2020-10-05 27946, 2020
livingsilver94 joined the channel
2020-10-05 27948, 2020
d4rkie has quit
2020-10-05 27926, 2020
D4RK-PH0ENiX joined the channel
2020-10-05 27920, 2020
c1e0 has quit
2020-10-05 27955, 2020
c1e0 joined the channel
2020-10-05 27958, 2020
c1e0 has quit
2020-10-05 27950, 2020
yvanzo has quit
2020-10-05 27935, 2020
yvanzo joined the channel
2020-10-05 27951, 2020
c1e0 joined the channel
2020-10-05 27956, 2020
jwf
bitmap: Hahah good to know I am not the only one who goes through music listening phases like that!
2020-10-05 27930, 2020
CatQuest
[14:07] <_lucifer> CatQuest: what's the difference between Bokmål, Norwegian and Norwegian ?
2020-10-05 27930, 2020
CatQuest
I assume you mean nynorsk and norwegian
2020-10-05 27930, 2020
CatQuest
basically nynorsk and bokmål are writing systems. one is based on danish which was "norwegianified" (bokmål) the other is a constructed language based on several dialects, primarly in west and middle norway
2020-10-05 27930, 2020
CatQuest
there is of course many dialects, and "norwegian" is basically any of them
2020-10-05 27930, 2020
CatQuest
I am fro moslo so I predominantly write "bokmål" (or a easter-dialecticaly modification of such (more "a" endings and difthnogs etc)) but i do not speeak it
2020-10-05 27913, 2020
CatQuest
the nynorsk-bokmål thing is kidna a big issue, historical abotu independance and so on, wikipedia can describe it better thna i can in irc here :D
2020-10-05 27916, 2020
_lucifer
CatQuest: ah that's too much info :D. i was looking at a locale error that popped in logs and traced it to that.
2020-10-05 27922, 2020
CatQuest
in general I want both in mb, but I also want a more generic header "norwegian" for things that are written as neither (like the Vazelina Bilopphøggers' band which sing and write titles in "toten" dialenct)
2020-10-05 27933, 2020
_lucifer
which one should we be using ?
2020-10-05 27934, 2020
_lucifer
oh ok
2020-10-05 27937, 2020
CatQuest
all 3
2020-10-05 27941, 2020
CatQuest
(ideally)
2020-10-05 27922, 2020
CatQuest
bokmål is a writing system. and the most used one, but in the districts they use nynorsk. and it's a mandatory thing that all important publications be written in both forms equally
2020-10-05 27904, 2020
CatQuest
but for music, especially more recently, people are more and more also writing in dialect, includiong socialect and "kebabnorsk" (other language inspired youth-language)
2020-10-05 27922, 2020
CatQuest
they have words from polis, turk, hindi and urdu :D
2020-10-05 27933, 2020
CatQuest
also people are taught both forms in school
2020-10-05 27944, 2020
CatQuest
they are mututally intellible (mostly)
2020-10-05 27949, 2020
alastairp
_lucifer: it's possible that there might be an issue confusing country codes and language codes here
2020-10-05 27950, 2020
CatQuest
some words i have no idea
2020-10-05 27954, 2020
CatQuest
yes
2020-10-05 27907, 2020
_lucifer
alastairp: yes, that's what i am thinking about
2020-10-05 27913, 2020
CatQuest
for all norway is country code is no/nor, for nynorsk, it's nn and bokmål is nb
2020-10-05 27941, 2020
alastairp
👍
2020-10-05 27901, 2020
alastairp
I had no idea that norway had different languages like this! super interesting, CatQuest
2020-10-05 27907, 2020
CatQuest
and I believe this is also the reason we once time removed the "generic" norwegian" (because als owikipedia did this I think) but for situiation where e"it's neither nn or nb i want it still
