monkey: on the above PR I have a failing test for SelectTimezone.test.tsx -- it checks to ensure that the output matches the expected output. But I can't find how it does this check -- I can't find another instance of that string.
2023-03-08 06750, 2023
mayhem
any tips on how to fix the test?
2023-03-08 06752, 2023
monkey
mayhem: Yes, it's the snapshots we use to ensure things like this don't change without us being aware of them.
2023-03-08 06710, 2023
monkey
test.sh has an option to update frontend snapshots
It becomes an annoying part of the workflow, like pep8, but it has saved my ass a couple of times
2023-03-08 06720, 2023
monkey
Actually, in this specific case since the change is just adding a letter you can do that directly in frontend/js/tests/user-settings/__snapshots__/SelectTimezone.test.tsx.snap if you don't want to re-run the whole test suite.
2023-03-08 06748, 2023
mayhem
ah, good to know. let me do the official updating for this round to learn the process.
2023-03-08 06732, 2023
mayhem
hmmm.
2023-03-08 06747, 2023
mayhem
so I did develop.sh build. then test.sh fe -u
2023-03-08 06753, 2023
mayhem
and it gives me: "frontend/js/src/explore/huesound/ColorPlay.tsx:7:23 - error TS2307: Cannot find module '@ebay/nice-modal-react' or its corresponding type declarations."
2023-03-08 06701, 2023
mayhem
which has your monkey paws all over it. :)
2023-03-08 06724, 2023
monkey
Ah, yes. Gotta rebuild the docker image…
2023-03-08 06737, 2023
monkey
Every time we add a dependency. Also annoying.
2023-03-08 06753, 2023
mayhem
isn't that develop.sh build ?
2023-03-08 06706, 2023
monkey
yes
2023-03-08 06709, 2023
mayhem
I did that.
2023-03-08 06733, 2023
mayhem
let me do a down, the build
2023-03-08 06737, 2023
monkey
Actually, `test.sh build`
2023-03-08 06752, 2023
monkey
Errr
2023-03-08 06702, 2023
monkey
`test.sh fe -b`
2023-03-08 06715, 2023
mayhem
thx
2023-03-08 06746, 2023
akshaaatt wishes everyone a happy holi✨
2023-03-08 06713, 2023
mayhem
happy holi! my favorite indian holiday. I need to be there for it someday.
2023-03-08 06706, 2023
akshaaatt
Really! I enjoyed a lot today. We celebrated holi at a popular hill station here
here's a sample of the output (it'll be zstd compressed when distributed), -original is the input document, -complete is a version of all rows from original that we were able fully match to a recording, -missing are everything else (only artist+release, or only artist, or only timestamp)
2023-03-08 06748, 2023
alastairp
I've eyeballed a bunch of these rows and they seem good to me, if someone else wants to look through them then I'm happy to help you with that
2023-03-08 06740, 2023
alastairp
wolf is about 30gb short to fit the whole dataset again (original is 270gb total but only 240 free, I'm taking a look to see if there are some low hanging fruit I can get rid of to free up space)
(I said it earlier too but it was probably too early :D)
2023-03-08 06728, 2023
TOPIC: 🎉 MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | BookBrainz: #bookbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, April Fools is getting near! (ghost of mayhem) 🎊
2023-03-08 06759, 2023
ZaphodBeeblebrox
I still maintain that "ghost of mayhem" is a great name for a band
2023-03-08 06728, 2023
monkey
+1
2023-03-08 06747, 2023
reosarevok
ZaphodBeeblebrox: they do have mountains! :)
2023-03-08 06741, 2023
ZaphodBeeblebrox
... yea of coruse.. I mean fak the hiest ones are in nepal over there so i guess there is snow
2023-03-08 06756, 2023
ZaphodBeeblebrox
.. course highest
2023-03-08 06720, 2023
ZaphodBeeblebrox
it just sounds so strange with "snow in india" taken out of context :D
2023-03-08 06728, 2023
ZaphodBeeblebrox
lik "heatwave in norway" i guess :DD
2023-03-08 06718, 2023
reosarevok
yvanzo, outsidecontext: MBS-12978 makes me want to cry T_T
outsidecontext, yvanzo: also, apparently they now decided the permalinks should be https
2023-03-08 06722, 2023
reosarevok
(so I guess we should change our cleanup too...)
2023-03-08 06743, 2023
reosarevok
But yeah, I'm starting to think maybe the whole check should be "is it a dnb link? ok, fine, add it, whatever" :p
2023-03-08 06751, 2023
outsidecontext
reosarevok: but the link fits into the definition we got from them in the mail I sent you. The link format is "https://d-nb.info/{ID}", where ID can consist of the characters [0-9X-]
2023-03-08 06749, 2023
outsidecontext
sorry, "https://d-nb.info/gnd/{ID}" for this case
2023-03-08 06708, 2023
TaJ joined the channel
2023-03-08 06743, 2023
outsidecontext
reosarevok: I don't think we should overcomplicate this. there are all kind of specific entity types that might have more detailed restrictions on the ID format. But to me this looks like over-engineering on our part, a more lax validation of the URLs should work well for us.
why only 3 cores, alastairp ? I am not using wolf at the moment, nor is anyone else. could go higher, if possible.
2023-03-08 06756, 2023
mayhem
reosarevok: ping
2023-03-08 06729, 2023
reosarevok
mayhem: pong, but off for dinner soon
2023-03-08 06747, 2023
mayhem
I'm wondering where to put the docs for the canonical datasets.
2023-03-08 06753, 2023
mayhem
wiki?
2023-03-08 06712, 2023
mayhem
I think I am going to put MHLD+ into the LB docs, so that has a good home.
2023-03-08 06729, 2023
reosarevok
Canonical in mbwiki/mbdoc seems fine
2023-03-08 06733, 2023
mayhem
ok
2023-03-08 06708, 2023
alastairp
mayhem: memory - we have a bunch of mapping dictionaries used to look up the canonical ids, redirects, metadata, etc, and they use about 30gb/process
2023-03-08 06741, 2023
mayhem
will memory use continue to grow?
2023-03-08 06747, 2023
mayhem
because it is at 25% now.
2023-03-08 06756, 2023
alastairp
my understanding is that we could have used threading with shared data, but as we're mostly CPU-bound, the GIL would stop us from getting more than 100% usage
2023-03-08 06705, 2023
alastairp
no, memory should be constant
2023-03-08 06727, 2023
alastairp
interesting, I was looking at RES in `top`, whcih shows 29g per process
2023-03-08 06728, 2023
mayhem
if memory is constant, why not start over and give it 6 threads. could be 8. there is enough memory available
2023-03-08 06752, 2023
alastairp
free -m shows 16gb avilable
2023-03-08 06700, 2023
alastairp
top shows 25% CPU, are you looking at that instead of memory?
2023-03-08 06706, 2023
mayhem
are these totally distinct processes? or one process and multiple threads?
2023-03-08 06734, 2023
alastairp
I'm running them as distinct processes
2023-03-08 06743, 2023
mayhem
ah, ok, never mind.
2023-03-08 06758, 2023
alastairp
one process + separate threads would definitely allow us to do more computation with shared data (since they're only lookup tables)
2023-03-08 06708, 2023
mayhem
yerp.
2023-03-08 06709, 2023
alastairp
but I was completely unsure if the GIL would screw us over there or not
2023-03-08 06715, 2023
mayhem
but you're afraid of threads, I understand.
2023-03-08 06720, 2023
alastairp
:D
2023-03-08 06708, 2023
alastairp
so I decided that waiting 1.25 days was better than spending 1 day trying to add threads and then finding out that it wouldn't work
