#metabrainz

yeah, I saw that LB configures watch automatically if mode is development. I see now how that fits in with the builder container - you always want it running. I added a separate command for it so that we could run build:dev once off if you're doing other stuff, or build;watch if you're developing js

about the jenkins issue, right now the script `jenkins-test.sh` is in the jenkins config. Maybe we could change the config to just run the script off the git repository instead.

if that script is only in jenkins, then I agree, probably a good idea

gabbo: you can use this to do it in one go: https://gist.github.com/yvanzo/2845d2a79fad9924...

but you're also right that it raises an interesting question about where files generated during runtime should be kept - in the source tree, or in a separate working directory

I see the value for a working dir, then this could just be another volume somewhere + a config option

we have a setup at the uni where we can start a container as a user that exists on the host. we do it by creating a user in the Dockerfile and then passing in the user's UID and name as env vars. then the entrypoint runs usermod to change the user's name and uid so that permissions work properly on linux, and it looks like you're logged in as yourself

yvanzo That's very helpful, thanks a million. I will keep you updated (:

yvanzo It tells me: `sir: error: unrecognized arguments: -e area`. Is this syntax correct?

ishaanshah[m]: https://tickets.metabrainz.org/browse/LB-479

LB-479: Ship more statistics

gabbo: Oops, sorry, it is --entity-type instead of -e

ishaanshah[m]: I think https://tickets.metabrainz.org/browse/LB-542 would be a good task for you next, if you get the time

LB-542: Expose the artist stats on an API endpoint

yvanzo Thanks again

gabbo: 'recording' search index takes over 80% of the building time, you might want to skip it if you don't need to send recording search requests.

iliekcomputers: btw, not sure if you saw that I removed the use of the manifest file in development, so now you don't have to restart the webserver when the asset hashes change

nice, i wasn't aware that that had to be done in AB :D

how does it work in LB?

[listenbrainz-server] paramsingh opened pull request #813 (master…param/release-drafter): Add an initial config to release-drafter https://github.com/metabrainz/listenbrainz-serv...

i think the manifest.json in dev just doesn't have the hash

right. so you'd still have to restart if you added a new file

oh - perhaps LB never used gulp

because it had so little js, we had no compilation steps, and compiled the less with lessc

so when you switched to webpack, you could already do the magic development ? nohash : hash trick, which we didn't have in gulp

yeah, that makes sense.

it was all Mr_Monkey magic :D

[listenbrainz-server] paramsingh merged pull request #813 (master…param/release-drafter): Add an initial config to release-drafter https://github.com/metabrainz/listenbrainz-serv...

[listenbrainz-server] release v-2020-04-28.0 has been published by release-drafter[bot]: https://github.com/metabrainz/listenbrainz-serv...

reosarevok: the edit-list test was disabled by the (merged) guess-feat. pr

itunes scripts is broken because of the itunes's new design... again https://wiki.musicbrainz.org/User:Colbydray/Use... https://music.apple.com/us/album/lock-down-sing...

yvanzo Do live indexing and replication work in my case (not all indexes have been built) ?

yvanzo I'm experiencing the same problem when I try to import the artist. Same kind of exception, the procedure won't end .. Should I open an issue?

yvanzo: ping

Zastai: pong

gabbo: yes please, report this issue with as much relevant logs as possible.

live indexing cannot skip a particular search index

Ok thank you. Are there other doable ways to access data from MusicBrainz directly offline?

Probably not, but it depends on the info you want to query.

I'm trying to get mb-solr op and running, and it's proving challenging. Most of my trouble seems to stem from Java 11 vs Java 8.

But also, for the tests: do I understand that foo.xml is the input document, and foo-list.xml/json are the expected outputs?

yvanzo I need a way to retrieve the MBID for artists and release-groups. Once I have those, I can exploit the DB version without the search indices to access other information, granted that I have the IDs at my disposal

gabbo: define offline. you can download the data dumps and load those into postgres, without setting up the full MB server, and just SQL queries for your data

and if you only want artists and rgs, you can _probably_ import only those dumps

yvanzo Offline meaning that I'm currently using the XML WS 2 Apis to do everything I have described before (through the Python library), but that is too slow for my purposes. I need to do so without performing any HTTP calls in the internet

But querying instead a local dump

no way to do that and have live data, but if local dump is fine, then you should be able to do as I said - create the postgres db and import the tables you want from the dumps. the you just access pg from python to do lookups there

[listenbrainz-server] paramsingh opened pull request #814 (master…param/fix-spark-driver-size-bug): fix spark max driver result size bug https://github.com/metabrainz/listenbrainz-serv...

yvanzo: see above. and I take it foo.xml (area.xml etc) are supposed to be what sir produces? are they supposed to comply with mmd2-schema? (currently they don't; e.g. <lifespan><ended>false</ended></lifespan> is not valid because <ended> can only contain true)

Zastai: tests are not up-to-date indeed.

I had to update some maven dependencies (com.sun.xml.bind -> javax.bind for brainz-mmd-schema, and jaxb 2.3.1 for mb-solr)

See https://tickets.metabrainz.org/browse/SEARCH-593

SEARCH-593: Update MB Solr tests to work with 3.1

but the tests are still not running (as in: failing for non-functional reasons), in ways I can't really diagnose properly (for example, the json output seems to be "<0>.")

rather hard to implement fixes for the json output when there is no baseline to work from

and debian no longer ships a JDK8; guess I'll install a third-party one like Adopt, to rule out issues from that angle

yvanzo What do you suggest?

yvanzo: assuming I have time to work on it, would PRs be accepted to try and get things to the point where they run on Java 11 (brainz-mmd-schema and mb-solr)?

gabbo: I will look into this issue tomorrow, in the meantime mostly what Zastai suggested.

Zastai: Is https://hub.docker.com/r/metabrainz/solr/ still useful?

Zastai: It is based on adoptopenjdk you just mentioned.

yvanzo Zastai I don't know if I get it right: you suggest to use the dumps with Postgres to retrieve the MBID of Artists and Release-Groups, and, for everything else, use the Offline Server without Search Indices (that I'm not able to build right now)?

gabbo: What do you use to retrieve MBID? Artist’s name? If so, there is no easy way for you to do it right now.

For example, if you had something like a barcode, it would be easy to directly query directly the database instead.

yvanzo Yes, the Artist name ..

Zastai: About Java 11, would be nice but not sure that would work, “Some versions of Java VM have bugs that may impact your implementation.” https://lucene.apache.org/solr/guide/7_7/solr-s...

That seems simpler to fix tests staying with Java 8 at first, just follow the easiest path for you, assuming you have time, that would be awesome.

Anyway, I could get late entry into GSoC?

Too many broken links in the timeline graph :( I'll stop filing tickets for now.

yvanzo: not quite up on docker. as for the PRs, the first ones would just be bumped/changing mavend dependencies. for example, brainz-mmd-schema uses a comn.sun.xxx one for jaxb, but that one gets "illegal reflective access" warnings/errors on java 11. switching that to the javax.bind version 2.3.1 resolves that

and fits in better with mb-solr, whih already uses that (it just needs to go from 2.3.0 to 2.3.1)