#metabrainz

/

      • Ether_Man
        noci, can you give an example of something missing?
      • 2017-05-01 12117, 2017

      • Freso
        noci: I see three instances of 'type="EP"' in the XML I get from that query/lookup.
      • 2017-05-01 12117, 2017

      • noci
        hm o__O
      • 2017-05-01 12124, 2017

      • noci
        maybe I'm just blind
      • 2017-05-01 12130, 2017

      • noci
        let me check again xD
      • 2017-05-01 12111, 2017

      • noci
        you're right, hm, let me get back to my code and see, I must have mixed somehting up...
      • 2017-05-01 12103, 2017

      • Ether_Man
        Should also note, that the reason there's only 3, is because of paging.
      • 2017-05-01 12147, 2017

      • github joined the channel
      • 2017-05-01 12147, 2017

      • github
        [musicbrainz-server] Sophist-UK opened pull request #505: MBS-9323: Fix long URLs distorting page layout. (master...patch-1) https://git.io/v9lkV
      • 2017-05-01 12147, 2017

      • github has left the channel
      • 2017-05-01 12121, 2017

      • Sophist_UK
        d4rkie: Fix submitted for web-page distortion when URL is too long.
      • 2017-05-01 12102, 2017

      • Ether_Man
        noci, http://musicbrainz.org/ws/2/release-group?artist=… gives you all 14 on a single page, though I don't suggest fetching 100 but it shows that it's not the query that is incorrect at least :)
      • 2017-05-01 12129, 2017

      • noci
        yes this is highly embarrassing
      • 2017-05-01 12102, 2017

      • noci
        I seem to have mixed my queries together
      • 2017-05-01 12107, 2017

      • Mineo joined the channel
      • 2017-05-01 12128, 2017

      • Ether_Man
        Dude. It beats my literally pulling my hair for three days for why a program wasn't reading a change in the config file... The program runs as a different user entirely... I could have changed the config till I was blue and it couldn't have cared less :)
      • 2017-05-01 12141, 2017

      • noci
        hah
      • 2017-05-01 12142, 2017

      • Ether_Man
        beats me*
      • 2017-05-01 12150, 2017

      • noci
        hmm, is "release group" what I should be asking for by the way, and not "release"? (I want the official discography for an artist, like what's on the "Overview" page for an artist on musicbrainz.org)
      • 2017-05-01 12133, 2017

      • noci
        I *think* I would want to query "release" with status=official instead
      • 2017-05-01 12152, 2017

      • Ether_Man
        That depends. Do you differentiate between say the 12" Vinyl version of 'Anti EP' and the CD version? Or do you perhaps differentiate between the 2 different versions of Envane on 12" Vinyl?
      • 2017-05-01 12124, 2017

      • noci
        ah no, I don't care about different versions really
      • 2017-05-01 12136, 2017

      • Ether_Man
        Then release group is what you want.
      • 2017-05-01 12142, 2017

      • noci
        okay
      • 2017-05-01 12110, 2017

      • noci
        but there is no way to get only official releases on release-group is there?
      • 2017-05-01 12125, 2017

      • Ether_Man
        Define official here?
      • 2017-05-01 12139, 2017

      • noci
        has status=official instead of status=bootleg
      • 2017-05-01 12116, 2017

      • Ether_Man
        Well you can... But if you make no difference between the individual versions, what does it even matter? That's a status on the releases, not the release-group isn't it?
      • 2017-05-01 12124, 2017

      • noci
        yeah, if I append status=official on the release-group query I get "status is not a valid parameter unless release are requested"... but, like, if I do a query for release-group on a popular band like Iron Maiden, I get 100+ albums back - 90% bootlegs
      • 2017-05-01 12128, 2017

      • drsaunders has quit
      • 2017-05-01 12129, 2017

      • Ether_Man
      • 2017-05-01 12150, 2017

      • Ether_Man
        or no... That can't be the right
      • 2017-05-01 12116, 2017

      • noci
        wrong artist ID I think
      • 2017-05-01 12123, 2017

      • noci
      • 2017-05-01 12133, 2017

      • noci
        that's weird, I wonder what yours is
      • 2017-05-01 12147, 2017

      • Ether_Man
        http://musicbrainz.org/ws/2/release-group?artist=… that has to be the one. 122 hits... But I'm not seeing any bootlegs
      • 2017-05-01 12152, 2017

      • noci
        hm
      • 2017-05-01 12118, 2017

      • noci
      • 2017-05-01 12132, 2017

      • noci
        they have released a lot, but not really 122 albums and eps
      • 2017-05-01 12103, 2017

      • noci
        I get it to about 50 or so
      • 2017-05-01 12106, 2017

      • Ether_Man
        Right. But the 122 isn't querying for only albums and eps, it's quering for all types
      • 2017-05-01 12122, 2017

      • Ether_Man
        or no, wait, album and ep only is 122 :)
      • 2017-05-01 12124, 2017

      • noci
        uhm, but you have type=album|ep ?
      • 2017-05-01 12146, 2017

      • Ether_Man
        It's 182 without that :)
      • 2017-05-01 12155, 2017

      • noci
        ah :P
      • 2017-05-01 12116, 2017

      • noci
        I think most Live albums are bootlegs in the XML
      • 2017-05-01 12129, 2017

      • Ether_Man
        You can list all of that, if you browse https://musicbrainz.org/artist/ca891d65-d9b0-4258… instead
      • 2017-05-01 12148, 2017

      • noci
        ah yes
      • 2017-05-01 12154, 2017

      • Ether_Man
        Naa. Some are, but far from all
      • 2017-05-01 12107, 2017

      • noci
        so is there anyway I can do "all=0" ? in the XML response? ;)
      • 2017-05-01 12121, 2017

      • noci
        I guess I can live with it...
      • 2017-05-01 12134, 2017

      • noci
        but there will be a lot paging
      • 2017-05-01 12114, 2017

      • Ether_Man
        Not afaik. You could go around it by getting all releases that isn't bootlegs, and include release group info and build an index of that.
      • 2017-05-01 12109, 2017

      • noci
        yeah that could work, but I would really have to query the API a whole lot... I would like to be nice and query as little as possible
      • 2017-05-01 12109, 2017

      • noci
        might be better if I just mirror the database
      • 2017-05-01 12147, 2017

      • noci
        would be interesting to see what query is used to build the Overview page for each artist on musicbrainz.org tho
      • 2017-05-01 12128, 2017

      • Mineo
      • 2017-05-01 12147, 2017

      • noci
        ohum
      • 2017-05-01 12133, 2017

      • noci
        at least I'm not alone in this endeavour then :D
      • 2017-05-01 12142, 2017

      • noci
        that last url is almost exactly what I need I think, except I think I need first-release-date
      • 2017-05-01 12145, 2017

      • noci
        dang
      • 2017-05-01 12152, 2017

      • zas
        ruaok: ping
      • 2017-05-01 12102, 2017

      • ruaok
        zong
      • 2017-05-01 12115, 2017

      • noci
        and the ticket to fix it is 4 years old #feelsbadman
      • 2017-05-01 12133, 2017

      • d4rkie
        Sophist_UK: commented on it. break-all should work better, IMO
      • 2017-05-01 12132, 2017

      • zas
        ruaok: we have a disk space issue on prince & boingo, related to search containers, but it looks something weird
      • 2017-05-01 12149, 2017

      • zas
        space is mostly used by /var/lib/docker/aufs/
      • 2017-05-01 12136, 2017

      • zas
        and it keeps growing, i have the feeling something isn't cleaned up properly
      • 2017-05-01 12146, 2017

      • ruaok
        <== pre coffee
      • 2017-05-01 12146, 2017

      • zas
        ruaok: np ;) i'm investigating but for now not sure about this issue and possible solutions (also i'm pre-second coffee)
      • 2017-05-01 12157, 2017

      • ruaok
        without looking at it, I'm nearly certain (120% - 147%) that this is your problem. why? you touched the code last.
      • 2017-05-01 12100, 2017

      • ruaok
        :-D
      • 2017-05-01 12155, 2017

      • zas
        ahah, i think it is something more with docker, i have a feeling we have some kind of "leak"
      • 2017-05-01 12107, 2017

      • zas
        i'll reboot prince to test something
      • 2017-05-01 12125, 2017

      • Sophist_UK
        d4rkie: Agreed - I have made the change.
      • 2017-05-01 12115, 2017

      • amanmehta joined the channel
      • 2017-05-01 12145, 2017

      • ruaok
        zas: yeah, that was my feeling too. reboot.
      • 2017-05-01 12122, 2017

      • Ether_Man
        Hmm... I must be missing something. https://musicbrainz.org/artist/183d6ef6-e161-47ff… has an image, but http://musicbrainz.org/ws/2/artist/183d6ef6-e161-… shows no image relation? huh?
      • 2017-05-01 12106, 2017

      • Sophist_UK
        Ether_Man: The clue is the text below the picture on the web page. This is a picture that MBS automatically extracted from their Wikipedia page (in the same way that Releases can get pictures from Amazon.) I don't think that Artists can have pictures directly associated with them in the way that Releases can, so presumably this is the only way that MBS can get Artist images.
      • 2017-05-01 12109, 2017

      • Ether_Man
        http://musicbrainz.org/ws/2/artist/f27ec8db-af05-… Sure they can. Artist image url is rel type 221132e9-e30e-43f2-a741-15afc4c5fa7c
      • 2017-05-01 12132, 2017

      • arbenina joined the channel
      • 2017-05-01 12149, 2017

      • Nyanko-sensei joined the channel
      • 2017-05-01 12148, 2017

      • d4rkie has quit
      • 2017-05-01 12122, 2017

      • noci
        woho, I almost got it working now... playing with the API is fun, I'm surprised you're allowed to insert pure lucene syntax into your query
      • 2017-05-01 12147, 2017

      • noci
        only thing missing is release date, but I guess I'll have to marry the result with a second query like someone suggested
      • 2017-05-01 12155, 2017

      • zas
        ruaok: search server volume is unnamed (/home/search/indexes), is it intentional ?
      • 2017-05-01 12153, 2017

      • drsaunders joined the channel
      • 2017-05-01 12143, 2017

      • zas
        ruaok: i reclaimed the wasted space (140Gb), but i had to stop docker, remove old /var/lib/docker, and start over, does the container sync the indexes on start ?
      • 2017-05-01 12118, 2017

      • Sophist_UK
        Ether_Man: The difference is that the Michael Jackson entry explicitly includes a picture URL whilst They Might Be Giants doesn't.
      • 2017-05-01 12130, 2017

      • drsaunders has quit
      • 2017-05-01 12152, 2017

      • Sophist_UK
        So I was wrong about not being able to add an explicit picture. But that is the explanation as to what is currently happening.
      • 2017-05-01 12107, 2017

      • Ether_Man
        Yea. Just feels... Weird >_<
      • 2017-05-01 12119, 2017

      • Sophist_UK
        However, from a technical perspective, if MBS can find the picture itself, why shouldn't it be able to add it as an image entry in the JSON? I suggest you raise an Enhancement Ticket to see if this functionality can be added.
      • 2017-05-01 12127, 2017

      • Sophist_UK
        See also MBS-9223 for a similar but different issue about Amazon Cover Art.
      • 2017-05-01 12127, 2017

      • BrainzBot
        MBS-9223: Enhance ws response for ASIN to include cover-art URL https://tickets.metabrainz.org/browse/MBS-9223
      • 2017-05-01 12139, 2017

      • ruaok
        zas: the volume being unnamed is for no particular reason.
      • 2017-05-01 12151, 2017

      • ruaok
        and yes, it picks-up and re-syncs on startup
      • 2017-05-01 12111, 2017

      • Ether_Man
        Maybe. Want to get at least basic functionality in my Plex Agent first. Was mainly that like, the artist page had an image, was getting annoyed at why my agent was not fetching it :)
      • 2017-05-01 12144, 2017

      • zas
        ruaok: well, i'm not sure about the exact issue, i think docker itself is leaking stuff, the unnamed volume isn't really the issue
      • 2017-05-01 12158, 2017

      • zas
        that's related to aufs, i cleaned up dangling containers/images/volumes, and rebooted to clear temp stuff, but that was only 3% of disk space. Then i stopped docker, moved /var/lib/docker, and restarted it, the difference is huge
      • 2017-05-01 12116, 2017

      • zas
        only did it on prince for now
      • 2017-05-01 12132, 2017

      • zas
        i think named volumes for /home/search/ dirs may reduce useless data transfer and possibly loads (since it will not start from zero on container restart)
      • 2017-05-01 12129, 2017

      • reosarevok
        Ether_Man: we mostly take them from Wikidata, https://github.com/metabrainz/musicbrainz-server/… is the relevant code in case it's useful
      • 2017-05-01 12105, 2017

      • reosarevok
        (I guess you figured the first half out, but just in case the second half helps :) )
      • 2017-05-01 12159, 2017

      • Ether_Man
        reosarevok, any reason you're not simply using the commons-api? Seems to be a lot of waste to go through wikidata
      • 2017-05-01 12129, 2017

      • reosarevok
        We're picking the image set as the artist image in Wikidata
      • 2017-05-01 12140, 2017

      • reosarevok
        (which is the link we have for the artist)
      • 2017-05-01 12111, 2017

      • Freso
        Ether_Man: We try to link as much as possible to Wikidata, which is also what the Wikimedia Foundation prefers to use as their unique identifiers.
      • 2017-05-01 12129, 2017

      • reosarevok
        If we have a manually-set image for the artist, then we use that instead, of course, but otherwise Wikidata is the obvious place to check :)
      • 2017-05-01 12138, 2017

      • Freso
        So Wikidata is the entry point to Wikipedia, Commons, Quotes, …
      • 2017-05-01 12141, 2017

      • Ether_Man
        Right. Which gives you the html for the image on commons. Isn't it more efficient to ask the commons api for the image url directly?
      • 2017-05-01 12124, 2017

      • reosarevok
        Can you ask directly for the image linked as main image to a specific wikidata ID?
      • 2017-05-01 12128, 2017

      • reosarevok
        If so, it'd be, I guess
      • 2017-05-01 12137, 2017

      • Ether_Man
      • 2017-05-01 12118, 2017

      • Ether_Man
        Gives you both the url to the html page, and the image directly, as well as the url to the html for that specific revision
      • 2017-05-01 12135, 2017

      • reosarevok
        Yes, but you're asking for "Image:AlYankovicByKristineSlipson.jpg"
      • 2017-05-01 12149, 2017

      • reosarevok
        We don't have that, we just have "this is then entity at Wikidata ID Q24343543" or whatever
      • 2017-05-01 12155, 2017

      • reosarevok
        *the entity
      • 2017-05-01 12149, 2017

      • zas
        ruaok: proceeding on boingo now
      • 2017-05-01 12150, 2017

      • Ether_Man
        Right. But the data you get from there, is the url to the html descriptor page of the image.
      • 2017-05-01 12156, 2017

      • ruaok
        k
      • 2017-05-01 12137, 2017

      • reosarevok
        Sure, and then we call Commons with it: https://github.com/metabrainz/musicbrainz-server/… etc
      • 2017-05-01 12109, 2017

      • Ether_Man
        Right, but you're essentially fetching that full html page and then just discarding everything but the image. Rather than asking the API for just the image.
      • 2017-05-01 12109, 2017

      • reosarevok
        get_commons_image does exactly the API call you said https://github.com/metabrainz/musicbrainz-server/…
      • 2017-05-01 12131, 2017

      • reosarevok
        (unless I'm missing something)
      • 2017-05-01 12159, 2017

      • Ether_Man
        And before anyone gets annoyed. I'm not meaning this as critique or anything, I'm genuinely interested which is more efficient here
      • 2017-05-01 12121, 2017

      • reosarevok
        And get_wikidata_properties also makes an API call: https://github.com/metabrainz/musicbrainz-server/…
      • 2017-05-01 12122, 2017

      • Ether_Man
        oh sorry then :)
      • 2017-05-01 12140, 2017

      • reosarevok
        It does mean we need to make two calls, of course :)
      • 2017-05-01 12141, 2017

      • Ether_Man
        I read it as it was reading the descriptor page :)
      • 2017-05-01 12123, 2017

      • reosarevok
        But at the same time, it saves us worrying about whether the images move or change or whatever, and if someone adds a new better image to the Wikidata page, we get it automatically, which is nice
      • 2017-05-01 12136, 2017

      • Ether_Man
        Well yea, but one minimal call, and then the image, rather than a whole html page, at least feels like it should be more efficient :)
      • 2017-05-01 12155, 2017

      • reosarevok
        (before we started basing everything on Wikidata, we'd have to fix a few hundred Wikipedia links a month for example because they kept removing them or redirecting them :D )
      • 2017-05-01 12125, 2017

      • reosarevok
        But that's also, I guess, why they're not automatically added to the ws results
      • 2017-05-01 12133, 2017

      • Freso
      • 2017-05-01 12155, 2017

      • reosarevok
        (because we'd have to call Wikidata/Commons every to give the URL, which the user might not even need, so it's better if the user does it)
      • 2017-05-01 12117, 2017

      • Freso
        Yeah. We provide the data we have, which includes a lot of additional IDs and links that data users can use to query for more/other data on other services.
      • 2017-05-01 12122, 2017

      • Ether_Man
        If only Plex Agents actually had a system to STORE all those links, they'd be really handy indeed for me :)
      • 2017-05-01 12116, 2017

      • agentsim has quit
      • 2017-05-01 12143, 2017

      • Norwich_ joined the channel
      • 2017-05-01 12106, 2017

      • Norwich_
        Do you sell CDs ?
      • 2017-05-01 12100, 2017

      • ruaok
        no. we dont sell anything.
      • 2017-05-01 12159, 2017

      • Norwich_
        Someone pinched my CRIMCD86 ... I have an empty Case !!
      • 2017-05-01 12132, 2017

      • Norwich_ has quit
      • 2017-05-01 12157, 2017

      • ZaphodBeeblebrox
        ..right
      • 2017-05-01 12113, 2017

      • ZaphodBeeblebrox
        I was expecting a leadup to a joke honestly.
      • 2017-05-01 12151, 2017

      • MajorLurker
        you got it
      • 2017-05-01 12125, 2017

      • ZaphodBeeblebrox
        "Do you dig graves?"
      • 2017-05-01 12125, 2017

      • ZaphodBeeblebrox
        "Yea, they're alright"
      • 2017-05-01 12125, 2017

      • ZaphodBeeblebrox
        "I think they're wonderful"