#musicbrainz-devel

/

      • ruaok
        hmmm.

        • 2015-03-03 06242, 2015

      • ruaok
        thats a totally new one.

        • 2015-03-03 06256, 2015

      • nikki
        I think I've seen ones like that before

        • 2015-03-03 06226, 2015

      • nikki
        although there are normally too many errors for me to keep track, I just refresh immediately

        • 2015-03-03 06251, 2015

      • ruaok
        when?

        • 2015-03-03 06217, 2015

      • nikki
        in general

        • 2015-03-03 06219, 2015

      • ruaok
        in the last 48 horus?

        • 2015-03-03 06227, 2015

      • nikki
        uhh, not sure

        • 2015-03-03 06250, 2015

      • nikki
        I have been getting errors in the inline search, but those don't show the actual error, just that the search failed

        • 2015-03-03 06222, 2015

      • ruaok
        is inline search now using indexed search?

        • 2015-03-03 06235, 2015

      • ruaok
        chirlu`: what search were you doing?

        • 2015-03-03 06241, 2015

      • ruaok
        index? direct? inline?

        • 2015-03-03 06243, 2015

      • nikki
        the default is the indexed search, yes, and has been for years

        • 2015-03-03 06249, 2015

      • ruaok
        .247 is the load balancer (nginx)

        • 2015-03-03 06227, 2015

      • ruaok
        and the search servers won’t be accessed for a while after they are load balanced.

        • 2015-03-03 06244, 2015

      • ruaok
        so, to me this sounds more like a front-end/ loadbal issue than search

        • 2015-03-03 06254, 2015

      • chirlu`
        Indexed search with advanced query syntax.

        • 2015-03-03 06223, 2015

      • ruaok
        intersting.

        • 2015-03-03 06241, 2015

      • reosarevok
        I've also gotten that a few times, but dunno if just recently

        • 2015-03-03 06225, 2015

      • ruaok
        ok, here comes the next index rotate.

        • 2015-03-03 06228, 2015

      • ruaok
        see how it goes this time.

        • 2015-03-03 06235, 2015

      • ruaok
        (still copying)

        • 2015-03-03 06246, 2015

      • chirlu`
        read timeout at /usr/share/perl5/LWP/Protocol/http.pm line 452, line 3.

        • 2015-03-03 06200, 2015

      • ruaok
        I think you hit it right during a reload.

        • 2015-03-03 06238, 2015

      • priyank joined the channel

        • 2015-03-03 06226, 2015

      • michaeljames joined the channel

        • 2015-03-03 06239, 2015

      • michaeljames
        http://search.musicbrainz.org/ws/2/recording/?max…

        • 2015-03-03 06241, 2015

      • michaeljames
        https://www.musicbrainz.org/ws/2/recording/?query…

        • 2015-03-03 06242, 2015

      • michaeljames
        does anyone know why this could be happening?

        • 2015-03-03 06245, 2015

      • chirlu`
        mix

        • 2015-03-03 06203, 2015

      • chirlu`
        michaeljames: Different queries, different results – doesn’t seem strange to me.

        • 2015-03-03 06246, 2015

      • chirlu`
        Also note that you probably want to make “jack johnson” a phrase.

        • 2015-03-03 06253, 2015

      • michaeljames
        What i'm trying to work out is what the website is requesting to the search server exactly

        • 2015-03-03 06223, 2015

      • michaeljames
        what does the request look like when its proxied to the search-server?

        • 2015-03-03 06243, 2015

      • michaeljames
        that way I can construct the requests for the search server directly

        • 2015-03-03 06201, 2015

      • michaeljames
        ahh are you saying that AND and + are different?

        • 2015-03-03 06204, 2015

      • michaeljames
        I didn't know that

        • 2015-03-03 06205, 2015

      • chirlu`
        Yes, + is just a space.

        • 2015-03-03 06218, 2015

      • michaeljames
        oh really? I had no idea

        • 2015-03-03 06234, 2015

      • chirlu`
        (In URLs.)

        • 2015-03-03 06256, 2015

      • michaeljames
        I thought %20 was the only way to resemble a space

        • 2015-03-03 06258, 2015

      • chirlu`
        Essentially, the server does not rewrite the queries in any way.

        • 2015-03-03 06206, 2015

      • michaeljames
        I was just having a look at the code it looks like it does some processing: https://bitbucket.org/metabrainz/musicbrainz-serv…

        • 2015-03-03 06204, 2015

      • michaeljames
        Thanks anyway. I now know + is a space.

        • 2015-03-03 06207, 2015

      • chirlu`
        No, all that code is for lookup and browse requests.

        • 2015-03-03 06223, 2015

      • chirlu`
        https://bitbucket.org/metabrainz/musicbrainz-serv…

        • 2015-03-03 06256, 2015

      • chirlu`
        If you want a real plus sign (which is not the same as AND), you need to percent-encode it.

        • 2015-03-03 06239, 2015

      • Freso
        michaeljames: https://musicbrainz.org/doc/Indexed_Search_Syntax

        • 2015-03-03 06240, 2015

      • michaeljames
        cool thanks I'll be using a library to make requests to this. I'll be doing encodeURIComponent in JS

        • 2015-03-03 06242, 2015

      • chirlu`
        And “artist:jack johnson” will search for “jack” in the artist field, and for “johnson” in the default field (title).

        • 2015-03-03 06244, 2015

      • michaeljames
        thanks

        • 2015-03-03 06209, 2015

      • michaeljames
        so I need artist:"jack johnson"

        • 2015-03-03 06210, 2015

      • michaeljames
        ?

        • 2015-03-03 06251, 2015

      • Freso
        Yep.

        • 2015-03-03 06219, 2015

      • chirlu`
        Or artist:(jack johnson)

        • 2015-03-03 06219, 2015

      • Freso
        Or artist:(jack johnson), I guess.

        • 2015-03-03 06224, 2015

      • Freso
        Heh.

        • 2015-03-03 06248, 2015

      • michaeljames
        cool, is there a preference over them?

        • 2015-03-03 06208, 2015

      • Freso
        Depends on how sure you are that "jack johnson" is how he is written.

        • 2015-03-03 06230, 2015

      • ruaok
        sudo: no tty present and no askpass program specified

        • 2015-03-03 06235, 2015

      • Freso
        artist:(jack johnson) search for "jack" and/or "johnson" on the artist field.

        • 2015-03-03 06240, 2015

      • chirlu`
        "jack johnson" searches for the phrase.

        • 2015-03-03 06255, 2015

      • Freso
        artist:"jack johnson" searches for "jack johnson" in it.

        • 2015-03-03 06211, 2015

      • michaeljames
        I see

        • 2015-03-03 06217, 2015

      • chirlu`
        So it will not find “Jack Joe Johnson”.

        • 2015-03-03 06204, 2015

      • michaeljames
        cool that makes sense

        • 2015-03-03 06208, 2015

      • CallerNo6
        this is true for 'method=advanced' only, isn't it?

        • 2015-03-03 06246, 2015

      • michaeljames
        O btw the search server I have setup I'm thinking that it might be useful to some people

        • 2015-03-03 06208, 2015

      • michaeljames
        so I'm thinking of sharing the AWS AMI instance publicly and have issues and docs on github

        • 2015-03-03 06217, 2015

      • chirlu`
        CallerNo6: Yes, but web service queries always used advanced syntax.

        • 2015-03-03 06209, 2015

      • michaeljames
        Is this something you guys may find useful?

        • 2015-03-03 06222, 2015

      • Freso
        ruaok, we can just make MB point to michaeljames' search instance and we won't have to host the search servers ourselves anymore! \o/

        • 2015-03-03 06228, 2015

      • ruaok
        perfect.

        • 2015-03-03 06237, 2015

      • Freso
        :p

        • 2015-03-03 06239, 2015

      • ruaok
        just make sure it can handle 10M queries in a day or so. :)

        • 2015-03-03 06208, 2015

      • michaeljames
        haha wow! Umm I'm not telling you the URL lol

        • 2015-03-03 06208, 2015

      • batmanav joined the channel

        • 2015-03-03 06210, 2015

      • reosarevok
        michaeljames: basically, not necessarily us but other people might - but you might end up finding too many people are hitting it, or with too many queries anwyay

        • 2015-03-03 06225, 2015

      • michaeljames
        yeah I won't make my url public

        • 2015-03-03 06230, 2015

      • Freso
        ;)

        • 2015-03-03 06240, 2015

      • Freso
        Just post it in the headphones forum.

        • 2015-03-03 06241, 2015

      • michaeljames
        just the image for people to setup their own server

        • 2015-03-03 06247, 2015

      • Freso
        (Well, I assume they have a forum anyway.)

        • 2015-03-03 06205, 2015

      • reosarevok
        Oh, ok

        • 2015-03-03 06205, 2015

      • Freso
        An image would be nice.

        • 2015-03-03 06207, 2015

      • michaeljames
        headphones forum is that on musicbrainz.org?

        • 2015-03-03 06211, 2015

      • reosarevok
        That would probably be helpful, yes :)

        • 2015-03-03 06220, 2015

      • michaeljames
        yeah I can share when its available

        • 2015-03-03 06226, 2015

      • Freso
        michaeljames: Nope. headphones is untirely unaffiliated with MB.

        • 2015-03-03 06233, 2015

      • reosarevok
        Nah, headphones is some app that people keep making their own VM servers for

        • 2015-03-03 06246, 2015

      • michaeljames
        i had issues getting the java servlet to build so I took the jar and war files from the blog site.

        • 2015-03-03 06258, 2015

      • Freso
        And the software likes to make a *lot* of requests in a short amount of time.

        • 2015-03-03 06209, 2015

      • michaeljames
        I see

        • 2015-03-03 06240, 2015

      • michaeljames
        whats the link to that app?

        • 2015-03-03 06250, 2015

      • michaeljames
        this is the jar and war files i'm using: http://blog.musicbrainz.org/?s=Search+server+jar%…

        • 2015-03-03 06217, 2015

      • michaeljames
        it was a last resort really as I couldn't get it to build

        • 2015-03-03 06240, 2015

      • michaeljames
        ah @chilù you mentioned the search server is completely outdated

        • 2015-03-03 06243, 2015

      • michaeljames
        in svn

        • 2015-03-03 06250, 2015

      • michaeljames
        where can I find the git repo?

        • 2015-03-03 06254, 2015

      • michaeljames
        http://chatlogs.musicbrainz.org/musicbrainz-devel…

        • 2015-03-03 06214, 2015

      • michaeljames
        chirlú

        • 2015-03-03 06214, 2015

      • reosarevok
        chirlu` :) (you can probably use tab to complete names that are hard-ish to write, like his)

        • 2015-03-03 06231, 2015

      • michaeljames
        ahh okay I'm an IRC NOOOB

        • 2015-03-03 06236, 2015

      • michaeljames
        I use slack now so...

        • 2015-03-03 06205, 2015

      • derwin
        SSDD, really

        • 2015-03-03 06231, 2015

      • michaeljames
        haha

        • 2015-03-03 06210, 2015

      • reosarevok
        michaeljames: https://bitbucket.org/metabrainz/search-server I think?

        • 2015-03-03 06258, 2015

      • michaeljames
        thanks

        • 2015-03-03 06259, 2015

      • michaeljames
        :)

        • 2015-03-03 06237, 2015

      • batmanav has left the channel

        • 2015-03-03 06212, 2015

      • michaeljames
        thanks for your help all

        • 2015-03-03 06222, 2015

      • michaeljames
        time to get back onto Slack.

        • 2015-03-03 06225, 2015

      • michaeljames
        :P

        • 2015-03-03 06252, 2015

      • ohrstrom joined the channel

        • 2015-03-03 06226, 2015

      • ruaok
        gah. chirlu` still up, dear research god?

        • 2015-03-03 06245, 2015

      • chirlu`
        :)

        • 2015-03-03 06236, 2015

      • ruaok
        on the search servers, I’ve got a script that restarts the search server.

        • 2015-03-03 06202, 2015

      • ruaok
        from the “search” account it is allowed to execute with NOPASSWD via the sudo file.

        • 2015-03-03 06213, 2015

      • ruaok
        works great when I have a terminal.

        • 2015-03-03 06238, 2015

      • ruaok
        but when the search indexer tries to remotely restart the server, the sudo refused to allow it and prompts for a passwd.

        • 2015-03-03 06218, 2015

      • ruaok
        the authorized_keys file on the target machine has the pre-qualifier in it:

        • 2015-03-03 06246, 2015

      • ruaok
        no-x11-forwarding,no-port-forwarding,no-agent-forwarding,command="./bin/receive-from-server", < normal SSH key follows >

        • 2015-03-03 06219, 2015

      • ruaok
        but it always asks for the password when I have no idea why it does.

        • 2015-03-03 06223, 2015

      • ruaok
        :(

        • 2015-03-03 06244, 2015

      • ruaok
        hmmm. playing around with it…

        • 2015-03-03 06254, 2015

      • ruaok
        it goes back to normal with out the command= argument

        • 2015-03-03 06226, 2015

      • chirlu`
        Hm.

        • 2015-03-03 06245, 2015

      • chirlu`
        What is when you put the command into the ssh invocation?

        • 2015-03-03 06212, 2015

      • ruaok
        I dont understand your question.

        • 2015-03-03 06221, 2015

      • chirlu`
        I mean, when you don’t run it via command="..." from authorized_keys, but (without that directive) from the ssh command line.

        • 2015-03-03 06255, 2015

      • chirlu`
        ssh host "./bin/receive-from-server"

        • 2015-03-03 06230, 2015

      • chirlu`
        It might also be a difference in working directories.

        • 2015-03-03 06247, 2015

      • ruaok
        Unrecognised command for bin/receive-from-server

        • 2015-03-03 06248, 2015

      • ruaok
        The command was:

        • 2015-03-03 06219, 2015

      • ruaok
        which means that the argument to the program was dropped.

        • 2015-03-03 06222, 2015

      • chirlu`
        Strange.

        • 2015-03-03 06251, 2015

      • chirlu`
        But how does it work via authorized_keys without an argument, then?

        • 2015-03-03 06259, 2015

      • chirlu`
        (Or is supposed to work.)

        • 2015-03-03 06209, 2015

      • ruaok
        via the command= argument.

        • 2015-03-03 06235, 2015

      • chirlu`
        Yes, but how does it get the “restart” argument?

        • 2015-03-03 06205, 2015

      • ruaok
        which specifies the script to run, which seems to get passed via $SSH_ORIGINAL_COMMAND the env var

        • 2015-03-03 06212, 2015

      • ruaok
        ok, if I skip that and use $1 instead of the var above, it gets the right command passed. yay.

        • 2015-03-03 06227, 2015

      • ruaok
        but then: “sudo: no tty present and no askpass program specified”

        • 2015-03-03 06230, 2015

      • ruaok
        FUCK.

        • 2015-03-03 06234, 2015

      • ruaok
        I’m going in circles.

        • 2015-03-03 06245, 2015

      • ruaok
        ok, no index updates for musicbrainz til I sleep and get back to it.

        • 2015-03-03 06229, 2015

      • derwin
        that sudo error

        • 2015-03-03 06237, 2015

      • derwin
        means you need to have passwordless sudo

        • 2015-03-03 06243, 2015

      • derwin
        for the user in question

        • 2015-03-03 06233, 2015

      • ruaok
        search ALL=(ALL) NOPASSWD: /home/search/search-indexer/bin/restart_search_server