#metabrainz

/

      • CatQuest
        am i des online!
      • arbenina_ joined the channel
      • MajorLurker
        apparently CatQuest
      • CatQuest has quit
      • or not
      • outsidecontext
        I am debugging the Discourse Onebox plugin on community.
      • obviously the default onebox implementations just use "Ruby" as user-agent. I guess this gets blocked by mb.org, right? @zas, @ruaok?
      • ruaok
        very likely yes.
      • outsidecontext
        freso: hey
      • yeah, got that e-mail. still thinking
      • ruaok
        not blocked outright, but throttled quite a lot.
      • setting a better UA string will fix this.
      • outsidecontext
        I just get 403s. unfortunately those Discourse plugins are hard to debug, as they just swallow all exceptions
      • will workaround this and add an implementation with proper UA
      • ruaok
        403s are not throttling issues.
      • must be something else. is this on ws/2 ?
      • outsidecontext
        yes
      • ruaok
        what is the request for?
      • outsidecontext
      • curl from command line works, same with browser. from ruby (with open-uri) I get a 403 (can reproduce in Ruby console)
      • ruaok
        that is odd -- there is no reason that URL should ever throw a 403.
      • bitmap: any ideas?
      • Rotab
        < HTTP/1.1 403 Forbidden
      • {"error": "Your requests are being throttled by MusicBrainz because the application you are using has not identified itself. Please update your application, and see http://musicbrainz.org/doc/XML_Web_Service/Rate... for more information."}
      • Freso
        chhavi_: Please go ahead and add it. :) And anything else you feel like adding too!
      • outsidecontext
        setting a differen UA works
      • ruaok
        ah, I guess the error code for incorrect UAs changed.
      • outsidecontext
        having the default "Ruby" is stupid anyway
      • Freso
        outsidecontext: 🙌
      • outsidecontext
        of course we could also consider whitelisting our own IPs :D
      • Freso
        Maybe make a "bug" for Discourse to change their default UA when pinging other sites?
      • I could see an IP whitelist getting outdated somewhat quickly. :)
      • outsidecontext
        it's actually inside their separate onebox library, and that could turn out to be rather generic anyway. I prefer to set a custom UA just for our plugin
      • Freso
        Sure.
      • ferbncode
        samj1912: Freso, Rotab: thanks :D
      • lazka has left the channel
      • Freso
        ferbncode: 👍 :) Have a good one!
      • ferbncode
        :)
      • Freso
        chhavi_, CallerNo6, reosarevok, etc.: https://community.metabrainz.org/t/documentatio...
      • ruaok
        airbnb washer update: Over the weekend I got a demand from the host for €150 because we broke the washer.
      • I told her it was the previous tenants and declined it.
      • she just got back to me and confirmed it. phew.
      • Freso
        !m ruaok
      • BrainzBot
        You're doing good work, ruaok!
      • D4RK-PH0ENiX has quit
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Meeting agenda (next: Oct. 9th): reviews, MBS-9353 (bitmap), GCI mentor comm. channel (Freso), MBS-9505 (yvanzo)
      • Freso
        yvanzo: How much time do you think that will be?
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Meeting agenda (next: Oct. 9th): reviews, MBS-9353 (bitmap), Google Code-in (Freso), MBS-9505 (yvanzo)
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Meeting agenda: reviews, MBS-9353 (bitmap), Google Code-in (Freso), MBS-9505 (yvanzo)
      • yvanzo
        Freso: very quick
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | Meeting agenda: reviews, MBS-9353 (bitmap), MBS-9505 (yvanzo), Google Code-in (Freso)
      • Freso
        Great, thanks. :)
      • adhawkins has quit
      • D4RK-PH0ENiX joined the channel
      • adhawkins joined the channel
      • D4RK-PH0ENiX has quit
      • D4RK-PH0ENiX joined the channel
      • adhawkins has quit
      • adhawkins joined the channel
      • Tykling has quit
      • Tykling joined the channel
      • khan joined the channel
      • outsidecontext
        freso: onebox plugin is updated, should work now again. maybe even more reliable than in the past, could well be we hit the rate limit earlier but did not recognize
      • last time I checked into this was the time the mb servers had load issues, and we assumed the loading issues were caused by this
      • ruaok
        what are your needs for the SOLR VM, samj1912 ?
      • Slurpee joined the channel
      • Slurpee has quit
      • samj1912
        ruaok: hey, what options do we have available and what will be feasible?
      • ruaok
        more options than you can count. :)
      • samj1912
        hehe :P
      • ruaok
        here is what we have so far:
      • 1) zas and I have been working on a VM that replicates the MB data
      • 2) we need to ensure that other VMs can connect to that postgres instance.
      • samj1912
      • deeptibaghel_ joined the channel
      • I am used to working with something like this
      • I think they will run solr fine
      • ruaok
        very likely overkill and kinda expensive.
      • you won't need to run postgres, which greatly reduces the overall footprint.
      • samj1912
        yeah, I usually take advantage of the student offers on digital ocean/GH :P
      • ruaok
        you need to run solr for indexing and serving.
      • samj1912
        any idea where this is hosted? https://solr.musicbrainz.org/solr/#/
      • ruaok
        this will be on Google Cloud where we are established and have sponsored credit.
      • at google.
      • samj1912
        okay
      • okay, so next question, do we use sol.mb instance or do I setup something from scratch?
      • s/sol.mb/solr.mb
      • ruaok
        oh, solr.mb.org is actually hosted at our hetzner setup.
      • but we all agree that we don't want it running there until the setup works well.
      • samj1912
        okay
      • so I will set it up from scratch on a google cloud vm?
      • ruaok
        I will do that for you.
      • samj1912
        okay
      • ruaok
        once we figure out our settings, I'll create the vm and hand you the IP address.
      • samj1912
        cool
      • ruaok
        we can change the VM config at any time.
      • samj1912
        meaning?
      • ruaok
        if we feel that it is over/underpowered, we stop it, change the configuration to add/remove what we need/don't need and then restart it.
      • so I am going to start small and then grow if you need it.
      • samj1912
        okay
      • ruaok
        2 cores 7.5GB of ram.
      • sounds like too little, but the VMs are quite powerful.
      • samj1912
        👍
      • ruaok
        200GB disk space?
      • samj1912
        is that hdd or ssd?
      • if possible, a smaller ssd space is preferrable
      • ruaok
        not sure.
      • its fast that is all I know.
      • we can't change the speed, but we can change the size.
      • samj1912
        cool :D
      • Slurpee joined the channel
      • Slurpee has quit
      • Slurpee joined the channel
      • ruaok
        if you think 100GB is enough, then we can do that.
      • disk isn't that expensive, so let's go with 200
      • should I use the SSH key you use on githhub?
      • with which username?
      • bitmap
        ruaok, outsidecontext: it looks like we don't specifically block "Ruby" but we 403 UAs <= 5 characters ("user-agent too short")
      • ruaok
        moin!
      • yea, it was the change from 503 -> 403 I wasn't aware of. makes sense though.
      • samj1912
        ruaok: https://github.com/metabrainz/syswiki/blob/mast... this works, username is "sam"
      • bitmap
        yeah, the message returned is a bit misleading since I think it is actually forbidden (we block them outright), not throttled
      • that's handled by openresty so zas would know more
      • UmkaDK joined the channel
      • ruaok
        samj1912: 35.192.95.65
      • log in and verify that you have sudo access
      • samj1912
        yup 👍
      • UmkaDK_ has quit
      • github joined the channel
      • github
        [musicbrainz-server] mwiencek closed pull request #557: MBS-9488: Changing "Limited editor" to "Beginner" (master...mbs-9488) https://git.io/vdZan
      • github has left the channel
      • drsaunde joined the channel
      • outsidecontext has quit
      • samj1912
      • ruaok
        any data there yet?
      • samj1912
        nope
      • just ran docker compose on docker-solr
      • seems to be working fine
      • ruaok
        good start.
      • samj1912
        thought of setting it up the normal way, but too many dependencies :P
      • ruaok
        and that would've been wrong, too.
      • samj1912 should get used to docker
      • you want to use docker, since that is what is going into production
      • samj1912
        yeah
      • ruaok
        yes, the sooner you do, the better it will be for you.
      • Sophist_UK joined the channel
      • Sophist-UK has quit
      • drsaunde has quit
      • samj1912
        ruaok: silly question but how do I make changes to the code and test a new build? the docker compose always pulls from mbsssss git repo and builds it, I want to change a few things there and then rebuild it
      • ruaok
        then you need to change how the container works.