#metabrainz

/

      • agentsim has quit
      • agentsim joined the channel
      • henadel
        ok, I finally managed to restart it, deleting and recreating the socket...
      • agentsim has quit
      • henadel has quit
      • ruaok returns
      • ruaok
        iliekcomputers: lol. :)
      • Slurpee joined the channel
      • Slurpee has quit
      • Slurpee joined the channel
      • everyone: we're about to unveil what zas has been working on.
      • zas
        (yes, i was working...)
      • ruaok
        I wish it was more positive, but still zas has some very good detective work.
      • Quesito: join us here.
      • Quesito
        hola
      • ruaok
        zas: please give us a short-ish recap of what happend.
      • zas
        short will be hard, but i'll try ;)
      • back to 2016, before we moved to hetzner, we started to see an increase in traffic
      • it was unexplained, we thought about people abusing our ws of course
      • but since we were on the move to new host, we didn't care that much
      • the whole move took us a lot of time and energy, until now
      • a week ago, there was a sudden drop in the traffic
      • so i started to dig logs to find an explanation
      • i noticed a lot of queries from 2 UAs, but from a lot of different IPs (>100k)
      • while i couldn't explain the drop in traffic, something was starting to appear
      • those 2 UAs were denied since a long time, being generic and non meaningful UAs (User Agent string in case of ...)
      • but still, the number of requests incoming was very high: about 60-65% of the total number of requests on our web service
      • so, we explored different possibilities, one was hacked devices, hosting botnet agents, querying us for unknown reason
      • so, i did few nmap and discovered that all random IPs i tested were having a common point: more or less same ports open to the internet
      • usual botnet agents are more smart, and use complex ways to hide their presence
      • ruaok
        these folks were anything but smart.
      • zas
        plus the botnet hypothesis would imply that someone was behind, someone very angry vs us....
      • so ... i used good'ol telnet, and discovered that all IPs were leading to a QNAP nas devicee
      • Leo_Verto[m]
        oh wow, that's really shitty of them
      • zas
        those devices are well known to be used by botnets ...
      • i started to block IPs of those devices
      • in the goal to be contacted by owners of those
      • and it worked, with the help of sam___,, rdswift, and few others we inspected those devices
      • and found nothing to worry about
      • but it confirmed that the requests were coming from them
      • why would a multimedia nas query mb.o ? .... ;)
      • either tagging or scraping
      • but no way to be sure until i got my hands on such device, so i ordered one yesterday, got it at home today, and started to look for what i was suspecting
      • and i found
      • reosarevok
        ...
      • zas
      • This Firefox/3.6.3 UA is one of the UAs hitting us
      • Leo_Verto[m]
        holy shit
      • they are intentionally faking the user agent?
      • zas
      • those programs belong to QNAP QTS system
      • so i googled for this libscrap.so
      • http://seclists.org/fulldisclosure/2017/Feb/35 is one document that was listed
      • ruaok
        Leo_Verto[m]: yes.
      • zas
      • i still have questions about the other UA (curl/7.43.0)
      • drsaunder joined the channel
      • but i think it was just the old version of libscrap.so without a UA set
      • both are sending requests in the exact same format (i found IPs emitting both, prolly someone having an old and a new NAS)
      • so here we are: QNAP NAS are sending tons of requests to us, users aren't aware of it, they are blocked since months (and no one cares)
      • it costs us a lot of resources, and time
      • that's all folks ;)
      • reosarevok
        So what now?
      • zas
        ^^ the question
      • ruaok
        now it goes over to quesito.
      • reosarevok
        I assume at least a "fuck these idiots" blog post, but is this something we can do something about more seriously?
      • ruaok
        with zas' info, her and I will work up the total damages done and equate them to $$$.
      • reosarevok
        Court-seriously?
      • Leo_Verto[m]
        there are API keys in one of the pastes, those aren't for anything MB, are they?
      • ruaok
        then we'll send them an invoice.
      • and just like the invoice that merkel received from trump, they are not going to pay.
      • zas
        Leo_Verto[m]: nope
      • ruaok
        but, we gotta do this the right way. introduce ourselves and then ask them to come clean.
      • zas
        but they'll be happy to know their API is used with hardcoded keys ;)
      • reosarevok
        I mean it's not just us, then? It's us, and IMDB, and who knows what else
      • Quesito
        step 1: play nice and ask nicely (even when you dont want to)
      • Leo_Verto[m]
        could you figure out if that same NAS software is hitting other services in the same way it hit MB? maybe teaming up would be a good idea, wouldn't it?
      • zas
        reosarevok: right, not just us
      • reosarevok
        Well I guess zas knows what else :D
      • Quesito
        we need to team up. this company is a giant.
      • we might need
      • ruaok
        think cake.
      • imdb == amazon
      • zas
        cake time!
      • ruaok
        they may not have noticed.
      • Quesito
        unless its like a class action...i donno.
      • ruaok
        we just need to follow the good cop route.
      • zas
        i don't think the court path is the right path, we actually want them to use our ws
      • ruaok
        no, this isn't class action worthy. this isn't worth a whole lot, really.
      • Quesito
        this is all still sinking in for me...this is f-d up.
      • zas
        but they have to respect rules
      • ruaok
        get another unicorn in the door and that is considerably much more money.
      • it is fucked up and we have a right to be angry.
      • but these people wasted our time, let's not waste too much more.
      • I get this feeling that these people are first rate douches.
      • and first rate douches know exactly what they are doing.
      • so, we can make a stink.
      • zas
        On https://stats.metabrainz.org/dashboard/db/mbsta... almost all 403 & 444 are from those devices
      • ruaok
        we can float this past the EFF and see if this fits some of the cases they are interested in fighting. if not, oh well.
      • samj1912
        So those nas services are purposefully using our api for commercial purposes without paying us and its not a botnet?
      • ruaok
        I doubt it is. with the 45 administration they have much more important things to do.
      • zas
        samj1912: exactly
      • ruaok
        Leo_Verto[m]: for fucking sure.
      • I think we should send everyone who helped us diagnose this some stickers. at the very least.
      • Quesito
        I will follow through with my tasks--but yeah--lets marinate on something further....
      • ruaok
        oh, I still have a ticketmaster voucher for canada.
      • zas
        yup, rdswift is one, aeromarine too
      • ruaok
        we can give it to sam________
      • zas
        sam__ == aeromarine now ;)
      • ruaok
        ah, great.
      • zas
        he learnt about irc during the process
      • Leo_Verto[m]
        I'm pretty sure enough media outlets would love the "huge company fucks over open source nonprofit" story
      • ruaok
        aeromarine: want a voucher for concerts?
      • Leo_Verto[m]: quite likely.
      • so, let me recap our plan of action:
      • 1. quantify the damages
      • 2. make contact, ask nicely for them to pay.
      • 3. make enough efforts of #2
      • 4. Dream up with a plan of action, then get board approval for this plan of action.
      • reosarevok
        So #2 is before press? boooring
      • ruaok
        5. Action. It may involve cakes, blog posts, media, etc.
      • we need to give them them benefit of the doubt.
      • agentsim joined the channel
      • Quesito
        boring but necessary....homework first....
      • suhas2go joined the channel
      • ruaok
        zas is planning to do a blog post about this tomorrow.
      • if the media picks up on that, fine.
      • zas
        contacting the guy behind http://seclists.org/fulldisclosure/2017/Feb/35 may help, he had contact with them
      • ruaok
        or on these chatlogs, even.
      • Quesito
        shouldn't we wait on that till after contact?
      • the blog post?
      • ruaok
        dear media: plz to be reading the backscroll!
      • samj1912
        Wow, that was kickass detective work btw, just read the complete backlog
      • ruaok
        Quesito: I was wondering about that.
      • samj1912
        !m zas
      • BrainzBot
        You're doing good work, zas!
      • zas
        thanks ;)
      • Quesito
        I think we should play conservative....
      • ruaok
        samj1912: that has been my thinking all along.
      • Quesito
        keep ours cards close to our chest.
      • ruaok
        Quesito: as much as I want to see this blog post, I think you're right.
      • Quesito
        amazing detective work zas
      • !!!
      • ruaok
        so, now it is time for Quesito to get her ducks in a row, then we'll see more.
      • zas
        about action, check the Timeline part at the end of http://seclists.org/fulldisclosure/2017/Feb/35
      • Quesito
        I can rock on this--but I'm gonna need at least one week. my file just started....
      • reosarevok
        Aww.
      • Since when did we become serious and responsible? :(
      • (but ok, I guess that makes sense)