#metabrainz

/

      • QuoraUK_
        just need the auth thing and its good to go
      • 2016-06-28 18053, 2016

      • kmiller_
        ruaok - just added a key to github, check now
      • 2016-06-28 18054, 2016

      • Zastai has quit
      • 2016-06-28 18043, 2016

      • LordSputnik
        OK, so what order should your branches in -site be merged into the master branch?
      • 2016-06-28 18051, 2016

      • demonimin joined the channel
      • 2016-06-28 18016, 2016

      • QuoraUK_
        I was going to ask about this
      • 2016-06-28 18030, 2016

      • LordSputnik
        We've got gamification-dummy-function, profile-page, Achievement-Admin-System and new-achievements, I think?
      • 2016-06-28 18040, 2016

      • QuoraUK_
        I may have done it in a strange way
      • 2016-06-28 18040, 2016

      • LordSputnik
        Do they follow on from each other?
      • 2016-06-28 18042, 2016

      • QuoraUK_
        yeah
      • 2016-06-28 18057, 2016

      • QuoraUK_
        so Achievement-Admin-System is the most up to date one
      • 2016-06-28 18012, 2016

      • LordSputnik
        OK, I think we'll just about be OK then
      • 2016-06-28 18040, 2016

      • LordSputnik
        Could you start by making a PR for the dummy functions branch tomorrow, and then we can get that merged and try and do some more
      • 2016-06-28 18049, 2016

      • QuoraUK_
        Im working on getting my git sorted out this week as I discussed with Leftmost, ill try and do better on that front :P
      • 2016-06-28 18001, 2016

      • QuoraUK_
        will do yeah :)
      • 2016-06-28 18017, 2016

      • ruaok
        kmiller_: try sshing to bert.musicbrainz.org
      • 2016-06-28 18038, 2016

      • LordSputnik
        And I'll try to close your data-js PR tonight if I can get to it
      • 2016-06-28 18007, 2016

      • kmiller_
        works
      • 2016-06-28 18021, 2016

      • LordSputnik
        (and good job on exploring MB a little too - I was going to type that but then we moved on!)
      • 2016-06-28 18029, 2016

      • QuoraUK_
        LordSputnik: cool, that'd be great :)
      • 2016-06-28 18034, 2016

      • LordSputnik
        So what's your plan for this week and for tomorrow in particular?
      • 2016-06-28 18037, 2016

      • ruaok
        ok, that is the active gateway. we have another one, but we've take it out of rotation for the duration of these tests.
      • 2016-06-28 18054, 2016

      • ruaok
        you currently do not have sudo. do you think you'll need it, kmiller_?
      • 2016-06-28 18039, 2016

      • kmiller_
        I don't think I will for now...I'll let you know though
      • 2016-06-28 18047, 2016

      • ruaok
        k.
      • 2016-06-28 18058, 2016

      • QuoraUK_
        Attempt to finish the list of achievements I have on the doc I think
      • 2016-06-28 18007, 2016

      • ruaok
        I'm kinda dragging ass this evening. 'twill be an early bed time for me. Maybe 90 minutes from now.
      • 2016-06-28 18017, 2016

      • QuoraUK_
        as well as add the hover over for title thing to the rest of the site
      • 2016-06-28 18019, 2016

      • ruaok
        so, if you need anything, kmiller_, shout before then, please.
      • 2016-06-28 18031, 2016

      • kmiller_
        ok, I'm gonna spend a few minutes mirroring this port then tcpdumping it, then stopping these pings and seeing what happens
      • 2016-06-28 18033, 2016

      • demonimin has quit
      • 2016-06-28 18037, 2016

      • kmiller_
        sounds good!
      • 2016-06-28 18042, 2016

      • ruaok
        k
      • 2016-06-28 18017, 2016

      • QuoraUK_
        Marketer and Recruiter may be difficult, and we may need to discuss them before I make a start
      • 2016-06-28 18044, 2016

      • zas
        kmiller_: i verified stats are stored in influxdb during events, no missing data. If this happens, most graphs would show nothing for missing points
      • 2016-06-28 18055, 2016

      • demonimin joined the channel
      • 2016-06-28 18025, 2016

      • LordSputnik
        QuoraUK_: OK, that sounds like a good plan. When we have the admin system in a PR I'll checkout that branch and give you some feedback
      • 2016-06-28 18047, 2016

      • LordSputnik
        Do you think you can fit in the drag and drop badge selection too this week?
      • 2016-06-28 18013, 2016

      • zas
        http://stats.metabrainz.org/dashboard/db/webserve… shows the drop is simultaneous to the 5 backend servers
      • 2016-06-28 18017, 2016

      • QuoraUK_
        Possibly, I may have to shift a few achievements back a bit
      • 2016-06-28 18044, 2016

      • QuoraUK_
        Perhaps that may be better since we could discuss the "sharing" ones in a bb meeting
      • 2016-06-28 18002, 2016

      • LordSputnik
        QuoraUK_: OK, well do whatever you feel most motivated to do when it's later in the week :)
      • 2016-06-28 18008, 2016

      • ruaok
        is it event o'clock?
      • 2016-06-28 18032, 2016

      • LordSputnik
        The important thing this week is continuing implementing achievements and getting more of your work merged into master, I think
      • 2016-06-28 18054, 2016

      • QuoraUK_
        ok, Ill make that the priority :)
      • 2016-06-28 18057, 2016

      • ruaok
        nice graph zas. I think some of these graphs could use some re-naming. hard to find the juicy stuff someone.
      • 2016-06-28 18020, 2016

      • LordSputnik
        QuoraUK_: OK, anything you wanted to ask?
      • 2016-06-28 18022, 2016

      • zas
        Yes, the whole thing needs a lot of clean up
      • 2016-06-28 18035, 2016

      • QuoraUK_
        also, I ran into some issues with works especially and the elastic search. Is there any documentation of how to set that up for BB?
      • 2016-06-28 18036, 2016

      • zas
        Many graphs are broken, or redundant
      • 2016-06-28 18015, 2016

      • LordSputnik
        QuoraUK_: hmm. Elasticsearch is tricky
      • 2016-06-28 18046, 2016

      • LordSputnik
        Leftmost is more familiar with that
      • 2016-06-28 18048, 2016

      • diana_olhovyk_ joined the channel
      • 2016-06-28 18008, 2016

      • QuoraUK_
        ok, I can ask him tommorow. Im not in a huge rush at the moment
      • 2016-06-28 18011, 2016

      • LordSputnik
        But I think it's a 2 step process - set it up by loading in the "schema?" definition, then reindex the data
      • 2016-06-28 18013, 2016

      • QuoraUK_
        alright, I can have a little go setting it up, if I get stuck Ill give Leftmost a shout :)
      • 2016-06-28 18037, 2016

      • LordSputnik
        Hmm actually I think I re-wrote it - you can just routes/search.js a little bit around line 74 to remove the user requirement then visit /search/reindex if elasticsearch is running
      • 2016-06-28 18055, 2016

      • LordSputnik
        Make sure you don't commit that change though
      • 2016-06-28 18014, 2016

      • zas
        http://stats.metabrainz.org/dashboard/db/all-esta… shows no hole in established connections
      • 2016-06-28 18017, 2016

      • QuoraUK_
        ok, ill give that a shot in the morning :)
      • 2016-06-28 18019, 2016

      • LordSputnik
        (unless you re-write it to check for the admin flag on the user)
      • 2016-06-28 18022, 2016

      • zas
        that interesting
      • 2016-06-28 18006, 2016

      • QuoraUK_
        So with my branching etc with git, do you have a recommended way I should go about it in future?
      • 2016-06-28 18007, 2016

      • zas
        my bad it does
      • 2016-06-28 18037, 2016

      • zas
      • 2016-06-28 18009, 2016

      • LordSputnik
        QuoraUK_: nope, it's mostly a problem because we haven't merged enough
      • 2016-06-28 18011, 2016

      • zas
      • 2016-06-28 18042, 2016

      • LordSputnik
        My only advise would be to make sure a feature is completely finished and production-ready (as much as possible) before starting a new branch
      • 2016-06-28 18048, 2016

      • LordSputnik
        *advice
      • 2016-06-28 18059, 2016

      • regagain has quit
      • 2016-06-28 18014, 2016

      • LordSputnik
        You may have already done that - we'll find out when we merge ;)
      • 2016-06-28 18048, 2016

      • QuoraUK_
        ahh, ok. Ill go through my stuff to check as we go through them. I think there are still little bits that may be needed before we merge, but it shouldn't be too much of a worry
      • 2016-06-28 18056, 2016

      • ruaok
        the peaks on pingu are interesting.
      • 2016-06-28 18007, 2016

      • ruaok
        but I dont see the correlation between the two links you posted.
      • 2016-06-28 18010, 2016

      • zas
        number of est. conns is increasing
      • 2016-06-28 18020, 2016

      • zas
        check time for first hole
      • 2016-06-28 18026, 2016

      • LordSputnik
        QuoraUK_: ok, that sounds good to me :)
      • 2016-06-28 18036, 2016

      • LordSputnik
        Gotta head of now, thanks for your time :)
      • 2016-06-28 18044, 2016

      • QuoraUK_
        alright, laters :)
      • 2016-06-28 18046, 2016

      • zas
        20:50->21:00
      • 2016-06-28 18012, 2016

      • ruaok
        yeah, the begin of the holes is near the end of the pingu spikes.
      • 2016-06-28 18023, 2016

      • zas
        on bert number of est conns is increasing, while number of requests on web traffic is decreasing
      • 2016-06-28 18058, 2016

      • ruaok
        it is a very small number though.
      • 2016-06-28 18000, 2016

      • zas
        in fact web traffic is showing the number of responses, which is much lower, it means the network is slower
      • 2016-06-28 18028, 2016

      • zas
        ~500
      • 2016-06-28 18034, 2016

      • diana_olhovyk_ has quit
      • 2016-06-28 18004, 2016

      • ruaok
        I'm not coming to any conclusions from these graphs. too tired.
      • 2016-06-28 18034, 2016

      • Slurpee joined the channel
      • 2016-06-28 18034, 2016

      • Slurpee has quit
      • 2016-06-28 18034, 2016

      • Slurpee joined the channel
      • 2016-06-28 18026, 2016

      • zas
        well, it is correlated by interfaces packets/s and bytes/s
      • 2016-06-28 18028, 2016

      • zas
        i think we could do something: create a permanent stream of data from ernie to a machine at DWNI, and see if we get the same drop
      • 2016-06-28 18000, 2016

      • zas
        something significant enough
      • 2016-06-28 18008, 2016

      • zas
        1/10 of bert traffic should be enough
      • 2016-06-28 18008, 2016

      • zas
        a rate limited stream (using http://www.ivarch.com/programs/pv.shtml + nc) can do
      • 2016-06-28 18025, 2016

      • zas
        kmiller_: still around ?
      • 2016-06-28 18027, 2016

      • kmiller_
        hey zas/ruoak, quick question - how does failover work between your two gateways? is it a keepalived/vrrp thing?
      • 2016-06-28 18029, 2016

      • kmiller_
        yep
      • 2016-06-28 18058, 2016

      • zas
        it is based on corosync + pacemaker, switching IPs and services
      • 2016-06-28 18010, 2016

      • Freso|koume joined the channel
      • 2016-06-28 18031, 2016

      • zas
        kmiller_: do you have a machine we could stream data to, with constant bandwidth ?
      • 2016-06-28 18037, 2016

      • modwizcode
        Just out of curiosity what have you guys been working on?
      • 2016-06-28 18002, 2016

      • ruaok
        we're debugging a network connectivity issue to the main servers.
      • 2016-06-28 18011, 2016

      • ruaok
        so far its baffled some intelligent people.
      • 2016-06-28 18015, 2016

      • ruaok
        (and zas and me)
      • 2016-06-28 18030, 2016

      • zas
        bert is currently the main gateway, so all traffic is going through
      • 2016-06-28 18036, 2016

      • zas
        ernie is unused
      • 2016-06-28 18039, 2016

      • zas
        i set up something like cat /dev/urandom | pv -L $RATE | nc $TARGET $TPORT and on $TARGET nc -l $TPORT > /dev/null
      • 2016-06-28 18054, 2016

      • zas
        and we see what happens during "events"
      • 2016-06-28 18025, 2016

      • zas
        if bandwidth drops... network level issue, if it doesn't ... well something wrong on our apps or local network
      • 2016-06-28 18037, 2016

      • zas
        kmiller_: ^^
      • 2016-06-28 18017, 2016

      • zas
        the main idea is to have a constant data stream at a known rate, and compare with measurements
      • 2016-06-28 18018, 2016

      • zas
        if the target machine is on DWNI network, it should be stable
      • 2016-06-28 18028, 2016

      • zas
        of course we can do the same the other direction
      • 2016-06-28 18031, 2016

      • kmiller_
        yeah, server on our side can be ops02.dwni.pil.sbp1.digitalwest.net
      • 2016-06-28 18043, 2016

      • kmiller_
        gimme a few seconds to allow stuff through our firewall
      • 2016-06-28 18015, 2016

      • yeeeargh has quit
      • 2016-06-28 18020, 2016

      • zas
        gimme the tcp port when you're done
      • 2016-06-28 18004, 2016

      • zas
        i will create a 100kbytes/s stream from ernie to ops02.dwni.pil.sbp1.digitalwest.net
      • 2016-06-28 18043, 2016

      • zas
        when done, we check if it appears on graphs (it should), and wait next event
      • 2016-06-28 18031, 2016

      • zas
        kmiller_: on your side, set up a screen session with nc -l $PORT|pv -r > /dev/null
      • 2016-06-28 18021, 2016

      • zas
        pv isn't installed by default, http://www.ivarch.com/programs/pv.shtml (packaged for most dists)
      • 2016-06-28 18057, 2016

      • zas
        i set up : cat /dev/urandom | pv -r -L 100k | nc ops02.dwni.pil.sbp1.digitalwest.net $PORT
      • 2016-06-28 18041, 2016

      • modwizcode has left the channel
      • 2016-06-28 18048, 2016

      • modwizcode joined the channel
      • 2016-06-28 18030, 2016

      • kmiller_
        ok zas sorry about the wait, had a phone call had to take - I set it up to port 25000
      • 2016-06-28 18044, 2016

      • ruaok
        ok, zas is off for a minute.
      • 2016-06-28 18022, 2016

      • ruaok
        but I have instructions hang on.
      • 2016-06-28 18032, 2016

      • ruaok
        it worked the first time, but it exited pretty quickly.
      • 2016-06-28 18001, 2016

      • ruaok
        yeah, command looks good here, but stopped very quickly.
      • 2016-06-28 18047, 2016

      • kmiller_
        yeah gimme a sec still think I missed a firewall thing on this server, forgot this one had an acl on it too
      • 2016-06-28 18006, 2016

      • hellska has left the channel
      • 2016-06-28 18007, 2016

      • kmiller_
        ok it should be good to test now ruaok
      • 2016-06-28 18015, 2016

      • ruaok
        k
      • 2016-06-28 18031, 2016

      • ruaok
        same result.
      • 2016-06-28 18035, 2016

      • ruaok
        not working, I guess.
      • 2016-06-28 18044, 2016

      • kmiller_
        I saw it close on this side too
      • 2016-06-28 18054, 2016

      • ruaok
        interesting.
      • 2016-06-28 18001, 2016

      • ruaok
        that means the network stuff is working.
      • 2016-06-28 18036, 2016

      • ruaok
        cat /dev/urandom | pv -r -L 100k | nc ops02.dwni.pil.sbp1.digitalwest.net 25000
      • 2016-06-28 18050, 2016

      • ruaok
        not sure why it terminates so quickly.
      • 2016-06-28 18012, 2016

      • ruaok
        run your side again?
      • 2016-06-28 18017, 2016

      • ruaok
        I'll skip the pv this time.
      • 2016-06-28 18018, 2016

      • kmiller_
        not sure, as soon as you start I see my side just close out and stop listening on port 25k
      • 2016-06-28 18045, 2016

      • ruaok
        try again.
      • 2016-06-28 18002, 2016

      • ruaok
        I've created a 10G file. and I'll try and transfer it.
      • 2016-06-28 18012, 2016

      • kmiller_
        ok ready
      • 2016-06-28 18039, 2016

      • ruaok
        same result each time. :(
      • 2016-06-28 18011, 2016

      • ruaok
        oh wait, I know. this is our own firewall blocking things.
      • 2016-06-28 18013, 2016

      • ruaok
        hang on