one thing I want to do is setup a new jenkins build that uses some sort of "pre-update" database, does the update, then runs all tests. and that will happen on every commit to the schema change branch
2012-04-17 10809, 2012
ocharles
so it's basically doing exactly what we'll do to deploy the update, to make sure that at least doesn't break
2012-04-17 10814, 2012
ruaok
+100
2012-04-17 10817, 2012
warp
ocharles: I need to talk to you about MBS-842 before I ship that to that jenkins then ;)
2012-04-17 10801, 2012
ruaok wonders if this topic is done
2012-04-17 10811, 2012
warp
ruaok: I think so, I'm happy to just ship to test.
2012-04-17 10819, 2012
ruaok
ok
2012-04-17 10825, 2012
ruaok
ruaok has changed the topic to: agenda: daily code review (reotab/ian/ocharles?), beta spec (r/i/n), web interface ratelimit (reotab), robots.txt (reotab/nikki), post-schema-change schedule (reotab)
2012-04-17 10825, 2012
ocharles
that and jenkins should give us a good start
2012-04-17 10829, 2012
warp
ocharles: do you have time later this week ot help me with the db update for MBS-842?
2012-04-17 10832, 2012
ruaok
reosarevok: you're up!
2012-04-17 10841, 2012
ocharles
warp: friday is best
2012-04-17 10850, 2012
reosarevok
So
2012-04-17 10854, 2012
warp
ocharles: ok, I'll poke you on friday when I get to the office.
2012-04-17 10854, 2012
ocharles
but i can give you half an hour tomorrow/thurs
2012-04-17 10816, 2012
reosarevok
We were debating beta the other day, and while nikki has her spec thingy now
2012-04-17 10833, 2012
reosarevok
One big problem we found was the fact that most code reviewing is done the last day or two
2012-04-17 10843, 2012
reosarevok
(so there is no time to actually test stuff)
2012-04-17 10849, 2012
ruaok
ah
2012-04-17 10850, 2012
nikki
and things are merged then immediately released
2012-04-17 10806, 2012
reosarevok
While the beta spec should avoid that
2012-04-17 10809, 2012
ruaok
I agree that this is less than ideal.
2012-04-17 10824, 2012
reosarevok
It still would make sense to have an allocated daily time for code review
2012-04-17 10830, 2012
ocharles
well if there are provisions for this in the beta spec, i don't really see it being a problem
2012-04-17 10843, 2012
ocharles
i don't like that idea ,some days i'm really not in the right mindset to do a review
2012-04-17 10851, 2012
ocharles
it just won't get the attention it deserves
2012-04-17 10852, 2012
warp
I think we just need to mandate a certain amount of time for testing, before such code can be released.
2012-04-17 10859, 2012
ruaok
ocharles: suggest some other method that gives beta more post-review time.
2012-04-17 10859, 2012
reosarevok
warp: that's part of the spec
2012-04-17 10811, 2012
ocharles
an explicit "ship it" from testers
2012-04-17 10817, 2012
reosarevok
So that's not a problem
2012-04-17 10822, 2012
warp
daily code review doesn't really make sense to me, that just breaks up my working day for no good reason.
2012-04-17 10832, 2012
ocharles
the natural order is: hack, code review, approval, user testing, approval, shipping
2012-04-17 10857, 2012
reosarevok
warp: not if it's the last or first thing you do on a day
2012-04-17 10817, 2012
ruaok
reosarevok: that key here is that you cant force the devs to do this daily.
2012-04-17 10820, 2012
ruaok
not going to work.
2012-04-17 10843, 2012
ocharles
and when the review is done is not really the problem, the problem now is there is no time for testing, so i'd rather solve that problem
2012-04-17 10846, 2012
reosarevok
ruaok: I don't trust that code reviewing everything in the last moment will lead to good reviewing, tbh
2012-04-17 10846, 2012
warp
but it's easy to force not shipping something until it has been on beta long enough.
2012-04-17 10846, 2012
ruaok
so, the spec has some time that post-review things must be available.
2012-04-17 10854, 2012
reosarevok
ruaok: it does, yes
2012-04-17 10856, 2012
ruaok
but, how to we ensure that we do this right?
2012-04-17 10824, 2012
ruaok
reosarevok: then lets fix the last minute part.
2012-04-17 10834, 2012
nikki
is it possible to un-ship-it on code review?
2012-04-17 10848, 2012
nikki
say I think it's fine and say ship-it, but later find a bug
2012-04-17 10857, 2012
ocharles
well, the thing is now, shipping goes to master and beta
2012-04-17 10802, 2012
ocharles
if it only goes to beta, then it's not been released
2012-04-17 10803, 2012
ruaok
what if we say that a code review posted on CR needs to be reviewed within a week's time?
2012-04-17 10810, 2012
ocharles
but we've had this problem before that people didn't know what they should be testing
2012-04-17 10843, 2012
ocharles
ruaok: and if it isn't?
2012-04-17 10804, 2012
ruaok
I come hunt you down and take away your beer? :)
2012-04-17 10805, 2012
reosarevok
ruaok: the beta spec says all code must be reviewed and merged into beta 7 days before release
2012-04-17 10807, 2012
warp
I'd say we should only ship to beta, and when testers have given approal actually ship it (to master).
2012-04-17 10818, 2012
reosarevok
If it is not, it can't be shipped to beta until after release
2012-04-17 10830, 2012
ocharles
ruaok: i do my reviews :)
2012-04-17 10835, 2012
ruaok
warp: that makes sense to me.
2012-04-17 10853, 2012
ocharles
personally, i'd rather be explicit and require signoff/approval rather than wait some period of time
2012-04-17 10855, 2012
ruaok
but ideally, devs should not be able to hold up the reviews.
2012-04-17 10805, 2012
warp
ocharles: I agree with that.
2012-04-17 10809, 2012
reosarevok
That doesn't work
2012-04-17 10820, 2012
ocharles
pour quoi?
2012-04-17 10821, 2012
reosarevok
Since the important thing is whether the whole of the stuff works
2012-04-17 10831, 2012
ocharles
oh, the entire integration
2012-04-17 10836, 2012
ocharles
well in that case, we freeze the release a week before release
2012-04-17 10841, 2012
reosarevok
Yeah, that's the point
2012-04-17 10852, 2012
warp
if there's a freeze there's going to be last minute reviewing again.
2012-04-17 10853, 2012
warp
:)
2012-04-17 10806, 2012
reosarevok
warp: that's why there should be daily reviewing, but you two object to it :p
2012-04-17 10812, 2012
warp
(ofcourse, last minute reviewing + 1 week of testing, so it's not as bad as it is now)
2012-04-17 10815, 2012
ruaok
round and round we go!
2012-04-17 10824, 2012
reosarevok
But yeah, it's a definite improvement
2012-04-17 10837, 2012
reosarevok
(since broken stuff should be found before it hits everyone)
2012-04-17 10846, 2012
ocharles
i don't think most regressions are cross cutting though
2012-04-17 10854, 2012
reosarevok
nikki, do you have that flowchart around?
2012-04-17 10856, 2012
nikki
warp: even if it's last minute reviewing, it means there's a week for testing instead of no time for testing
2012-04-17 10856, 2012
ocharles
most regressions can be traced back to a single commit, on a single branch
2012-04-17 10806, 2012
ocharles
but the freeze sounds good to me
2012-04-17 10810, 2012
reosarevok
ocharles: sure, but the job of testers is to test that all works
which is an implementation detail to a degree, but it might help me and warp
2012-04-17 10818, 2012
ruaok
did we already slide into the next topic?
2012-04-17 10820, 2012
ruaok
beta spec?
2012-04-17 10821, 2012
reosarevok
For now, yes. I'll bring the code review issue again if it seems like reviews are rushed and lots of bugs appear :p
2012-04-17 10837, 2012
ocharles
it has provisions for freezing releases and hotfixing them (which is what we're essentially doing in the beta phase)
2012-04-17 10840, 2012
warp
ruaok: they're related topics.
2012-04-17 10840, 2012
reosarevok
I'm willing to try with just the freeze for now
2012-04-17 10847, 2012
ocharles
i think 2 week releases might be too fast for this though now too
2012-04-17 10853, 2012
reosarevok
ocharles: why?
2012-04-17 10801, 2012
ocharles
that's 3 days of my time per release, basically
2012-04-17 10808, 2012
reosarevok
The freeze days go on the next release
2012-04-17 10815, 2012
nikki
other than the freezing bit, the other important bit is checking for beta.mb bugs. we released the last release with an open bug for beta.mb which was then rereported by two more people :/
2012-04-17 10817, 2012
reosarevok
(except if there are too many bugs to fix)
2012-04-17 10851, 2012
warp
nikki: I think that's why ocharles wants explicit approvals for beta.
2012-04-17 10856, 2012
warp
or for specific tickets.
2012-04-17 10813, 2012
ocharles
right
2012-04-17 10814, 2012
reosarevok
warp: for specific bugs found in beta, yes
2012-04-17 10829, 2012
reosarevok
For specific commits / tickets which the release is trying to fix, no
2012-04-17 10832, 2012
reosarevok
Different things
2012-04-17 10832, 2012
warp
nikki: so we need a clear way for you testers to communicate to us developers that we broke the frozen beta, and shouldn't ship it.
2012-04-17 10848, 2012
ocharles
it'd be nice to have: "hack, review, user test, ship" for 2 weeks, then a week freeze for complete integration testing
2012-04-17 10855, 2012
nikki
I've been prefixing all tickets that are specifically for beta with "beta.mb:"
2012-04-17 10855, 2012
reosarevok
Adding a beta version, which you will check every morning? :)
2012-04-17 10803, 2012
ocharles
however...
2012-04-17 10816, 2012
ocharles
making testers test one ticket pretty much requires separate instances per branch, which is not great
2012-04-17 10822, 2012
hawke_2
I think we still need a clear communication of what needs to be tested/what’s new in beta, or did I miss that somewhere?
2012-04-17 10825, 2012
djce joined the channel
2012-04-17 10831, 2012
warp
nikki: but I look at tickets when I finished a ticket and need more work, not when we're doing a release.
2012-04-17 10846, 2012
reosarevok
warp: you should look if there are beta tickets daily
2012-04-17 10851, 2012
reosarevok
If not, keep working on future releases
2012-04-17 10853, 2012
warp
reosarevok: again with the daly :)
2012-04-17 10854, 2012
reosarevok
If yes, fix
2012-04-17 10804, 2012
reosarevok
Again because you'll have 7 days to fix them :p
2012-04-17 10810, 2012
nikki
I just feel like I'm expected to report things in 100 different places
2012-04-17 10823, 2012
nikki
I need to make a ticket, comment on code review, prod people on irc...
2012-04-17 10830, 2012
nikki
prod them again on irc...
2012-04-17 10831, 2012
ocharles
you shouldn't need to prod in irc
2012-04-17 10840, 2012
reosarevok
ocharles: I think we all agree with that
2012-04-17 10845, 2012
adhawkins joined the channel
2012-04-17 10847, 2012
reosarevok
Now, how do we change it
2012-04-17 10848, 2012
reosarevok
?
2012-04-17 10802, 2012
ocharles
public filters can be good
2012-04-17 10827, 2012
ocharles
but i'm still unclear of exactly what's wanted and what we're trying to solve. is this spec thing available for reading anywhere?
2012-04-17 10835, 2012
ocharles
I need to be on the same page as testers, and understand how it is for them right now
2012-04-17 10842, 2012
ocharles
i have an idea, but i need to be certain i really understand it all
2012-04-17 10805, 2012
warp
nikki: yes, I understand that is frustrating.
2012-04-17 10809, 2012
reosarevok
nikki, link please
2012-04-17 10810, 2012
ocharles
i'm not trying to make people jump through hoops either, just trying to avoid us going round and round like this :)
2012-04-17 10815, 2012
ocharles
reosarevok: well i can't read it now
2012-04-17 10826, 2012
nikki
the flowchart that I already linked is all we have now
2012-04-17 10833, 2012
ocharles
if it can be posted to mb-devel or something, that'd be great
2012-04-17 10838, 2012
nikki
everything we came up with was just a really wordy version of the flowchart
2012-04-17 10842, 2012
ocharles
ok, the flowchart is a good start
2012-04-17 10842, 2012
reosarevok
It shouldn't really need more than what is on that flowchart, yeah
2012-04-17 10800, 2012
nikki
so if there's something you want other than that, you should be more specific about what you want :P
2012-04-17 10844, 2012
ocharles
well, guess i'm not sure what i'm unclear about then
2012-04-17 10805, 2012
ocharles
maybe we just continue with the week freeze thing, and problems in that period get their own ticket, assigned to the upcoming version?
2012-04-17 10813, 2012
ocharles
cause that version should have no open tickets at the point of freeze
2012-04-17 10825, 2012
ocharles
so it's easy to see after the 7 day freeze period if anything is still to be done
2012-04-17 10835, 2012
reosarevok
As long as you look at that version's tickets often to see if stuff has been added, yes
2012-04-17 10848, 2012
ocharles
and what happens if we do have problems in that period? do we have another 7 days, or do we discuss it when it happens?
