I'm sorry we can't make everything perfect at once, ocharles.
2012-09-04 24845, 2012
ocharles
ijabz: I spent time on the ticket in question
2012-09-04 24802, 2012
ruaok
jira tickets, nikki !
2012-09-04 24803, 2012
ocharles
and trying to fit that into our current constraints spent more time than the naive solution
2012-09-04 24824, 2012
reosarevok
We have an Anonymous Jira user, which both nikki and me see no real point for
2012-09-04 24834, 2012
reosarevok
Apart from avoiding us from contacting ticket reporters
2012-09-04 24844, 2012
reosarevok
I mean, someone's apparently even been voting on tickets with it
2012-09-04 24849, 2012
nikki
we also have one weird person who keeps using the anonymous user to watch and vote on tickets
2012-09-04 24858, 2012
warp
we have a large community of people submitting bug reports on jira, so I expect most bugs will be reported if we disable anonymous.
2012-09-04 24859, 2012
reosarevok
(at least one :p)
2012-09-04 24812, 2012
warp
+still
2012-09-04 24839, 2012
warp
so I'm ok with disabling anonymous.
2012-09-04 24847, 2012
ruaok
I'm ok with nuking that account.
2012-09-04 24807, 2012
nikki
I don't like that people have to create a second account to report stuff, but since anonymous tickets mean we can't contact people, it makes those tickets hard to deal with
2012-09-04 24817, 2012
warp
nikki, reosarevok: I assume one of you has enough admin power to disable it?
2012-09-04 24829, 2012
nikki
I see a delete link
2012-09-04 24832, 2012
nikki
not sure if it works :P
2012-09-04 24836, 2012
warp
:D
2012-09-04 24841, 2012
ruaok
one way to find out.
2012-09-04 24852, 2012
ruaok
onward to NES, ocharles!
2012-09-04 24815, 2012
ocharles
ok, just a wrap up topic to see where we stand on this; what to do next, etc
2012-09-04 24838, 2012
ocharles
i personally have been researching how the granularity plays in haskell, and it lends itself to some interesting graph reduction problems, but that's not very relevant :)
2012-09-04 24850, 2012
ocharles
i see you added a few comments in the doc, but are we happy with the questions and answers in that doc?
2012-09-04 24802, 2012
ruaok
yes, for now.
2012-09-04 24819, 2012
ruaok
the main point that I think we need to address now is the client perspective for the tiny granularity.
2012-09-04 24828, 2012
ruaok puts on ijabz' hat for a second.
2012-09-04 24837, 2012
ocharles
hmm, this is just data access though
2012-09-04 24843, 2012
ocharles
it doesn't affect clients at all
2012-09-04 24853, 2012
ruaok
I think it does.
2012-09-04 24858, 2012
warp
ruaok: this is not the webservice. so you're have ijabz search server hat, not the jaikoz hat.
2012-09-04 24859, 2012
ijabz
i think it does in an insidious way
2012-09-04 24801, 2012
ocharles
only if we wrote /ws/3 at the same time
2012-09-04 24804, 2012
warp
yay english
2012-09-04 24810, 2012
ruaok
for me, ws/3 might just be a thin layer around the data access stuff.
2012-09-04 24825, 2012
ocharles
i still worry about doing data access/nes and /ws/3 at once
2012-09-04 24834, 2012
warp sides with charles.
2012-09-04 24841, 2012
ijabz
and the data access layer will push how the ws/3 is done
2012-09-04 24846, 2012
ocharles
i don't think
2012-09-04 24847, 2012
ruaok
I'm not suggesting that we do that.
2012-09-04 24847, 2012
ocharles
so
2012-09-04 24800, 2012
ocharles
it's so granular you can easily compass into something much monolithic, if you want
2012-09-04 24812, 2012
ocharles
but you can't go backwards, hence me advocating granularity where possible
2012-09-04 24817, 2012
ocharles
ruaok: ok, cool
2012-09-04 24827, 2012
ruaok
but I am suggesting that we think about it now.
2012-09-04 24849, 2012
ruaok
ocharles: I haven't had a chance to look at the doc since I returned.
2012-09-04 24811, 2012
ianmcorvidae
so working on it, but only as far as "let's try to not make the data layer break immediately when we try to do /ws/3" ?
2012-09-04 24815, 2012
ruaok
did you have any comments on my "make as many requests as you want, wait for results from the server async" approach?
2012-09-04 24836, 2012
ruaok
ianmcorvidae: sure, that is one way of looking at it.
2012-09-04 24842, 2012
ocharles
that's actually part of what i've been working on currently, though not quite with that question in mind
2012-09-04 24808, 2012
ocharles
I have a prototype where you can describe a graph of queries, and hand it to an engine which figures out what can be run in parallel, what can be aggregated, and so on
2012-09-04 24817, 2012
ocharles
this was mostly for fun, but maybe it does actually relate to that
2012-09-04 24825, 2012
ruaok
ocharles: is it async?
2012-09-04 24858, 2012
ocharles
not designed with that in mind yet
2012-09-04 24809, 2012
ocharles
because it wasn't designed for a client, but more controllers before rendering templates
2012-09-04 24815, 2012
ruaok
ah.
2012-09-04 24823, 2012
ruaok
can you think about it in the context of clients?
2012-09-04 24832, 2012
kepstin-work
async doesn't make sense for a web service, because there's no way for a server to initate a connection to a client.
2012-09-04 24843, 2012
ruaok
I would love to bring the granularity to all levels of data access.
2012-09-04 24802, 2012
ocharles
i would, but not now. I'd like to just have /ws/2 talk to this and act exactly the same
2012-09-04 24816, 2012
warp
kepstin-work: it could push results back on the existing channel when they become available.
2012-09-04 24825, 2012
ocharles
kepstin-work: web sockets :)
2012-09-04 24849, 2012
kepstin-work
warp: I suppose, but then you need to add more complicated framing and request ids to id
2012-09-04 24858, 2012
ocharles
ruaok: really to think about this from a client side does mean I need to think about /ws/3 a lot more
2012-09-04 24809, 2012
ruaok
ocharles: I would like that.
2012-09-04 24811, 2012
ocharles
i can do that if you want me to, but we're juggling a lot of abstractions around as it is
2012-09-04 24840, 2012
ruaok
we didn't do enough future proofing when we did ws/2.
2012-09-04 24844, 2012
ijabz
Be nice if you could write down clearly the advantages of granularity, as its still a vague concept to me
2012-09-04 24848, 2012
ruaok
and this data abstraction will impact a lot of things.
2012-09-04 24804, 2012
ruaok
so, it would be nice if we can think more down the road to consider the changes that might come.
2012-09-04 24845, 2012
ocharles
ok, so this comes back to the data access problem - what questions am I answering? :)
2012-09-04 24847, 2012
ruaok
ijabz: I think you understand (and hate it) quite clearly. :)
2012-09-04 24810, 2012
ocharles
do we want a schema for /ws/3? a map of end points?
2012-09-04 24830, 2012
ijabz
What i don't understand is why everyone seems to think its going to solve a load of problems, I see it only solving one
2012-09-04 24823, 2012
ruaok
ocharles: the question to answer is how a client can live with a system where they need to make a lot of more calls to fetch the data.
2012-09-04 24849, 2012
ruaok
because inside of the current ws/2 model that concept is pure insanity.
2012-09-04 24854, 2012
ocharles
ruaok: there are various answers, but they all depend on an implementation
2012-09-04 24808, 2012
ruaok
of course they do.
2012-09-04 24813, 2012
ocharles
ie: make parallel requests and have a good library to glue it together, remove/reduce the rate limit, use web sockets to stream stuff
2012-09-04 24821, 2012
ruaok
then lets outline the various answers so we can consider them.
2012-09-04 24838, 2012
ocharles
I mostly like the idea of submitting a query tree to the server and having it inflate it at the moment
2012-09-04 24843, 2012
ocharles
but that's not very RESTy
2012-09-04 24850, 2012
ijabz
Surely making parallel requests is circumventing rate limiting ectera
2012-09-04 24858, 2012
ruaok
and maybe REST needs to go. I don't know.
2012-09-04 24817, 2012
kepstin-work
well, a 1/s ratelimit makes no sense at all with a granular webservice
2012-09-04 24820, 2012
nikki
note: I haven't deleted the user, but I have changed the password and removed the references to it on the login screen (apparently deleting it will make the comments attached to it a bit weird and then someone could just go and recreate it)
2012-09-04 24826, 2012
ianmcorvidae
we aren't very RESTful right now, in fairness
2012-09-04 24837, 2012
kepstin-work
right now we have apps all making /massive/ queries with lots of inc= to workaround the ratelimit
2012-09-04 24846, 2012
ruaok
ianmcorvidae: and I think us trying to be restful is making our lives harder.
2012-09-04 24802, 2012
ijabz
I wish I wash't the only paying user seemingly bothered to get involved in this discussion
2012-09-04 24812, 2012
ruaok
ijabz: you're not.
2012-09-04 24822, 2012
ruaok
I personally am representing your side in this.
2012-09-04 24837, 2012
ianmcorvidae
I think we need to either be more strictly REST about it or drop it entirely, yes
2012-09-04 24852, 2012
ruaok
ianmcorvidae: agreed.
2012-09-04 24854, 2012
ianmcorvidae
I think "trying to be restful" isn't really the problem so much as "not really doing REST, but some weird perversion of it"
2012-09-04 24801, 2012
warp
haha
2012-09-04 24812, 2012
ruaok
we're struggling to fit our data into the REST model and it doesn't work well like that.
2012-09-04 24821, 2012
ocharles
i don't think we are
2012-09-04 24821, 2012
ruaok
we're mapping documents to a graph.
2012-09-04 24829, 2012
ruaok
and that is really ugly.
2012-09-04 24832, 2012
ocharles
because we never designed how it would look with REST
2012-09-04 24836, 2012
warp
a more granual model would make it fit better with REST though
2012-09-04 24840, 2012
warp
granular
2012-09-04 24843, 2012
ruaok
warp: it would.
2012-09-04 24848, 2012
ianmcorvidae
yeah, granularity is a lot more RESTful
2012-09-04 24854, 2012
kepstin-work
well, we just need to make the stuff returned from a request include hyperlinks to linked data in the graph, rather than including the data itself
2012-09-04 24800, 2012
ruaok
but the HTTP RESTful way of doing will drive clients insane.
2012-09-04 24805, 2012
ruaok
(in the membrane)
2012-09-04 24807, 2012
ianmcorvidae
and some of the "but we need so many different URLs!" problem gets solved by being RESTful by using hyperlinks
2012-09-04 24826, 2012
ijabz_ joined the channel
2012-09-04 24828, 2012
ianmcorvidae
obviously the ratelimit needs help, but I suspect that's true *whatever* we do
2012-09-04 24844, 2012
ijabz_
bad time to disconnect, Why don't we we create a document that clearly lists all the problems with ws/2 and then see what solutions we can come up with
2012-09-04 24847, 2012
ruaok
so, really, I think we need to consider all of our options on this front before we proceed with data access layer work.
2012-09-04 24859, 2012
kepstin-work
ideally, using smaller simpler queries that are easier to cache would allow the ratelimit to be increased.
2012-09-04 24812, 2012
ruaok
kepstin-work: sure.
2012-09-04 24817, 2012
warp
the ratelimit just needs to be tweaked so a fetching full release in picard/jaikoz doesn't take longer than it does for /ws/2. so the amount of data recieved / time should stay constant.
2012-09-04 24824, 2012
ruaok
but the overhead of making an HTTP call becomes very great at that point.
2012-09-04 24826, 2012
ianmcorvidae
ijabz_: that's the idea with asking to list options so we can consider them, I think -- but agreed
2012-09-04 24832, 2012
ruaok
not the best use of resources/time
2012-09-04 24846, 2012
ijabz_
ianmcorvidae :might have missed that offline
2012-09-04 24809, 2012
ocharles
so for the next week, what am I doing?
2012-09-04 24811, 2012
ianmcorvidae
ijabz_: fair enough -- ruaok is wanting a document that explores possible solutions; I think what you're saying needs to be part of that :)
2012-09-04 24815, 2012
ruaok
warp: i disagree. HTTP call overhead is a problem.
2012-09-04 24818, 2012
ocharles
i'm hearing a lot of general worries here, but nothing I can specifically focus on
2012-09-04 24833, 2012
warp
ruaok: ok, a valid criticsm :)
2012-09-04 24856, 2012
ruaok
ok, I can take a stab at creating this document.
2012-09-04 24806, 2012
ocharles
!
2012-09-04 24808, 2012
ruaok
listing existing issues so we can enumerate them.
2012-09-04 24811, 2012
ocharles
please share it around again, it worked great last time
2012-09-04 24818, 2012
ruaok
ok, will do.
2012-09-04 24826, 2012
ocharles
ijabz added a lot of great questions
2012-09-04 24838, 2012
ruaok
shall we triage some bugs today?
2012-09-04 24846, 2012
warp
sure
2012-09-04 24847, 2012
ianmcorvidae
I can try to be the super-pedantic-about-REST voice for you :)
2012-09-04 24857, 2012
ruaok
ianmcorvidae: please do that!
2012-09-04 24858, 2012
ianmcorvidae really needs to read Fielding's dissertation at some point
2012-09-04 24801, 2012
ocharles
ianmcorvidae: i have that base covered too :)
2012-09-04 24804, 2012
ocharles
ha, already read it
2012-09-04 24810, 2012
warp
lol, ocharles wins.
2012-09-04 24814, 2012
ianmcorvidae
hah, yes
2012-09-04 24816, 2012
ruaok
ocharles: bug triage?
2012-09-04 24821, 2012
ianmcorvidae
I mostly volunteered to make myself read the damn thing :)
2012-09-04 24822, 2012
ocharles
i'm not sure how much time i have
2012-09-04 24826, 2012
warp
I just read the o'reilly books on the topic.
2012-09-04 24832, 2012
ianmcorvidae
I can triage for a bit of others want to
2012-09-04 24849, 2012
ocharles
you're welcome to go ahead and i'll leave when i'm not free
