so we should perform this one step on line53, and then generate highlevel data for this model (#88)
and we will then have two pieces of information: the ground truth in the dataset editor; and the high-level estimates from running our model
I have code which will give an overall accuracy of the model based on these two bits of information
if you got it to this stage, it'd be a full dataset/evaluation cycle
Gentlecat
alastairp: do we need to generate history file during every evaluation?
and where should we store it?
ariscop joined the channel
alastairp
yeah, I'm not sure at what stage we should generate it
if we wait, then the evaluation / parameter selection will happen faster
since there is one less step
so we could not do it during every evaluation; only when someone is happy with their accuracy
you could just store it in a data directory
data/models/uuid/date/file.history
Gentlecat
ok
uuid of evaluation job?
alastairp
ah, is there uuid for dataset and for evaluation?
(I was thinking of dataset)
but if there is one for evaluation, use that. you don't need a date then (since you can get that data from the database)
johtso joined the channel
D4RK-PH0ENiX joined the channel
CatQuest
uhm. am I doing something wrong? release editor doesn't go through. any change and clicking "enter edit" and it spins for a while and then goes akc ot normal but no submitting is done? tired both beta and regular
Gentlecat
alastairp: do you know how long this step is?
I think having that running separately makes things unnecessarily more complicated
we probably need another queue for generating history files
evaluation script will need significant changes if we are going to reuse it for this task
alastairp
It's basically the same length as a parameter permutation
I don't think I even changed anything. it's the same VM, same dataset
alastairp
hmm
did you update gaia?
because I made a change recently to the python bindings
Gentlecat
I don't think so
I'll try to set it up again
alastairp
I'll try a few demos - we should just be able to load up the python interface in a shell to see where the problem is
I'll revert my change and see if that was the problem
ahhhh
Gentlecat: double-check that your class names are strings, not uuid objects
in fact, strings. not anything else
you should be able to reproduce it in a python shell with p = gaia2.Point(); p[9] = "foo"
whereas p["string"] = "foo" will give a more normal error
Gentlecat
let me check
I think it's a unicode string
this raises the same exception: p[unicode("hello")] = "test"
alastairp
ah!
there we go
1) open ticket in gaia
2) convert to str
Gentlecat
alastairp: wait, it's different function
these examples reference Point_setValue function
alastairp
I just changed your create_groundtruth method
line 91
str(cls["name"])
Gentlecat
I mean Point_setLabel
alastairp
fixed
Gentlecat
maybe it's better to use .encode("UTF-8")?
alastairp
yeah, that should work just as well
do we have any restrictions on label names?
perhaps we should actually limit them to [a-zA-Z0-9]+
Gentlecat
no, not really
yeah, that's one way to do it, but seems hacky
alastairp
I don't mind if you str() or .encode
oh - I bet it worked for you before you set the default encoding of the database connection, right? :)
Gentlecat
right!
that's what it was :)
alastairp
OK, cool. I was trying to fix this issue before I gave you some more feedback on the website interface
now that I've succesfully done a model, I'll do this for you too
Gentlecat
feedback \o/
ruchiranga joined the channel
ruchiranga
how should a query to the web service be written with something like 7" vinyl ? for example /ws/2/release?fmt=json&query=format:(7" vinyl)&offset=0 does not work because of the quotation mark
alastairp
can you urlencode it? " is %22
Gentlecat: did you open a ticket in gaia for non-string values?
Gentlecat
not yet, it's on the list
alastairp
ok, cool
ruchiranga
alastairp: tried it but the same error comes
{"error":"Search server could not complete query: Bad request"}