I would mark those not rechable first and only delete them in the second run
nikki_
yeah, same
ruaok
nikki_: simpl, base it on time.
nikki_
so you'd need a lastchecked thingy somewhere
ruaok
nope.
nikki_
hmm?
ruaok
time() mod (60 * 60 * 24)
gives you day you're in.
nikki_ blinks
mod that by 30 and you get a date 0 - 29
on the first day, do the first 30th of the URLs
on the second day, do the second 30th of the URLs
your script, once invoked per day, would do the right think.
some border conditions could be lost. eh.
nikki_ will take your word for it
you'd get the next month.
ruaok hates keeping state of ANYTHING
nikki_
why?
Shepard
oh and of course wp, discogs, ... urls can be excluded
ruaok
its a pain in the arse.
yllona is listening to Frankie And Johnny by Brook Benton from Best of Brook Benton [Mercury] (1961) (0:13 / 2:28)
just think of you having to rmeember things.
how oftern do you forget?
wolfsong13 is listening to Leela James - Soul Food [from "A Change Is Gonna Come" (2005)]
keeping track of state is always a pain
that's why HTTP is a stateless protocol -- its easy and simple.
yllona switches up for wolf
yllona is listening to A Change Is Gonna Come by Sam Cooke from Ain't That Good News (1964) (0:06 / 3:14)
keeping track of state in HTTP is cookies and all the compllicated mess.
Shepard thinks that way we would drop thousands of wikipedia urls just because there server is down when the script is running ;)
Shepard
*their
wolfsong13 wanders off to eat
ruaok
relax, we're not just going to delete them.
Shepard
but also bombing their server? ;)
Nyght
yeah, that's sorta why I personally think it's best to have a person checking them
one was a really slow load, but it was present.. things like that.
ruaok
Shepard: > wait a second, do next one.
common web practice.
one hit per second is not bombing
Shepard thinks it doesn't work without a bit field keeping state
Shepard
no you misunderstand
ruaok
state of a url?
Shepard
the bombing was just a joke
yes
Nyght
and I honestly have no idea how the program is "hitting" but it seems like it's about every fifteen seconds, or so.. I hope it's not making a load on the server.
ruaok
write a report: todays URLs that didn't answer [ Delete ]
Shepard
what do you want to do when the script checks a url and the server is not reachable?
hm
well I think a script could delete them automatically
ruaok
it could.
and the script could be written to keep some state.
Nyght
I would imagine you would want to spot check the list at least.
ruaok
if the state happens to get lost, its start from the beginning.
Shepard
why should it get lost?
do my albums in the db get lost, too?
ruaok
the feewer state files we need to keep track of, the fewer things we can screw up.
thus is something recovers gracefully, we don't have to really care.
thus leaving more time for the things that we do need to care about.
Shepard
if (however that should happen) the state gets lost, that's not critical
ruaok
ding
ruaok can live with that
nikki_: are you done yet? :-)
all this keeps distrating me from 1090
Shepard
pasta needs its time...
ruaok
as do regexps
yllona
shep: what type of pasta?
Shepard
I don't know, ask nikki
ruaok wasn't asking about the pasta
I know
ruaok reminisces of the last pasta he ate. on a glacier a week ago
nikki_ hat keinen hunger mehr!
ruaok
yay
nikki_
yllona: farfalle
well, it was
now it's pasta-mush in my stomach :D
Shepard
du meinst, du bist satt?
nikki_
ja!
yllona
farfalle with waht type of sauce?
nikki_
tomato
Shepard
keinen hunger mehr haben would mean you don't want to eat it :)
yllona
nikki: tomato and ?
yllona apologizes for the specificity ;)
nikki_ wouldn't want to eat regurgitated pasta...
nikki_
er
sun-dried tomato and herby stuff and whatever else!
but these herbs grow wild and/or natuarally in any "southern clime"
most of these herbs are everywhere in cali.. same for spain or italy. but the packaged dried herbs are really convenient. super tasty and work well for veggie, meat, pulry or fish.
*poultry or fish.
so yes, shep an essential for the ktichen :)
Shepard
yup
yllona
good choice
yllona remembers when shep was shyly asking questions about cooking
Shepard
I did?
yllona is amazed how quickly shep has learned
oh no, those we have used here for years ;) I'm still not a big cook
yllona
shep: yes, ssenrupus was cooking a pie... it was a while gao
Shepard
oh yeah, I recall
yllona wants everyone to cook (well) it's so relaxing (and healthy). a major creative out let
nikki_
it's not relaxing or healthy when you're like me :/
I made very bad cake!
the first was ok, but it got worse
yllona
nikki_: you're a special case :)
nikki_
I guess so!
I can boil water and make pasta!
but pretty much everything I try and *cook* is a disaster
yllona
shep: all i'm gonna say on this matter -- all of the men i know are excellent cooks. mostly began for purely economic --- eating out is too costly
nikki_
apparently my supposed father was a good cook
I didn't inherit his skills, it seems
Shepard
or it is cheap at the uni canteen but doesn't taste at all