‘NASA Search 1.0’ ??? Something Google should worry about ???

Having written my own WordPress logging / statistics plug-in over the weekend – which still in prototype, consider it a ‘coming soon’ – I have started to notice more and more peculiar User-Agents visiting my blog.
I quite like to keep an eye on what spiders / bots visit my sites, how often they return and try to infer something about how they were designed by watching them visit.
I was surprised recently to see that the big three ( Yahoo!, MSN & Google ) actually pull RSS feeds as well as HTML pages – of course this makes sense from a efficiency & bandwidth side of things, the RSS feed is the interesting stuff already stripped out.
Today’s one is a real winner though, coming from the following net block and advertising itself as “NASA Search 1.0”.
[code]Comcast Cable Communications, Inc. NJ-SOUTH-4 (NET-68-46-128-0-1)
68.46.128.0 – 68.46.191.255[/code]
The bot / spider crawled my entire site within a few minutes, starting from my ‘changes-in-wordpress-152’ post and was completely oblivious to my robots.txt (it didn’t even request it).
Also, it appeared to be quite a primitive HTTP client, providing no referrer information or any of the usual headers “Connection: close”, “Accept: */*” even though it was sending a “HTTP/1.1” request. Surprisingly though it did persist a session cookie for the duration of its visit.
I Google’d for the phrase “NASA Search 1.0” and only seemed to find results where auto-generated-stats pages list visiting User-Agents.
It would be quite interesting (and maybe even fun – in a very geeky way) to write a WordPress plug-in that watches for these peculiar bots and pings their details to a centralised stats database – forming a sort of spider-spider.
Anyway, I will be keeping a keen eye out for the return of “NASA Search 1.0” … Could it be the next greatest NASA funded project? Or is it just some smart a** that has figured out how to change the User-Agent string in his favourite spider/bot.
Stay tuned!