/2007/12/20/spiders-how-often-is-too-often/ Tips and tricks to get technology to work for you, by Michael Boyd Clark Sat, 20 Feb 2016 14:14:51 +0000 hourly 1 /2007/12/20/spiders-how-often-is-too-often/comment-page-1/#comment-14612 Thu, 24 Jan 2008 00:39:12 +0000 http://www.planetmike.com/journal/2007/12/20/spiders-how-often-is-too-often/#comment-14612 A site like CNN or Foxnews has news, which will change frequently. But 99% of the web sites out there don’t need to be spidered any more frequently than once a day.

]]> /2007/12/20/spiders-how-often-is-too-often/comment-page-1/#comment-14609 Tue, 22 Jan 2008 17:21:15 +0000 http://www.planetmike.com/journal/2007/12/20/spiders-how-often-is-too-often/#comment-14609 Could you perhaps explain why every 30 min is to often for spiders to come calling.Does that take up bandwidth or something.I know most people do not post every 30 minutes but would have thought perhaps 6 times in a 24 hour period would be okay?

]]> /2007/12/20/spiders-how-often-is-too-often/comment-page-1/#comment-14570 Tue, 08 Jan 2008 11:41:11 +0000 http://www.planetmike.com/journal/2007/12/20/spiders-how-often-is-too-often/#comment-14570 Well, tomorrow when you have a couple more mikes writing articles here, you will need it to come every 30 minutes. I would imagine cnn or foxnews get spired every 5 minutes. 😉

]]>