Resolution to "Server too Busy" Messages

[GameMaster]

Bond, is it possible you could alter any part of the url?

Like instead of ".net/forums" you could change it to ".net/forum" so that would most likely confuse all our bot visitors.

And I still think you should require people to register before they can view.

[Bond]

Originally Posted by GameMaster

Bond, is it possible you could alter any part of the url?

Like instead of ".net/forums" you could change it to ".net/forum" so that would most likely confuse all our bot visitors.

And I still think you should require people to register before they can view.

That probably is possible, although every single vB forum I have seen has "/forums/" Even if I was able to do that all of our pages would get screwed up and it would probably create more trouble then it is worth.

I might require people to register to view the board, but that didn't stop the bots. I even tried closing the forum overnight and it didn't stop them.

[GameMaster]

When you change a major part of the address like that doesn't it automatically change the part of the address in all the pages?

Edit: I just checked the "Who's Online" area and it listed almost all the guests as Google Spiders. Do you think maybe you could try contacting Google and ask them to stop scanning our site.

[Bond]

All of the addresses may change automatically, but I'm not sure and I wouldn't want to take that risk.

I've done a few hours of research (ugh) on how to get rid of the spider bots, and I found how to create this file that the bots should check to see what folders they are allowed to index and whether or not you even want them indexing your website. Obviously the google spider isn't checking the file, which it should every 48 hours or so. I'm trying a lot of other different things also, so hopefully something will work.

[Ace195]

Originally Posted by Bond

All of the addresses may change automatically, but I'm not sure and I wouldn't want to take that risk.

I've done a few hours of research (ugh) on how to get rid of the spider bots, and I found how to create this file that the bots should check to see what folders they are allowed to index and whether or not you even want them indexing your website. Obviously the google spider isn't checking the file, which it should every 48 hours or so. I'm trying a lot of other different things also, so hopefully something will work.

I would contact google like GameMaster said I mean they should be reasonable people.

[GameMaster]

Bond, I think I have the answer to our problem.

http://www.google.com/webmasters/faq.html#nocrawl

[Bond]

Originally Posted by GameMaster

Bond, I think I have the answer to our problem.

http://www.google.com/webmasters/faq.html#nocrawl

That's actually the file I was talking about. Which isn't working...

I'll review that page though and make sure everything is in order.

Alright, I edited the file a little bit so hopefully it will work now.

I also added a meta tag that should tell them to go away.