Using Sitemapper 4.6 on Windows XP
I can't seem to stop some of my subfolders being indexed. I've doing a site spider of http://www.frequencycast.co.uk , and am trying to exclude certain folders from the spider. An example of one I want to get rid of: /blogs/2008/, and any folders below it.
Under Tools > Options > "subfolders to ignore", I've tried various combinations:
http://www.frequencycast.co.uk/blog/2008
http://www.frequencycast.co.uk/blog/2008/
http://www.frequencycast.co.uk/blog/2008/*
... but these subfolders keep appearing in the results.
What am I doing wrong?
Pete
Sitemapper 4.6 not excluding...
Did you try inserting the Meta between head tags
<META NAME="ROBOTS" CONTENT="INDEX,N0 FOLLOW">
<META NAME="ROBOTS" CONTENT="INDEX,N0 FOLLOW">
Pete,
You might also try entering the ignore as /blog/2008/ and see if that works.
You might also try entering the ignore as /blog/2008/ and see if that works.
Have something to add? We’d love to hear it!
You must have an account to participate. Please Sign In Here, then join the conversation.