will this block both directories in robots.txt file?

[Josh Collins]

Disallow: /misc/hpnet_files/

I want only the /hpnet_files directory to be blocked from scanning by
robots. But will this line also cause the /misc directory from being
scanned? I do have webpages in the /misc directory I want scanned.

 

[Mark Fitzpatrick]

it shouldn't as you're still indicating only one directory as the target.

Hope this helps,
Mark Fitzpatrick
Microsoft MVP - FrontPage

 

[Thomas A. Rowe]

Note: Using a robot.txt file provides a directly path for hackers to find
content that you don't want found or indexed on your web site.

--

==============================================
Thomas A. Rowe (Microsoft MVP - FrontPage)
WEBMASTER Resources(tm)

FrontPage Resources, Forums, WebCircle,
MS KB Quick Links, etc.
==============================================

 

[Josh Collins]

Ahh, but all my directories in the robots file are password protected. Also,
each directory has and index.htm file, so if they try to access a directory
with a partial address they get prompted with a "no permission to access"
page.

Your tip is good advice for the typical webmaster though. Thanks.

 

[Josh Collins]

I did not want my password-protected directories to show in search engines.
I was not aware that robots would not scan such directories. That is very
useful info. Thanks alot.

As far as your comment about hackers, I don't understand the difference
between some hacker getting a directory path from the robots.txt file or the
directory path from merely clicking on links on my hompage. That is, most
hyperlinks to deeply embedded webpages will have several directory paths
listed in the hyperlink to the final file, such as this one:

http://fakeaddress.com/files/papers/research/test.html

What difference does it make if this is in a robots file when the same info
can be gotten from clicking on a link from the main page?

Thanks again for your time.