Robots.txt

Joined
Feb 5, 2003
Messages
127
Reaction score
0
I have created a robots.txt file but when I run a sitemap maker for google two files that I have listed a .jpg and .gif file continue to be listed in the sitemap maker even though they are on the disallow area of the robots.txt.

Other diallowed criteria is successfully bypassed.

Any ideas why these two are not and is there a disallow syntax that can miss all jpeg and gif files?
 

Ian

Administrator
Joined
Feb 23, 2002
Messages
19,873
Reaction score
1,499
I don't think sitemap makers will take robots.txt into account, but google should set robots.txt criteria over the sitemap file (I think!).
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top