| Home | Forums | Reviews | Articles | Register |
 |
| Thread Tools | Rate Thread |
|
Guest
Posts: n/a
|
|
|
|
||
|
|
| |
|
Guest
Posts: n/a
|
|
|
|
|
||
|
Guest
Posts: n/a
|
Thanks Tom!!! The validator said "No errors found in this robots.txt file"
YEA!!!!!! I did it!!!!  I did mine like this, because I read that it should have at least one disallow. What do you think, I don't have a password on the cgi-bin file tho. User-agent: * Disallow: /cgi-bin/ Thanks again!!! Susie "Tom Willett" wrote: > It look fine, Susie. You can validate it at: > http://tool.motoricerca.info/robots-checker.phtml > -- > Tom Willett > Microsoft MVP - FrontPage > FrontPage Support: http://www.frontpagemvps.com/ > ---------- > "Susie" <(E-Mail Removed)> wrote in message > news:96601DD4-C263-4D5B-8282-(E-Mail Removed)... > >I made the robots.txt file in notepad, and saved it into my website folder > > thats located on my desktop, so now it's in my list of pages in my fp 2003 > > program. All my pages are in alphabetical order, so its where R would be. > > It has a little typed page symbol next to it, NOT a folder like _private, > > or > > cgi-bin. Is that right? > > > > I published, and if I look at http://mysite.com/robots.txt I see my robot > > page. > > > > Sorry, this stuff just confuses me to no end! > > > > Thanks, > > Susie > > > |
|
|
|
|
|||
|
Guest
Posts: n/a
|
The whole purpose of a robots.txt file is to disallow files. So, if you
didn't have at least one, why would you need it? -- Tom Willett Microsoft MVP - FrontPage FrontPage Support: http://www.frontpagemvps.com/ ---------- "Susie" <(E-Mail Removed)> wrote in message news:B9F25C4E-4641-4F60-9139-(E-Mail Removed)... > Thanks Tom!!! The validator said "No errors found in this robots.txt > file" > YEA!!!!!! I did it!!!! > > I did mine like this, because I read that it should have at least one > disallow. What do you think, I don't have a password on the cgi-bin file > tho. > > User-agent: * > Disallow: /cgi-bin/ > > Thanks again!!! > Susie > > "Tom Willett" wrote: > >> It look fine, Susie. You can validate it at: >> http://tool.motoricerca.info/robots-checker.phtml >> -- >> Tom Willett >> Microsoft MVP - FrontPage >> FrontPage Support: http://www.frontpagemvps.com/ >> ---------- >> "Susie" <(E-Mail Removed)> wrote in message >> news:96601DD4-C263-4D5B-8282-(E-Mail Removed)... >> >I made the robots.txt file in notepad, and saved it into my website >> >folder >> > thats located on my desktop, so now it's in my list of pages in my fp >> > 2003 >> > program. All my pages are in alphabetical order, so its where R would >> > be. >> > It has a little typed page symbol next to it, NOT a folder like >> > _private, >> > or >> > cgi-bin. Is that right? >> > >> > I published, and if I look at http://mysite.com/robots.txt I see my >> > robot >> > page. >> > >> > Sorry, this stuff just confuses me to no end! >> > >> > Thanks, >> > Susie >> >> >> |
|
|
|
|
|||
|
Guest
Posts: n/a
|
I read that a robot file is good to have for search engines. Can you tell me
if I have put it in the root directory? I was very confused by how to get to the root directory. Thanks, Susie "Tom Willett" wrote: > The whole purpose of a robots.txt file is to disallow files. So, if you > didn't have at least one, why would you need it? > > > -- > Tom Willett > Microsoft MVP - FrontPage > FrontPage Support: http://www.frontpagemvps.com/ > ---------- > "Susie" <(E-Mail Removed)> wrote in message > news:B9F25C4E-4641-4F60-9139-(E-Mail Removed)... > > Thanks Tom!!! The validator said "No errors found in this robots.txt > > file" > > YEA!!!!!! I did it!!!! > > > > I did mine like this, because I read that it should have at least one > > disallow. What do you think, I don't have a password on the cgi-bin file > > tho. > > > > User-agent: * > > Disallow: /cgi-bin/ > > > > Thanks again!!! > > Susie > > > > "Tom Willett" wrote: > > > >> It look fine, Susie. You can validate it at: > >> http://tool.motoricerca.info/robots-checker.phtml > >> -- > >> Tom Willett > >> Microsoft MVP - FrontPage > >> FrontPage Support: http://www.frontpagemvps.com/ > >> ---------- > >> "Susie" <(E-Mail Removed)> wrote in message > >> news:96601DD4-C263-4D5B-8282-(E-Mail Removed)... > >> >I made the robots.txt file in notepad, and saved it into my website > >> >folder > >> > thats located on my desktop, so now it's in my list of pages in my fp > >> > 2003 > >> > program. All my pages are in alphabetical order, so its where R would > >> > be. > >> > It has a little typed page symbol next to it, NOT a folder like > >> > _private, > >> > or > >> > cgi-bin. Is that right? > >> > > >> > I published, and if I look at http://mysite.com/robots.txt I see my > >> > robot > >> > page. > >> > > >> > Sorry, this stuff just confuses me to no end! > >> > > >> > Thanks, > >> > Susie > >> > >> > >> > > > |
|
|
|
|
|||
|
Guest
Posts: n/a
|
The only reason you would need to have one for the search engines is to
prevent 404 errors in your logs when they attempt to search it. And, an empty robots.txt file would accomplish that (which is what I use). The robots will visit your site whether you have one or not. Many will simply not index pages that are denied in your file. Yes, the file must be in the *root* of the main root web, and not in a sub site. -- Tom Willett Microsoft MVP - FrontPage FrontPage Support: http://www.frontpagemvps.com/ ---------- "Susie" <(E-Mail Removed)> wrote in message news:160A3B37-934B-430A-99DF-(E-Mail Removed)... >I read that a robot file is good to have for search engines. Can you tell >me > if I have put it in the root directory? I was very confused by how to get > to > the root directory. > Thanks, > Susie > > "Tom Willett" wrote: > >> The whole purpose of a robots.txt file is to disallow files. So, if you >> didn't have at least one, why would you need it? >> >> >> -- >> Tom Willett >> Microsoft MVP - FrontPage >> FrontPage Support: http://www.frontpagemvps.com/ >> ---------- >> "Susie" <(E-Mail Removed)> wrote in message >> news:B9F25C4E-4641-4F60-9139-(E-Mail Removed)... >> > Thanks Tom!!! The validator said "No errors found in this robots.txt >> > file" >> > YEA!!!!!! I did it!!!! >> > >> > I did mine like this, because I read that it should have at least one >> > disallow. What do you think, I don't have a password on the cgi-bin >> > file >> > tho. >> > >> > User-agent: * >> > Disallow: /cgi-bin/ >> > >> > Thanks again!!! >> > Susie >> > >> > "Tom Willett" wrote: >> > >> >> It look fine, Susie. You can validate it at: >> >> http://tool.motoricerca.info/robots-checker.phtml >> >> -- >> >> Tom Willett >> >> Microsoft MVP - FrontPage >> >> FrontPage Support: http://www.frontpagemvps.com/ >> >> ---------- >> >> "Susie" <(E-Mail Removed)> wrote in message >> >> news:96601DD4-C263-4D5B-8282-(E-Mail Removed)... >> >> >I made the robots.txt file in notepad, and saved it into my website >> >> >folder >> >> > thats located on my desktop, so now it's in my list of pages in my >> >> > fp >> >> > 2003 >> >> > program. All my pages are in alphabetical order, so its where R >> >> > would >> >> > be. >> >> > It has a little typed page symbol next to it, NOT a folder like >> >> > _private, >> >> > or >> >> > cgi-bin. Is that right? >> >> > >> >> > I published, and if I look at http://mysite.com/robots.txt I see my >> >> > robot >> >> > page. >> >> > >> >> > Sorry, this stuff just confuses me to no end! >> >> > >> >> > Thanks, >> >> > Susie >> >> >> >> >> >> >> >> >> |
|
|
|
|
|||
|
Guest
Posts: n/a
|
Thanks Tom, but I have to say that now I am confused, I read this comment by
Steve Raston, (he was replying to someone that wanted to make sure people found certain things.) Steve said, *And If* you have your robots.txt file set up to allow search engine spiders to index your web. So I thought I had to have one. And this site http://www.seoconsultants.com/robots-text-file/ says in important notes, that it needs to disallow at least one. "Tom Willett" wrote: > The only reason you would need to have one for the search engines is to > prevent 404 errors in your logs when they attempt to search it. And, an > empty robots.txt file would accomplish that (which is what I use). > > The robots will visit your site whether you have one or not. Many will > simply not index pages that are denied in your file. > > Yes, the file must be in the *root* of the main root web, and not in a sub > site. > -- > Tom Willett > Microsoft MVP - FrontPage > FrontPage Support: http://www.frontpagemvps.com/ > ---------- > "Susie" <(E-Mail Removed)> wrote in message > news:160A3B37-934B-430A-99DF-(E-Mail Removed)... > >I read that a robot file is good to have for search engines. Can you tell > >me > > if I have put it in the root directory? I was very confused by how to get > > to > > the root directory. > > Thanks, > > Susie > > > > "Tom Willett" wrote: > > > >> The whole purpose of a robots.txt file is to disallow files. So, if you > >> didn't have at least one, why would you need it? > >> > >> > >> -- > >> Tom Willett > >> Microsoft MVP - FrontPage > >> FrontPage Support: http://www.frontpagemvps.com/ > >> ---------- > >> "Susie" <(E-Mail Removed)> wrote in message > >> news:B9F25C4E-4641-4F60-9139-(E-Mail Removed)... > >> > Thanks Tom!!! The validator said "No errors found in this robots.txt > >> > file" > >> > YEA!!!!!! I did it!!!! > >> > > >> > I did mine like this, because I read that it should have at least one > >> > disallow. What do you think, I don't have a password on the cgi-bin > >> > file > >> > tho. > >> > > >> > User-agent: * > >> > Disallow: /cgi-bin/ > >> > > >> > Thanks again!!! > >> > Susie > >> > > >> > "Tom Willett" wrote: > >> > > >> >> It look fine, Susie. You can validate it at: > >> >> http://tool.motoricerca.info/robots-checker.phtml > >> >> -- > >> >> Tom Willett > >> >> Microsoft MVP - FrontPage > >> >> FrontPage Support: http://www.frontpagemvps.com/ > >> >> ---------- > >> >> "Susie" <(E-Mail Removed)> wrote in message > >> >> news:96601DD4-C263-4D5B-8282-(E-Mail Removed)... > >> >> >I made the robots.txt file in notepad, and saved it into my website > >> >> >folder > >> >> > thats located on my desktop, so now it's in my list of pages in my > >> >> > fp > >> >> > 2003 > >> >> > program. All my pages are in alphabetical order, so its where R > >> >> > would > >> >> > be. > >> >> > It has a little typed page symbol next to it, NOT a folder like > >> >> > _private, > >> >> > or > >> >> > cgi-bin. Is that right? > >> >> > > >> >> > I published, and if I look at http://mysite.com/robots.txt I see my > >> >> > robot > >> >> > page. > >> >> > > >> >> > Sorry, this stuff just confuses me to no end! > >> >> > > >> >> > Thanks, > >> >> > Susie > >> >> > >> >> > >> >> > >> > >> > >> > > > |
|
|
|
|
|||
|
Guest
Posts: n/a
|
Susie:
Don't worry about it. You did just fine ;-) Move on to your next project. -- Tom Willett Microsoft MVP - FrontPage FrontPage Support: http://www.frontpagemvps.com/ ---------- "Susie" <(E-Mail Removed)> wrote in message news:C17BF186-4FF4-4C7E-8C5D-(E-Mail Removed)... > Thanks Tom, but I have to say that now I am confused, I read this comment > by > Steve Raston, (he was replying to someone that wanted to make sure people > found certain things.) Steve said, *And If* you have your robots.txt > file > set up to allow search engine spiders to index your web. > > So I thought I had to have one. And this site > http://www.seoconsultants.com/robots-text-file/ says in important notes, > that > it needs to disallow at least one. > > > > > "Tom Willett" wrote: > >> The only reason you would need to have one for the search engines is to >> prevent 404 errors in your logs when they attempt to search it. And, an >> empty robots.txt file would accomplish that (which is what I use). >> >> The robots will visit your site whether you have one or not. Many will >> simply not index pages that are denied in your file. >> >> Yes, the file must be in the *root* of the main root web, and not in a >> sub >> site. >> -- >> Tom Willett >> Microsoft MVP - FrontPage >> FrontPage Support: http://www.frontpagemvps.com/ >> ---------- >> "Susie" <(E-Mail Removed)> wrote in message >> news:160A3B37-934B-430A-99DF-(E-Mail Removed)... >> >I read that a robot file is good to have for search engines. Can you >> >tell >> >me >> > if I have put it in the root directory? I was very confused by how to >> > get >> > to >> > the root directory. >> > Thanks, >> > Susie >> > >> > "Tom Willett" wrote: >> > >> >> The whole purpose of a robots.txt file is to disallow files. So, if >> >> you >> >> didn't have at least one, why would you need it? >> >> >> >> >> >> -- >> >> Tom Willett >> >> Microsoft MVP - FrontPage >> >> FrontPage Support: http://www.frontpagemvps.com/ >> >> ---------- >> >> "Susie" <(E-Mail Removed)> wrote in message >> >> news:B9F25C4E-4641-4F60-9139-(E-Mail Removed)... >> >> > Thanks Tom!!! The validator said "No errors found in this >> >> > robots.txt >> >> > file" >> >> > YEA!!!!!! I did it!!!! >> >> > >> >> > I did mine like this, because I read that it should have at least >> >> > one >> >> > disallow. What do you think, I don't have a password on the cgi-bin >> >> > file >> >> > tho. >> >> > >> >> > User-agent: * >> >> > Disallow: /cgi-bin/ >> >> > >> >> > Thanks again!!! >> >> > Susie >> >> > >> >> > "Tom Willett" wrote: >> >> > >> >> >> It look fine, Susie. You can validate it at: >> >> >> http://tool.motoricerca.info/robots-checker.phtml >> >> >> -- >> >> >> Tom Willett >> >> >> Microsoft MVP - FrontPage >> >> >> FrontPage Support: http://www.frontpagemvps.com/ >> >> >> ---------- >> >> >> "Susie" <(E-Mail Removed)> wrote in message >> >> >> news:96601DD4-C263-4D5B-8282-(E-Mail Removed)... >> >> >> >I made the robots.txt file in notepad, and saved it into my >> >> >> >website >> >> >> >folder >> >> >> > thats located on my desktop, so now it's in my list of pages in >> >> >> > my >> >> >> > fp >> >> >> > 2003 >> >> >> > program. All my pages are in alphabetical order, so its where R >> >> >> > would >> >> >> > be. >> >> >> > It has a little typed page symbol next to it, NOT a folder like >> >> >> > _private, >> >> >> > or >> >> >> > cgi-bin. Is that right? >> >> >> > >> >> >> > I published, and if I look at http://mysite.com/robots.txt I see >> >> >> > my >> >> >> > robot >> >> >> > page. >> >> >> > >> >> >> > Sorry, this stuff just confuses me to no end! >> >> >> > >> >> >> > Thanks, >> >> >> > Susie >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> |
|
|
|
|
|||
|
|
| |
|
| Thread Tools | |
| Rate This Thread | |
|
|
