Hiding a line of text from the SE crawler

  • Thread starter Thread starter Fred
  • Start date Start date
F

Fred

Is there a way I can hide a line of text from the search engine
crawler/spider when it crawls my pages. I thought I saw something a while
ago on this group that indicated that if you placed an ! in front of the
line of text it would be ignored by the spider. I want the spider to read
and crawl all the other information on the page but not the navigation bar
links as this information is not relevant to the page. Any ideas would be
most welcome. Fred
 
Put this in your header:

<meta name=robots content=index, nofollow>

It will index the page but not follow links.


Regards Jens Peter Karlsen. Microsoft MVP - Frontpage.


nntp://msnews.microsoft.com/microsoft.public.frontpage.client/<[email protected]>

Is there a way I can hide a line of text from the search engine
crawler/spider when it crawls my pages. I thought I saw something a while
ago on this group that indicated that if you placed an ! in front of the
line of text it would be ignored by the spider. I want the spider to read
and crawl all the other information on the page but not the navigation bar
links as this information is not relevant to the page. Any ideas would be
most welcome. Fred



[microsoft.public.frontpage.client]
 
Not possible to make the text a comment, if you want the text to also appear on the page for users.

If the text is part of your navigation, then it should be readable by the spider, so that they
follow the link to the next page and then index that page.

--
==============================================
Thomas A. Rowe (Microsoft MVP - FrontPage)
WEBMASTER Resources(tm)

FrontPage Resources, WebCircle, MS KB Quick Links, etc.
==============================================
 
Fred said:
Is there a way I can hide a line of text from the search engine
crawler/spider when it crawls my pages. I thought I saw something a while
ago on this group that indicated that if you placed an ! in front of the
line of text it would be ignored by the spider. I want the spider to read
and crawl all the other information on the page but not the navigation bar
links as this information is not relevant to the page. Any ideas would be
most welcome. Fred
Click to expand...

When I picked myself up off the floor from laughing ..... I thought I'd
better tell you that they are very releveant to seach engines and more so to
you, if you want to be found via search engines. Using Index, nofollow will
not do you any good at all using it generally on every page, that's for use
in certain situations and as part of optimising your site that's a no don't
do that.

Go join a seo group and find out more before you make this big mistake.

http://groups.yahoo.com/group/SEO_Techniques/
free classes.

hth Tina


--
http://accessfp.net/ - FrontPage Resource Centre
http://anybackpage.com/bytes/ - Ezine with Free E-books
http://artdoodle.com/ - Original Art
http://addonfp.com - FrontPage Add-ons
http://frontpage-tips.com - FrontPage Tips
 
Actually a meta tag has nothing to do with a search engine "finding" a site. Search engines (
search bots ) "find" sites by querying the Domain Name Server system and then visit those "domains"
that are found.

If a web master has a page with links that they don't want followed, using <meta name=robots
content=index, nofollow>
is an excellent way of accomplishing this because the search engine will index the page title and
content on the page, but will *not* follow the links contained in the page and will therefore not
index the content at the links.


--
Steve Easton
Microsoft MVP FrontPage
95isalive
This site is best viewed............
........................with a computer
 
Nice insult, Tina.

--
===
Tom "Pepper" Willett
Microsoft MVP - FrontPage
---
About FrontPage 2003:
http://office.microsoft.com/home/office.aspx?assetid=FX01085802
FrontPage 2003 Product Information:
http://www.microsoft.com/office/frontpage/prodinfo/default.mspx
Understanding FrontPage:
http://msdn.microsoft.com/office/understanding/frontpage/
===
|
| | > Is there a way I can hide a line of text from the search engine
| > crawler/spider when it crawls my pages. I thought I saw something a
while
| > ago on this group that indicated that if you placed an ! in front of the
| > line of text it would be ignored by the spider. I want the spider to
read
| > and crawl all the other information on the page but not the navigation
bar
| > links as this information is not relevant to the page. Any ideas would
be
| > most welcome. Fred
|
| When I picked myself up off the floor from laughing ..... I thought I'd
| better tell you that they are very releveant to seach engines and more so
to
| you, if you want to be found via search engines. Using Index, nofollow
will
| not do you any good at all using it generally on every page, that's for
use
| in certain situations and as part of optimising your site that's a no
don't
| do that.
|
| Go join a seo group and find out more before you make this big mistake.
|
| http://groups.yahoo.com/group/SEO_Techniques/
| free classes.
|
| hth Tina
|
|
| --
| http://accessfp.net/ - FrontPage Resource Centre
| http://anybackpage.com/bytes/ - Ezine with Free E-books
| http://artdoodle.com/ - Original Art
| http://addonfp.com - FrontPage Add-ons
| http://frontpage-tips.com - FrontPage Tips
|
|
 
Fred said:
Is there a way I can hide a line of text from the search engine
crawler/spider when it crawls my pages. I thought I saw something a while
ago on this group that indicated that if you placed an ! in front of the
line of text it would be ignored by the spider. I want the spider to read
and crawl all the other information on the page but not the navigation bar
links as this information is not relevant to the page. Any ideas would be
most welcome. Fred
Click to expand...

Ok ... let me explain, there is no insult .. I speak in text as though I'm
stood there next to you ... so that's how I 'talk' in email .. if you don't
like it don't read my posts... and if your stood next to me walk away.. that
simple. (that to tom)

Now to explain why I found the above question so funny.

First you should not use noindex,nofollow anywhere on a business site.

to index a page
there have to be links to a page
if there are no links to a page
it is an orphan in googles eyes
they will ignore it

For a page in your site .... there has to be links to a page for a page to
be indexed

The REASON so many people THINK the no index tag is important is because
they run some generic searches on their site name. This of course brings up
several pages in which Google identical se descriptions will be shown. In
more specific searches though this is not an issue. This is a GOOD thing,
because you have no way in knowing which page will catch someone's
attention. ( business can come to through one's site map - an unlikely page
for business contacts )

In other words your BACKLINKING your own site ... you would not turn down
links on other relevent sites would you? Why turn them down on your own?

If you use includes just put them into a folder marked with an underscore
_inc
the includes page itself you don't have to worry about because they only
follow links - no links to follow to the include itself
but if you put no follow in an include - that instruction could follow back
to the page the only thing that matters on this page is the content so you
keep everything barebones.
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
that's all you need (or whatever you use)
for the include page.

and if you disagree with the above we will have to agree to disagree I'm
afraid.

Tina
 
In Line

--
Steve Easton
Microsoft MVP FrontPage
95isalive
This site is best viewed............
........................with a computer

Tina Clarke said:
Ok ... let me explain, there is no insult .. I speak in text as though I'm
stood there next to you ... so that's how I 'talk' in email .. if you don't
like it don't read my posts... and if your stood next to me walk away.. that
simple. (that to tom)

Now to explain why I found the above question so funny.

First you should not use noindex,nofollow anywhere on a business site.

to index a page
there have to be links to a page
if there are no links to a page
it is an orphan in googles eyes
they will ignore it

For a page in your site .... there has to be links to a page for a page to
be indexed
Click to expand...

The posters original question was, I have a page in my site with links I don't want followed.
The meta index,nofollow is a perfect answer for his question.

The REASON so many people THINK the no index tag is important is because
they run some generic searches on their site name. This of course brings up
several pages in which Google identical se descriptions will be shown. In
more specific searches though this is not an issue. This is a GOOD thing,
because you have no way in knowing which page will catch someone's
attention. ( business can come to through one's site map - an unlikely page
for business contacts )

In other words your BACKLINKING your own site ... you would not turn down
links on other relevent sites would you? Why turn them down on your own?
Click to expand...

Not true.
If you use includes just put them into a folder marked with an underscore
_inc
the includes page itself you don't have to worry about because they only
follow links - no links to follow to the include itself
but if you put no follow in an include - that instruction could follow back
to the page the only thing that matters on this page is the content so you
keep everything barebones.
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
that's all you need (or whatever you use)
for the include page.
Click to expand...


The frontpage include is a design time include, the content is included during the publish process.
The content of the head tags in an include is irrelevant, as the only thing that is included is the
content between the <body></body> tags. Everything else is ignored.
It makes no difference where you "_bury" the include page, it is included when published and will be
indexed, "unless" it is a server side include ( SSI )
and if you disagree with the above we will have to agree to disagree I'm
afraid.
Click to expand...


I disagree.
 
Let me be more precise .....

My opinion is that using nofollow,noindex or any combination of that in your
include menus or menus on a page will result in lost of seo, it's a part of
one of the many things you do .. but every part is vital.

Everyone else's (that answered) in this thread (and apparently elsewhere)
is the opposite ...

the original poster can make up their own mind and also find out from
experimenting with both sets of ideas.

Perhaps doing searches at the suggested site might help ... though it
depends on what year your reading ... the advice changes on any given se
subject. http://searchenginewatch.com/searchday/article.php/2159711 This is
the nearest I can find in support of my stance in the time I devoted to
searching for any info on noindex. There were more insteresting looking
links but they were pay per view only.

The 'laughing' btw was not aimed at the orginal poster but at the advice
given .... since we can't agree on the subject or on the way I post we will
agree to disagree. End of subject for me.

Tina - Microsoft MVP - FrontPage
 
Your laughing was in reply to the OP, and was condescending to *all* persons
in this newsgroup. Please re-read it.

I will no longer post to this reply in public newsgroups, but will be happy
to discuss it in the private group, where it has also been addressed.

Tom Pepper Willett
Microsoft MVP - FrontPage

| Let me be more precise .....
|
| My opinion is that using nofollow,noindex or any combination of that in
your
| include menus or menus on a page will result in lost of seo, it's a part
of
| one of the many things you do .. but every part is vital.
|
| Everyone else's (that answered) in this thread (and apparently elsewhere)
| is the opposite ...
|
| the original poster can make up their own mind and also find out from
| experimenting with both sets of ideas.
|
| Perhaps doing searches at the suggested site might help ... though it
| depends on what year your reading ... the advice changes on any given se
| subject. http://searchenginewatch.com/searchday/article.php/2159711 This
is
| the nearest I can find in support of my stance in the time I devoted to
| searching for any info on noindex. There were more insteresting looking
| links but they were pay per view only.
|
| The 'laughing' btw was not aimed at the orginal poster but at the advice
| given .... since we can't agree on the subject or on the way I post we
will
| agree to disagree. End of subject for me.
|
| Tina - Microsoft MVP - FrontPage
|
|
 
Tina,

When you use the FP Include Page Component or Shared Borders Component to include Navigation Menus
on other pages within a site, setting the NoIndex,NoFollow meta tag only effect the physical Include
or Shared Border page, as FP only includes the content that is between the
<body>...content...</body> tag in other pages, so the meta tags from the Included pages, are never
included in the pages containing the Included content.

FYI: I mark all of my Include Pages with NoIndex,NoFollow, as I don't want the search engines to
index the Include Pages or follow the links from the Include page. The search engine should only
index and follow links that are contained on pages that a site visitor would normally see when they
browser the site and view pages containing the Included Menu content.

--
==============================================
Thomas A. Rowe (Microsoft MVP - FrontPage)
WEBMASTER Resources(tm)

FrontPage Resources, WebCircle, MS KB Quick Links, etc.
==============================================
 
Tina Clarke said:
Let me be more precise .....

My opinion is that using nofollow,noindex or any combination of that in your
include menus or menus on a page will result in lost of seo, it's a part of
one of the many things you do .. but every part is vital.

Everyone else's (that answered) in this thread (and apparently elsewhere)
is the opposite ...

the original poster can make up their own mind and also find out from
experimenting with both sets of ideas.

Perhaps doing searches at the suggested site might help ... though it
depends on what year your reading ... the advice changes on any given se
subject. http://searchenginewatch.com/searchday/article.php/2159711 This is
the nearest I can find in support of my stance in the time I devoted to
searching for any info on noindex. There were more insteresting looking
links but they were pay per view only.

The 'laughing' btw was not aimed at the orginal poster but at the advice
given .... since we can't agree on the subject or on the way I post we will
agree to disagree. End of subject for me.

Tina - Microsoft MVP - FrontPage
Click to expand...


Well Tina..since you replied to me here goes:

After the original question was asked..there were only 2 replies before
yours...One by Thomas
Rowe..and one by Jens Peter Karlsen.. both of whom answered the original
posters question. So I guess you were laughing at them then? The poster did
*not* ask about sea..he *wanted* the page to be the end of the line with
regards to spidering.

Now I'm done with it too.
 
Joe,

Actually the original question was not about hiding any pages from the search engines, but a
specific line of text, but then the original poster goes on to say that he want the spiders to index
all content on the page, but exclude the navigation bar link content. This however is not possible.
--
==============================================
Thomas A. Rowe (Microsoft MVP - FrontPage)
WEBMASTER Resources(tm)

FrontPage Resources, WebCircle, MS KB Quick Links, etc.
==============================================
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Web Crawler Architecture 3
Hiding Javascript 1
Email link that hides from SPAM worms? 8
C# Crawler and performance (speed of crawling) 1
Protecting Email links 6
Hiding a download page URL 1
Crawler Toolbar - EULA advice needed 5
The need for robot.txt file on website 4

Back
Top