Getting data from another website dynamically

[Atul]

Hi

Theres a website that books hotels . user enters the information and
according to that results are displayed to the user.Let it be website
A.

Now I want to create a new project with designing page same as in
website A.
But wen user clicks on submit button on my website then the same
process occurs as is happening when user clicks submit button in
website A. but the results that are fetched from websiteA are shown on
my page.

We can't edit/change/add anything in code of websiteA.

any Ideas...........
Thanks'''''''''
Atul

 

[Zeba]

Hi Atul,

I'm not sure if/how you can do this by fetching the data from your
Website A, but I think the usual way of going about it is to use APIs
that are provided by the website A, to access its services from
elsewhere. If it is an open API you can use do that for free, or you
may have to pay royalty for it.

 

[Mehdi]

Theres a website that books hotels . user enters the information and
according to that results are displayed to the user.Let it be website
A.

Now I want to create a new project with designing page same as in
website A.
But wen user clicks on submit button on my website then the same
process occurs as is happening when user clicks submit button in
website A. but the results that are fetched from websiteA are shown on
my page.

We can't edit/change/add anything in code of websiteA.

2 solutions:

1) See if website A provides an API (under the form of a Web Service
usually) to allow third party to perform searches using its search engine.
If it does and its license suits what you want to do, use it, that's the
simplest solution.

2) If website A does not provide any API then whenever your users perform a
search on your site, you'll have to perform a search on site A, retrieve
the HTML code of the result page, parse it to extract the results (see
regular expressions) and display the results on your web page. Of course,
you should check the T&C of site A so make sure that you are allowed to use
its results and display them on your web page.

 

[Guest]

Atul,
this is really more of an ASP.NET group question than a C# language-specific
one.

First, you should find out whether what you propose to do is actually
permitted by the web site owner. If not, you will be dealing with legal
issues instead of programming issues.

Take a look at Simon Mourier's HttpAgilityPack, which is specifically
designed for this type of "web scraping".

Peter

Site: http://www.eggheadcafe.com
UnBlog: http://petesbloggerama.blogspot.com
Short urls & more: http://ittyurl.net

 

[Guest]

Sorry, that's HtmlAgilityPack, and it can be found here:

http://www.codeplex.com/Project/ProjectDirectory.aspx?ProjectSearchText=htmlagilitypack

Peter

 

[Atul]

Atul,
this is really more of an ASP.NET group question than a C# language-specific
one.

First, you should find out whether what you propose to do is actually
permitted by the web site owner. If not, you will be dealing with legal
issues instead of programming issues.

Take a look at Simon Mourier's HttpAgilityPack, which is specifically
designed for this type of "web scraping".

Peter

Site: http://www.eggheadcafe.com
UnBlog: http://petesbloggerama.blogspot.com
Short urls & more: http://ittyurl.net









- Show quoted text -

Hi Peter

Thanks 4 replying


I have permissions from website owner but he is not allowing to edit
anything in his code.
what i want is:

I want to create a new project with designing page same as in website
A.
But when user fills the form and clicks on submit button on my website
then automatically the form of website A will be filled and its submit
button is clicked. Now the results, instead of showing on website A ,
are shown on my page.

The user is never directed to websiteA.

I hope u r getting me. My main aim is to get that results that are
fetched from website A.

Thanks'
Atul

 

[Atul]

2 solutions:

1) See if website A provides an API (under the form of a Web Service
usually) to allow third party to perform searches using its search engine.
If it does and its license suits what you want to do, use it, that's the
simplest solution.

2) If website A does not provide any API then whenever your users perform a
search on your site, you'll have to perform a search on site A, retrieve
the HTML code of the result page, parse it to extract the results (see
regular expressions) and display the results on your web page. Of course,
you should check the T&C of site A so make sure that you are allowed to use
its results and display them on your web page.

Hi

How to do 2nd solution How i will perform search on websiteA

when user fills the form and clicks on submit button on my website
then automatically the form of website A will be filled and its
submit
button is clicked. Now the results, instead of showing on website A ,
are shown on my page.


The user is never directed to websiteA.


I hope u r getting me. My main aim is to get that results that are
fetched from website A.


Thanks'
Atul

 

[Mehdi]

How to do 2nd solution How i will perform search on websiteA

when user fills the form and clicks on submit button on my website
then automatically the form of website A will be filled and its
submit
button is clicked. Now the results, instead of showing on website A ,
are shown on my page.

You'll have to have a look at how website A works. For example, if you go
to google.com and do a search for "development", you are redirected to
<http://www.google.co.uk/search?hl=en&q=development&btnG=Google+Search&meta=>.
So you now know that to retrieve the result of a Google search, you can
simply download the page
http://www.google.co.uk/search?hl=en&q=SEARCH_KEYWORD&btnG=Google+Search&meta=

(of course, in the particular case of Google, you don't need to do that as
Google provides an API for developers to do serches using its search
engine).

Other sites might use POSTDATA instead so you'll need to have a look at
that (there are firefox extensions that allow you to see all the data sent
to a web site whenever you submit a form, I'd suggest that you'd start by
using one of these extension and study how the site works).