Friday, January 2, 2009

Search Web Services or API for Commercial Use

Suppose, you need to collect some data based on Internet search for your business. What options are there to use existing web search services? Google? Yahoo? Anyone else? Here's what I have found.

Here's what I have found:

  • Google SOAP search web services seem not to be available for new applications any more. In 2006, Google seems to have stopped issuing new API keys.
  • There is another option from Google: API not SOAP based. The terms don't exclude commercial use, but there is a restriction on caching the results of the search. I guess, if we process the results and store the processed data, it wouldn't be considered caching.
  • The API services have exclusivity requirement: you are not supposed to use on the same site any other search provider.
Here's what I have found:
  • Y! makes the search available via web services (REST but not SOAP webservices).
  • Each application has to use Application ID, which Y! issues for free. There is a rate limit for each application id.
  • Allowed usage: web search API is available to commercial use, however a form of exclusivity restraint is imposed: you are not supposed to integrate the results with results from other search providers.
  • Storage restrictions: "you may not store any user data collected through the Yahoo! APIs for more than 24 hours". Again, would storing the processed, for example aggregated data, be subject to this restriction? I suppose, not.
MSDN: Live Search WebServices API
Terms of use summary:
  • SOAP-based webservices call Silky Road.
  • No restrictions other than an application ID needed. But their license is more complex and I might have missed something.
I'm sure there are/may be other search web services available, and some of them may be free. DoubleClick? I didn't analyze other options.

No comments: