org.niocchi.urlpools
Class TimeoutURLPool

java.lang.Object
  extended by org.niocchi.urlpools.TimeoutURLPool
All Implemented Interfaces:
URLPool

public class TimeoutURLPool
extends java.lang.Object
implements URLPool

this class is an URLPool wrapper that drops all subsequent Queries from hosts that have reached too many consecutive timeouts. Implementation detail: the dropQuery methods calls _url_pool.setProcessed with a singleton Resource by assuming _url_pool will not store it.


Constructor Summary
TimeoutURLPool(URLPool pool_)
           
 
Method Summary
 Query getNextQuery()
           
 boolean hasNextQuery()
           
 void setMaxConsecutiveTimeouts(int max_)
           
 void setProcessed(Query query)
          This method is called by the crawler when the query has been processed.
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TimeoutURLPool

public TimeoutURLPool(URLPool pool_)
Method Detail

hasNextQuery

public boolean hasNextQuery()
Specified by:
hasNextQuery in interface URLPool
Returns:
true if there is at least one query to crawl (i.e. getNextQuery will return a query at some point), false if no more queries to crawl (a call to getNextQuery will throw an URLPoolException).

getNextQuery

public Query getNextQuery()
                   throws URLPoolException
Specified by:
getNextQuery in interface URLPool
Returns:
null if no query is yet available, otherwise return a Query. Throws URLPoolException is no query will be available (a call to hasNextQuery() must returns false).
Throws:
URLPoolException

setProcessed

public void setProcessed(Query query)
Description copied from interface: URLPool
This method is called by the crawler when the query has been processed. That gives the oportunity to the url pool to implement specific behaviors (for instance, send back the query to the crawler if it got a timeout.

Specified by:
setProcessed in interface URLPool

setMaxConsecutiveTimeouts

public void setMaxConsecutiveTimeouts(int max_)