com.knowgate.lucene
Class Crawler

java.lang.Object
  com.knowgate.lucene.Crawler

public class Crawler
extends java.lang.Object

Simple HTML crawler for Lucene

See Also:

http://lucene.apache.org/java/2_3_0/api/core/index.html

Constructor Summary

Crawler()

Method Summary

void	crawl(java.lang.String sBasePath, java.lang.String sFileFilter, java.lang.String sIndexDirectory, boolean bRebuild) Add contents to a Lucene Index
static void	main(java.lang.String[] argv)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Crawler

public Crawler()

Method Detail

crawl

public void crawl(java.lang.String sBasePath,
                  java.lang.String sFileFilter,
                  java.lang.String sIndexDirectory,
                  boolean bRebuild)
           throws java.io.IOException,
                  MalformedPatternException

Add contents to a Lucene Index

Parameters:

sBasePath - Base Path for crawling

sFileFilter - Perl5 Regular Expression filter for file names

sIndexDirectory - Lucene index target directory

bRebuild - true if index must be deleted and fully rebuild.

Throws:

java.io.IOException

java.io.FileNotFoundException - If sBasePath direcory does not exist

MalformedPatternException - If sFileFilter is not a valid Perl5 regular expression pattern

main

public static void main(java.lang.String[] argv)
                 throws java.lang.NoSuchFieldException,
                        java.io.IOException,
                        java.io.FileNotFoundException,
                        MalformedPatternException

Throws:

java.lang.NoSuchFieldException

java.io.IOException

java.io.FileNotFoundException

MalformedPatternException