Fedora Generic Search Service

To install the service:

• Download genericsearch-1.1.zip and copy the included .war file into the webapps directory of your web server, e.g. the tomcat supplied with Fedora, or similar. You may rename the .war file, before you copy it into the webapps directory, in order to give it another webapp name.
• Edit the configuration settings.

The SOAP service operations are deployed with the .war file, and the .wsdl file is available here.

• Edit the configuration files in .../webapps/<WEBAPPNAME>/WEB-INF/classes/config/ (where <WEBAPPNAME> by default is 'fedoragsearch'):

  fedoragsearch.properties:
  	fedoragsearch.soapBase         = http://<HOST:PORT>/<WEBAPPNAME>/services
  	fedoragsearch.soapUser         = <SOAPUSER>
  	fedoragsearch.soapPass         = <SOAPPASSWORD>
  	fedoragsearch.deployFile       = <WEBSERVERPATH>/webapps/<WEBAPPNAME>/WEB-INF/classes/config/deploy.wsdd
                                      (<WEBSERVERPATH> may be "${catalina.home}" in the Tomcat case)
                                      (in general, you may insert system variable values in property values in this way)
  	fedoragsearch.default<OPERATION>RestXslt = <your default Rest stylesheet for the operation>
  	fedoragsearch.mimeTypes        = <a sequence of mimeTypes used by the application>
  	                                 The first accessible mimeType will decide, 
  	                                 which of the datastreams is fetched and indexed.
  	                                 Currently "text/plain text/html application/pdf" are implemented.
  	fedoragsearch.repositoryNames  = <REPOSNAMES>
  	fedoragsearch.indexNames	       = <INDEXNAMES>
  	                                 
  For each repository:
  
  repository/<REPOSNAME>/repository.properties:
  	fgsrepository.fedoraSoap       = http://<HOST:PORT>/fedora/services (e.g. http://localhost:8080)
  	fgsrepository.fedoraUser       = <FEDORAUSER>
  	fgsrepository.fedoraPass       = <FEDORAPASSWORD>
  	fgsrepository.fedoraObjectDir  = <absolute path to FedoraObjects directory with FOXML files>
  	
  For each index:
  
  index/<INDEXNAME>/index.properties:
  	fgsindex.indexDir              = <path to Lucene index directory, which must exist>
  
  .../webapps/<WEBAPPNAME>/WEB-INF/classes/log4j.xml:
      <param name="File" value="/<LOGPATH>/<LOGFILENAME>"/> /LOGPATH must exist
  						

  Edit the three files rest/demo*.xslt, and the file index/DemoOnLucene/demoFoxmlToLucene.xslt, where the xsl:include elements shall have absolute paths instead of relative paths.
• Then you may restart <WEBAPPNAME> and call http://<HOSTPORT>/<WEBAPPNAME>/rest in order to index and search. The name "rest" may be reconfigured in .../webapps/<WEBAPPNAME>/WEB-INF/web.xml
• Try the demo command line client. Change directory to

  .../webapps/<WEBAPPNAME>/client/

  make the file executable, and run

  sh runRESTClient.sh

  then you will get the usage instruction.
• Tailor the demos for your own purpose by editing renamed copies of the demo xslt stylesheets in

  .../webapps/<WEBAPPNAME>/WEB-INF/classes/config/rest/

  Then edit fedoragsearch.properties.
• Tailor the demo Lucene indexing stylesheet for your own purpose by editing a renamed copy of the demo xslt stylesheet in

  .../webapps/<WEBAPPNAME>/WEB-INF/classes/config/index/<INDEXNAME>/demoFoxmlToLucene.xslt

  For the sake of the example, the stylesheet indexes only active Fedora objects with PID starting with "demo" The options for tailoring include fields from other metadata datastreams than DC, field types and field boosts, see the stylesheet for options. Then edit index.properties.
• For your real applications, you must carefully edit all stylesheets for your purpose.
• Inspect the Lucene index with Luke. Notice, Luke cannot open an empty Lucene index.
• You may tailor the highlight of search terms in demo.css.
• You may want to experiment with more than one configuration, in which case you may maintain them under different names in parallel to the "config" configuration, which is the default configuration. In order to activate an alternative configuration you may use the semi-secret operation configure with parameter configName, either using the demo command line client or the REST interface.

As of Fedora 2.2, it is possible to configure Fedora to send a signal to the Generic Search Service when objects are added, modified, and purged. We expect future versions of Fedora to use more sophisticated messaging techniques (like JMS) to accomplish this, but for now Fedora can be made to send simple REST calls to the service at appropriate times.

To enable this, edit your fedora.fcfg file and change the class of the fedora.server.storage.DOManager module to fedora.server.storage.GSearchDOManager. Then populate the following module parameters as needed:

• gSearchRESTURL - The REST endpoint for GSearch, for example, http://localhost:8080/fedoragsearch/rest
• gSearchUsername - If GSearch is protected by authentication, this is the username that Fedora should use to authenticate.
• gSearchPassword - The password for the above user, if applicable

Lucene

The Lucene plugin comes as the java package dk.defxws.fgslucene together with the Apache Lucene java libraries. The set of classes was inspired by the JDBC API specification.

The Lucene plugin is used by configuration as explained above, see also the DemoOnLucene example.

Lucene has a very rich functionality, and this plugin exploits a small fraction of it. As a java programmer, you may have ideas for further exploitation, which you may realize by implementing an enhanced version of the plugin. Please, share such ideas and implementations with the Fedora community.

Zebra

The Zebra plugin comes as the java package dk.defxws.fgszebra .

The Zebra plugin is used by configuration as seen from the DemoOnZebra example, which includes a README file, which explains how to get and install Zebra, and how to configure it.

• All engine specific operations return an engine specific xml answer, which is transformed by an engine-specific xslt stylesheet into result page xml. For a SOAP request this is the answer. For a REST request this is transformed to an html answer. There may be any number of xslt stylesheets to select from, the default ones are selected in the properties file. Selecting a copy stylesheet will allow the transfer of an answer untransformed. An alternative result page format is OpenSearch, which is an RSS2.0 extension.
• Parameters allow clients to select repository, index, and xslt stylesheets by name. In a real application, these values may be determined by the developer in the code, or by the administrator in the properties file.

• Objects in the Fedora repository are exported in FOXML format, transformed into an appropriate document format by the indexing stylesheet, and indexed by the engine in question. The XML datastreams are indexed as decided in the stylesheet. One managed or external datastream may be indexed per FedoraObject (which one is configurable), assuming that they contain the same text in different mimetypes.
• The following updateIndex actions are available:
  • createEmpty - creating or emptying the index. For a new index, you have to run createEmpty once, before you can run the other actions.
  • fromFoxmlFiles ( filePath ) - indexing FOXML records; filePath may be null, in which case the configured Fedora Object Directory is used, so that the whole of the Fedora registry is indexed.
  • fromPid ( PID ) - indexing one FOXML record, as exported by Fedora API-M; in case a previous index document with the same PID exists, it is first deleted. This is the incremental update operation that shall be called after all of Fedora's API-M operations that modifies a FedoraObject.
  • deletePid ( PID ) - deleting one index document.

A typical application will index one repository in one index. However, you have the possibility to index many repositories in one or more indexes in parallel, as illustrated here:

• There are OperationsImpl classes for Zebra and for Lucene. The configManyToMany example has indexes for both engines, therefore similar searches may be compared.

Multilingual Configuration

Luis Zorita had this problem and solved it (his mail to fedora-users on 24 August 2006):