solr 5.1 DIH

Recently I had to use Data Import Handler to index data from postgres database. Unfortunately I had to encounter few issues, I'm blogging the steps and the issues faced.


Setting up DataImportHandler

• Edit your solrconfig.xml to add the request handler

  <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
  <lst name="defaults">
    <str name="config">data-config.xml</str>
  </lst>
  </requestHandler>

• Create a data-config.xml file as follows and save it to the conf dir

  <dataConfig>
    <dataSource type="JdbcDataSource" 
                driver="org.postgresql.Driver"
                url="jdbc:postgresql://host:port/dbname" 
                user="username" 
                password="password"/>
    <document>
      <entity name="col_id" 
              query="select * from report_ks">
      </entity>
    </document>
  </dataConfig>

•  You need to add table fields in schema.xml

• Put rmdbs driver, in my case postgres driver in $SOLR_HOME/dist folder and point it in solrconfig.xml

  <lib dir="${solr.install.dir:../../../..}/dist/" regex="postgresql.*\.jar" />

•  add dataImportHandler jars in solrconfig.xml

  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" />

• Now upload the conf file to zookeeper for cloud mode and reload your collection. To start the import

  http://solr-host:port/solr/collection/dataimport?command=full-import