Category Archives: autosuggest

FuzzyLookupFactory is not detecting spelling mistake after first word, Apache SOLR, Autosuggest

I am trying to implement auto-suggest in solr-6.5.1 but its not working properly. I am using FuzzyLookupFactory and AnalyzingInfixLookupFactory, but this configuration is not detecting typos after first word, only working for first word. My configurations are as follow:

Data that is indexed :

{
"name": "singh davinder gosain",
"age":29,
"address":"sector 23 gurgaon"
}

Request Handler :

<requestHandler name="/correctMe" class="solr.SearchHandler" startup="lazy">
  <lst name="defaults">
    <str name="suggest">true</str>
    <str name="suggest.count">10</str>
    <str name="df">name</str>
    <str name="suggest.dictionary">suggestions</str>
    <str name="suggest.dictionary">results</str>      
  </lst>
  <arr name="components">
    <str>correctMeSuggest</str>
  </arr>
</requestHandler>

Search component :

<searchComponent name="correctMeSuggest" class="solr.SuggestComponent">
    <lst name="suggester">
    <str name="name">suggestions</str>
    <str name="lookupImpl">FuzzyLookupFactory</str>
    <str name="dictionaryImpl">DocumentDictionaryFactory</str>
    <str name="field">name</str>
    <str name="weightField">popularity</str>
    <str name="suggestAnalyzerFieldType">text</str>
    <str name="payloadField">address</str>
  </lst>
  <lst name="suggester">
    <str name="name">results</str>
    <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
    <str name="dictionaryImpl">DocumentDictionaryFactory</str>
    <str name="field">name</str>
    <str name="weightField">popularity</str>
    <str name="suggestAnalyzerFieldType">text</str> 
    <str name="payloadField">address</str>
  </lst>
</searchComponent>

Indexed field :

<field name="name" type="text"  indexed="true"  stored="true" multiValued="false"/>
<field name="address" type="text"  indexed="false"  stored="true" multiValued="false"/>

Field Type :

<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>

  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>

  </analyzer>
</fieldType>

Following query working fine as it detects spelling mistake in first word of name field. http://localhost:8983/solr/basic/correctMe?q=sigh&df=name

<response>
<lst name="suggest">
    <lst name="suggestions">
        <lst name="sigh">
            <int name="numFound">1</int>
            <arr name="suggestions">
            <lst>
                <str name="term">singh davinder gosain</str>
                <long name="weight">0</long>
                <str name="payload">sector 23 gurgaon</str>
            </lst>
            </arr>
        </lst>
    </lst>
    <lst name="results">
    <lst name="sigh">
    <int name="numFound">0</int>
    <arr name="suggestions"/>
    </lst>
    </lst>
</lst>
</response>

Following query is not able to detect spelling mistake in second word of name field. http://localhost:8983/solr/basic/correctMe?q=davindr&df=name

<response>
    <lst name="suggest">
        <lst name="suggestions">
            <lst name="davindr">
                <int name="numFound">0</int>
                <arr name="suggestions"/>
            </lst>
        </lst>
        <lst name="results">
            <lst name="davindr">
                <int name="numFound">0</int>
                <arr name="suggestions"/>
            </lst>
        </lst>
    </lst>
</response>

Thanks

write.lock issue in apache solr using AnalyzingInfixLookupFactory

I am using AnalyzingInfixLookupFactory for Auto Suggest feature in our application.But when I try to use auto suggest feature and search for terms in the text box after some time it throws a write.lock error. Below is my configuration in solr-config.xml file for the suggestor / suggest component and suggest request handler :

<searchComponent name="suggest" class="solr.SuggestComponent">
  <lst name="suggester">
    <str name="name">mySuggester</str>
    <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
    <str name="dictionaryImpl">DocumentDictionaryFactory</str>
    <str name="field">text</str>
    <str name="weightField">price</str>
    <str name="payloadField">prod_id</str>
    <str name="contextField">ancestors</str>  
    <str name="suggestAnalyzerFieldType">text_general</str>  
    <str name="buildOnStartup">false</str>
  </lst>
</searchComponent>

<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
  <lst name="defaults">
    <str name="suggest">true</str>
    <str name="suggest.count">10</str>
  </lst>
  <arr name="components">
    <str>suggest</str>
  </arr>
</requestHandler>

Any idea or solution how I can circumvent this ? Thanks.

solar fails when search requirement is fuzzy and wildcard search

what i need

  • i need to build search like amazon and other e-com site using suggestive search .

  • i have tried with wilcard search it works like q=string; but i fails to search when wrong word says plnts , bcs only works 0,1 form.

  • when i used fuzzy search q="string~1" it give result when plants->written plnts , but it fails in searching keyword zer for fertilizer.

  • i have used suggestor but it is not fullfling search requirement link: http://wiki.apache.org/solr/Suggester.

  • im being struggling for 3 days to get solution .

  • i have googled found link : https://gist.github.com/dalegaspi/20b7db4ef2d4d55be661 typehaed js which twitter used.

  • code i have used

        function startSolrSearch ()
        {
        var srchval = $("#productSearch").val().trim();
        if (srchval.length >= 1){
        srchval = srchval.replace(/'|"/g,"").replace(/ /g,"*+*");
    
        srchval = srchval.replace(/ /g,"+AND+");
            console.log(srchval);
        srchval = "*"+srchval+"*";
    
    
        for(var i = 0; i < solrPendingRequests.length; i++){
        solrPendingRequests[i].abort();
        }
        console.log(srchval);
        solrPendingRequests.push($.ajax({
        type : "GET",
        url : solrbasepath + "solr/afeef/select?q="+srchval+'~'+1+"&facet=true&facet.field=title&facet.field=keywords&facet.field=category&indent=on&qt=spellchecker&wt=json&hl=true&hl.fl=title+keywords+model_name&hl.simple.pre=<b>&hl.simple.post=<%2Fb>&hl.usePhraseHighlighter=true&hl.highlightMultiTerm=true&hl.snippets=1&start=0&rows=100",
        dataType : 'JSONP',
        cache : false,
        jsonp: 'json.wrf',
        success : function(html) {
    
        getSearchData (html);
    
        }
        }));
        } 
        else {
    
    
        $('#searchsuggestion .linkresults').html('');
        $('#searchsuggestion').hide();
        $('.close_search').hide();
        }
    

    url

      http://www.supplified.com:8983/solr/vikas/select?q=zer~1 OR q="*zer*" &ind   nt=on&q.op=AND&facet=true&facet.field=title&facet.field=keywords&debugQuery=on&qt=spellchecker&wt=json&hl=true&hl.fl=title+keywords+model_name&hl.simple.pre=%3Cb%3E&hl.simple.post=%3C%2Fb%3E&hl.usePhraseHighlighter=true&hl.highlightMultiTerm=true&hl.snippets=1&start=0&rows=10&json.wrf=jQuery111205813984868582338_1454571708911&_=1454571708913
    
  • i need suggestion how i could used suggestive search like amazon used in there e-com sites.

java.lang.IllegalArgumentException with Solr Spellcheck query

Hi guy iam trying to test solr spellcheck feature with "/spell" in the request but im getting error:

    {
  "responseHeader": {
    "status": 500,
    "QTime": 6
  },
  "error": {
    "msg": "All checkers need to use the same Analyzer.",
    "trace": "java.lang.IllegalArgumentException: All checkers need to use the same Analyzer.\r\n\tat org.apache.solr.spelling.ConjunctionSolrSpellChecker.addChecker(ConjunctionSolrSpellChecker.java:79)\r\n\tat org.apache.solr.handler.component.SpellCheckComponent.getSpellChecker(SpellCheckComponent.java:514)\r\n\tat org.apache.solr.handler.component.SpellCheckComponent.prepare(SpellCheckComponent.java:123)\r\n\tat org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197)\r\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\r\n\tat org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:246)\r\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\r\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)\r\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)\r\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)\r\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\r\n\tat org.eclipse.jetty.server.Server.handle(Server.java:368)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)\r\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)\r\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)\r\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)\r\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\r\n\tat org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\r\n\tat java.lang.Thread.run(Unknown Source)\r\n",
    "code": 500
  }
}

here is the portion of solrconfig dealing with spellcehck feature

    <searchComponent name="spellcheck" class="solr.SpellCheckComponent">

    <str name="queryAnalyzerFieldType">text_general</str>

    <!-- Multiple "Spell Checkers" can be declared and used by this
         component
      -->

    <!-- a spellchecker built from a field of the main index -->
    <lst name="spellchecker">
      <str name="name">default</str>
      <str name="field">content</str>
      <str name="classname">solr.DirectSolrSpellChecker</str>
      <str name="buildOnCommit">true</str>
      <!-- the spellcheck distance measure used, the default is the internal levenshtein -->
      <str name="distanceMeasure">internal</str>
      <!-- minimum accuracy needed to be considered a valid spellcheck suggestion -->
      <float name="accuracy">0.5</float>
      <!-- the maximum #edits we consider when enumerating terms: can be 1 or 2 -->
      <int name="maxEdits">2</int>
      <!-- the minimum shared prefix when enumerating terms -->
      <int name="minPrefix">1</int>
      <!-- maximum number of inspections per result. -->
      <int name="maxInspections">5</int>
      <!-- minimum length of a query term to be considered for correction -->
      <int name="minQueryLength">4</int>
      <!-- maximum threshold of documents a query term can appear to be considered for   correction -->
      <float name="maxQueryFrequency">0.01</float>
      <!-- uncomment this to require suggestions to occur in 1% of the documents
        <float name="thresholdTokenFrequency">.01</float>
      -->
    </lst>

    <!-- a spellchecker that can break or combine words.  See "/spell" handler below for usage -->
    <lst name="spellchecker">
      <str name="name">wordbreak</str>
      <str name="classname">solr.WordBreakSolrSpellChecker</str>      
      <str name="field">content</str>
      <str name="combineWords">true</str>
      <str name="breakWords">true</str>
      <int name="maxChanges">10</int>
    </lst>

    <!-- a spellchecker that uses a different distance measure -->
    <!--
       <lst name="spellchecker">
         <str name="name">jarowinkler</str>
         <str name="field">spell</str>
         <str name="classname">solr.DirectSolrSpellChecker</str>
         <str name="distanceMeasure">
           org.apache.lucene.search.spell.JaroWinklerDistance
         </str>
       </lst>
     -->

    <!-- a spellchecker that use an alternate comparator 

         comparatorClass be one of:
          1. score (default)
          2. freq (Frequency first, then score)
          3. A fully qualified class name
      -->
    <!--
       <lst name="spellchecker">
         <str name="name">freq</str>
         <str name="field">lowerfilt</str>
         <str name="classname">solr.DirectSolrSpellChecker</str>
         <str name="comparatorClass">freq</str>
      -->

    <!-- A spellchecker that reads the list of words from a file -->
    <!--
       <lst name="spellchecker">
         <str name="classname">solr.FileBasedSpellChecker</str>
         <str name="name">file</str>
         <str name="sourceLocation">spellings.txt</str>
         <str name="characterEncoding">UTF-8</str>
         <str name="spellcheckIndexDir">spellcheckerFile</str>
       </lst>
      -->
  </searchComponent>


  <requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
    <lst name="defaults">
      <str name="df">content</str>
      <!-- Solr will use suggestions from both the 'default' spellchecker
           and from the 'wordbreak' spellchecker and combine them.
           collations (re-written queries) can include a combination of
           corrections from both spellcheckers -->
      <str name="spellcheck.dictionary">default</str>
      <str name="spellcheck">on</str>
      <str name="spellcheck.extendedResults">true</str>       
      <str name="spellcheck.count">10</str>
      <str name="spellcheck.alternativeTermCount">5</str>
      <str name="spellcheck.maxResultsForSuggest">5</str>       
      <str name="spellcheck.collate">true</str>
      <str name="spellcheck.collateExtendedResults">true</str>  
      <str name="spellcheck.maxCollationTries">10</str>
      <str name="spellcheck.maxCollations">5</str>         
    </lst>
    <arr name="last-components">
      <str>spellcheck</str>
    </arr>
  </requestHandler>

please help me to fix this. thanks