Recently i am working on a problem like this too. I couldn’t find any formal documentation about this issue but when i investigate the results with “explain api”, it seems like “queryNorm” is not applied to the score directly coming from “functions” field. This means that you can not directly normalize script value. However, i … Read more
As you said some-word -> [doc385, doc211, doc39977, …] (sorted by rank, descending), I think the search engine may not do this, the doc list should be sorted by doc ID, each doc has a rank according to the word. When a query comes, it contains several keywords. For each word, you can find a … Read more
Without understanding the meaning of the words like @DrYap suggests, the next logical unit to compare two words (if you are not looking for misspellings) is syllables. It is very easy to modify Levenshtein to compare syllables instead of characters. The hard part is breaking the words into syllables. There is a Java implementation TeXHyphenator-J … Read more
There are several parts to a search engine. Broadly speaking, in a hopelessly general manner (folks, feel free to edit if you feel you can add better descriptions, links, etc): The crawler. This is the part that goes through the web, grabs the pages, and stores information about them into some central data store. In … Read more
First, I recreated your current configuration in Play: https://www.found.no/play/gist/867785a709b4869c5543 If you go there, switch to the “Analysis”-tab to see how the text is transformed: Note, for example that Heaney ends up tokenized as [hn, heanei] with the search_analyzer and as [HN, heanei] with the index_analyzer. Note the case-difference for the metaphone-term. Thus, that one is … Read more
I suggest you read Full Text Search Engines vs. DBMS. A one-liner would be: If the bulk of your use case is full text search, use Lucene. If the bulk of your use case is joins and other relational operations, use a database. You may use a hybrid solution for a more complicated use case.
Let me write here an answer from the superuser.com question due to my complete solidarity with the author: quote from the Ask Metafilter: The only possible way to make keyword searching efficient over hundreds of terabytes (or whatever their index is up to these days) is to precompute an index of words. In fact a … Read more
No, we (Google) won’t see the content behind tabs iff the content under the tab is dynamically generated (i.e. not just hidden). You can also see what we “see” using Fetch as Google in Search Console (former Webmaster Tools); read more about the feature in our post titled Rendering pages with Fetch as Google.
Just an update to my earlier answer – since there was so much discussion I didn’t want this update to get lost. You can download it here:
Short answer: it depends on your use case, but you probably don’t want to use it as a primary store. Longer answer: You should really understand all of the possible issues that can come up around resiliency and data loss. Elastic has some great documentation of these issues which you should really understand before using … Read more