TextCritical.net

• The match counts represents the number of matching verses, not matches within the verses. See (ὅτι OR ἔργων) (section:"Galatians 2").
• The variations are throwing off the results count

Some solutions:

1. Kick off a different search that introspects the results and looks up the results manually.

See https://whoosh.readthedocs.org/en/latest/api/reading.html and http://stackoverflow.com/questions/35565900/how-do-i-get-the-list-of-all-terms-in-a-whoosh-index.

I wonder if term_info(fieldname, text) can do what I want it to do. It has a max_weight() function that may indicate I could use weights to get these counts. weight() can be used to find the frequency in all documents.

I could list the terms that did not match with the following:

q.all_terms() - results.terms()

There is a score() function that could likely indicate the total raw number of matches.

See http://stackoverflow.com/questions/35591302/how-do-i-get-the-bag-of-words-representation-of-document-content-with-whoosh

This will get the bag-of-words (it needs vector=True in the schema):

from reader.contentsearch import *
inx = WorkIndexer.get_index()
searcher = inx.searcher()

from reader.models import *
vs = Verse.objects.filter(division__work__title_slug='new-testament')
docnum = searcher.document_number(verse_id=vs[0].id)
g = searcher.vector(docnum,"content").items_as("frequency")

for h in g:
    print h[0], h[1]

searcher.idf("no_diacritics", "και") # Returns 1.4421964751262426
searcher.idf("no_diacritics", "κυριος") # Returns 4.8326777289115554

I wonder if I could use a collector for this: search_with_collector (http://whoosh.readthedocs.org/en/latest/api/searching.html)

python manage.py make_search_indexes -w new-testament -c

I'm starting to think I should implement a word summary on the morphological dialog that lists:

• Count of this word in the current division
• Count of this word in the current work
• Count of this word's related forms in the current division
• Count of this word's related forms in the current work

I could use document_numbers() to get a list of the documents within a given division or work and then look for the related words:

for doc in searcher.document_numbers(work="new-testament"):
    print doc

The following is a good test case:

work:"New Testament" section:"Galatians" νόμον

With stored=False, the search indexes for the New Testament is 18.7 MB.

With store=True, they are 20.1 MB.