Bug #2600
Search fails
Start date:
02/11/2020
Due date:
% Done:
100%
Associated revisions
Fixing search issues
Reference #2600
Fixing issue where bytes stored in the index causes issues
Reference #2600
History
#1
Updated by Luke Murphey over 5 years ago
Not Found: /api/search/work:berean-bible section:"Genesis 1" beginning
Performing a search, page=1, page_len=10, include_related_forms=False, search_query="work:berean-bible beginning"
Search query parsed, default_search_field="content", raw_query="(work:<berean-bible> AND content:<beginning>)"
Looking for variations of the search term in order to perform a search, word=beginning
Unable to find work in matched terms, work_slug=b'berean-bible'
Performing a stats search, limit=2000, include_related_forms=False, search_query="work:berean-bible beginning"
Search query parsed, default_search_field="content", raw_query="(work:<berean-bible> AND content:<beginning>)"
Looking for variations of the search term in order to perform a search, word=beginning
Internal Server Error: /api/search/work:berean-bible beginning
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.7/site-packages/django/utils/decorators.py", line 149, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/usr/src/app/reader/views.py", line 857, in api_search
return render_api_response(request, results_set)
File "/usr/src/app/reader/views.py", line 636, in render_api_response
raw_content = json.dumps(content)
File "/usr/local/lib/python3.7/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/local/lib/python3.7/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.7/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
TypeError: keys must be str, int, float, bool or None, not bytes
Qs:
- How am I handling the unicode with the search functionality?
- [Found] What field is wrong in views.py?
- matched_terms_verses
- matched_terms_no_diacritics
- [None] What field in search_stats() is wrong?
- stats['matches']
- stats['matched_works']
- stats['matched_terms']
- stats['results_count']
- [Found] What is the line where the failure is?
- [No] Do my tests show that string normalization returns bytes?
- [No] Do any of my tests return bytes?
- Search indexing providing bytes?
- A search for the following works when diacritics are set to be ignored: work:berean-bible "the beginning"
- The issue likely comes from:
- VerseSearchResults::__init__()
- results parameter comes from searcher.search_page()
- This comes from Whoosh. It seems like Whoosh is returning bytes
- I commented out matched_terms and matched_terms_no_diacritics in VerseSearchResults::__init__(); this didn't seem to change functionality?!
- It turns out that I'm not even using this functionality
- results parameter comes from searcher.search_page()
- VerseSearchResults::__init__()
- This looks similar: https://stackoverflow.com/questions/59006409/haystack-whoosh-convert-string-to-bytes-like-object-cannot-use-a-string-patter
- The stack is:
- src/reader/views.py::api_search
- src/reader/views.py::search_verses
- VerseSearchResults::__init__()
- searcher.search_page()
#2
Updated by Luke Murphey over 5 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100