Bug #2600
Search fails
Start date:
02/11/2020
Due date:
% Done:
100%
Associated revisions
Fixing search issues
Reference #2600
Fixing issue where bytes stored in the index causes issues
Reference #2600
History
#1 Updated by Luke Murphey almost 5 years ago
Not Found: /api/search/work:berean-bible section:"Genesis 1" beginning Performing a search, page=1, page_len=10, include_related_forms=False, search_query="work:berean-bible beginning" Search query parsed, default_search_field="content", raw_query="(work:<berean-bible> AND content:<beginning>)" Looking for variations of the search term in order to perform a search, word=beginning Unable to find work in matched terms, work_slug=b'berean-bible' Performing a stats search, limit=2000, include_related_forms=False, search_query="work:berean-bible beginning" Search query parsed, default_search_field="content", raw_query="(work:<berean-bible> AND content:<beginning>)" Looking for variations of the search term in order to perform a search, word=beginning Internal Server Error: /api/search/work:berean-bible beginning Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python3.7/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/usr/src/app/reader/views.py", line 857, in api_search return render_api_response(request, results_set) File "/usr/src/app/reader/views.py", line 636, in render_api_response raw_content = json.dumps(content) File "/usr/local/lib/python3.7/json/__init__.py", line 231, in dumps return _default_encoder.encode(obj) File "/usr/local/lib/python3.7/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/local/lib/python3.7/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) TypeError: keys must be str, int, float, bool or None, not bytesQs:
- How am I handling the unicode with the search functionality?
- [Found] What field is wrong in views.py?
- matched_terms_verses
- matched_terms_no_diacritics
- [None] What field in search_stats() is wrong?
- stats['matches']
- stats['matched_works']
- stats['matched_terms']
- stats['results_count']
- [Found] What is the line where the failure is?
- [No] Do my tests show that string normalization returns bytes?
- [No] Do any of my tests return bytes?
- Search indexing providing bytes?
- A search for the following works when diacritics are set to be ignored: work:berean-bible "the beginning"
- The issue likely comes from:
- VerseSearchResults::__init__()
- results parameter comes from searcher.search_page()
- This comes from Whoosh. It seems like Whoosh is returning bytes
- I commented out matched_terms and matched_terms_no_diacritics in VerseSearchResults::__init__(); this didn't seem to change functionality?!
- It turns out that I'm not even using this functionality
- results parameter comes from searcher.search_page()
- VerseSearchResults::__init__()
- This looks similar: https://stackoverflow.com/questions/59006409/haystack-whoosh-convert-string-to-bytes-like-object-cannot-use-a-string-patter
- The stack is:
- src/reader/views.py::api_search
- src/reader/views.py::search_verses
- VerseSearchResults::__init__()
- searcher.search_page()
#2 Updated by Luke Murphey almost 5 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100