Feature #1726
Add ability to include empty matches
100%
Description
Adding the ability to include empty matches would help in cases where I am using two selectors and I need to include empty matches so that I can match up the results accordingly (like in cases where the the results come in pairs).
For example, use the selector "td:nth-child(1), td:nth-child(5)" on http://downdetector.com/top10/. Notice how empty rows are not included making it difficult to know what trend the site is for.
Associated revisions
Adding ability to include empty matches
Reference #1726
Adding ability to include empty matches
Reference #1726
Updating unit tests to test empty matching
Reference #1726
History
#1 Updated by Luke Murphey almost 8 years ago
Make sure to check:
- The manager page
- The main page: that it gets loaded and saved
- Preview works
- Search command works
#2 Updated by Luke Murphey almost 8 years ago
- % Done changed from 0 to 50
#3 Updated by Luke Murphey almost 8 years ago
empty_matches only prevents entire fields that are blank from being included; it doesn't include blank matches between the separator, like with this example:
| webscrape selector="tr" url="http://downdetector.com/top10/" depth_limit=25 text_separator="::" empty_matches=1
I'm also not seeing the empty fields in the results and in the search command output.
#4 Updated by Luke Murphey almost 8 years ago
I can get empty strings to be included with text separators in get_text() by not using append_if_not_empty().
#5 Updated by Luke Murphey almost 8 years ago
I'm seeing that the raw_match_count is the expected value. It appears the values are getting stripped before being outputted somehow.
#6 Updated by Luke Murphey almost 8 years ago
Need to fix btool errors too.
#7 Updated by Luke Murphey almost 8 years ago
It looks like it still might not be including the empty matches:
| webscrape selector="tr.dot > td" url="http://downdetector.com/top10/" depth_limit=25 empty_matches=1 | table raw_match_count match
Note that the raw match count is less than the actual.
#8 Updated by Luke Murphey almost 8 years ago
get_result_single() seems to be including all of the entries. It appears that Splunk is filtering them in splunk.Intersplunk.outputResults().
#9 Updated by Luke Murphey almost 8 years ago
I might want to switch to have a default value when no value is found (like fullnull). It could be called: null_value, value_for_empty, empty_value.
#10 Updated by Luke Murphey almost 8 years ago
Need to still:
- Add the option to the modular input page
- Add the option to the wizard page
- Make the search command use the option
- Make sure the preview window uses the option too
- Add the option to the search BNF
- Add the option to inputs.conf.spec
- Add test cases
#11 Updated by Luke Murphey almost 8 years ago
- Status changed from New to Closed
- % Done changed from 50 to 100