Bug #1639
Updated by Luke Murphey over 7 years ago
|_. inputs.conf |_. search bnf |_. web_scrape.py |_. | _. web controller |
| url | url | url | url XXXX |
| interval | (excluded) | (excluded) | (excluded) XXXX |
| title | (excluded) | (excluded) | (excluded) XXXX |
| selector | selector | selector | selector XXXX |
| username | username | username | username XXXX |
| password | password | password | password XXXX |
| name_attributes | | name_attributes | name_attributes XXXX |
| user_agent | | | user_agent XXXX |
| use_element_name | use_element_name | use_element_name | use_element_name XXXX |
| page_limit | page_limit | page_limit | page_limit XXXX |
| url_filter | url_filter | url_filter | url_filter XXXX |
| depth_limit | depth_limit | depth_limit | depth_limit XXXX |
| raw_content | include_raw_content | include_raw_content | raw_content -> include_raw_content XXXX |
| text_separator | text_separator | text_separator | text_separator XXXX |
| browser | browser | browser | browser XXXX |
| timeout | timeout | timeout | timeout XXXX |
| output_as_mv | output_as_mv | *output_matches_as_mv* | output_as_mv -> output_matches_as_mv XXXX |
| | | output_matches_as_separate_fields | |
| | | include_empty_matches | include_empty_matches |
| | | match_prefix | match_prefix |
| | | proxy* | |
Based on this analysis, the following changes are needed:
* Add user-agent to the search BNF
* Correct output_matches_as_mv in search command
* Change search command to take raw_content instead of include_raw_content
* Add name_attributes to search command