Project

General

Profile

Task #2501

Task #2484: Python 3 support

WebClient is expected to return either binary string or Unicode

Added by Luke Murphey over 4 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Luke Murphey
Category:
Input: Web Spider
Target version:
4.5.8
Start date:
11/07/2019
Due date:
% Done:

100%

Associated revisions

Revision 732 (diff)
Added by lmurphey over 4 years ago

Moving the content decoding functionality into the web-client

Reference #2501

History

#1 Updated by Luke Murphey over 4 years ago

Options:

  1. Return binary consistently
  2. Return whatever the underlying libs return
  3. Return Unicode consistently
  4. Return str consistently

#2 Updated by Luke Murphey over 4 years ago

Questions:
  • What do the clients return natively?
    • MechanizeClient: seems to return binary_string
  • Should I move the content decoding to the client?
    • Its in get_result_single() currently

#3 Updated by Luke Murphey over 4 years ago

ant test -Dtest=TestWebInput

     [exec] ======================================================================
     [exec] ERROR: test_scrape_page_include_empty_matches (__main__.TestWebInput)
     [exec] ----------------------------------------------------------------------
     [exec] Traceback (most recent call last):
     [exec]   File "/Users/lmurphey/git/splunk-web-input/tests/unit_test_web_server.py", line 89, in _decorator
     [exec]     return func(self, *args, **kwargs)
     [exec]   File "unit.py", line 423, in test_scrape_page_include_empty_matches
     [exec]     self.assertEqual(result['match'][0], ',,,Text_1,,Text_2,,,')
     [exec] KeyError: 'match'
     [exec] 
     [exec] ----------------------------------------------------------------------

Questions:
  • Is this browser specific?
    • export TEST_BROWSERS=""; ant test -Dtest=TestWebInput
    • No

#4 Updated by Luke Murphey over 4 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF