Project

General

Profile

Bug #2225

Input doesn't respect any sort of size limit

Added by Luke Murphey over 6 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Luke Murphey
Category:
Input: Web Spider
Target version:
4.5.2
Start date:
03/12/2018
Due date:
% Done:

100%

Description

The input doesn't seem to respect any sort of size limit.

This may be hard to implement in mechanize. See http://wwwsearch.sourceforge.net/mechanize/doc.html

See test TestWebInput.test_scrape_page_download_limit for proof.

Associated revisions

Revision 651 (diff)
Added by lukemurphey over 6 years ago

Fixing issue where inputs could fail when handling large files

Reference #2225

History

#1 Updated by Luke Murphey over 6 years ago

Questions:
  1. Can read() take a length argument?
    1. Yes, it can
import mechanize
opener = mechanize.OpenerFactory(mechanize.SeekableResponseOpener).build_opener()
response = opener.open("http://127.0.0.1:8080/bigfile")
response.read(100)
response.close()

#2 Updated by Luke Murphey over 6 years ago

Test via this:

ant test.run_unit -Dtest=TestWebInput.test_scrape_page_download_limit

#3 Updated by Luke Murphey over 6 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF