Bug #1739: Figure why selector doesn't match - Website Input - LukeMurphey.net

Bug #1739

Figure why selector doesn't match

Added by Luke Murphey almost 8 years ago. Updated over 7 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Luke Murphey

Category:

Input: Web Spider

Target version:

4.1

Start date:

02/11/2017

Due date:

% Done:

100%

Description

Try a selector of "#siteTable .title" on https://www.reddit.com/r/Splunk/new/. It shows matches and it matches in a browser manually. However, it produces no results.

Associated revisions

Revision 357 (diff)
Added by lukemurphey over 7 years ago

Fixing no matches on some IDs

Fixing no matches on selectors using non-lowercase IDs

Reference #1739

History

#1 Updated by Luke Murphey almost 8 years ago

Repro:

| webscrape selector="#siteTable .title" url="https://www.reddit.com/r/Splunk/new/"

This doesn't work either:

| webscrape selector="#siteTable .title" url="https://www.reddit.com/r/Splunk/new/" browser="firefox"

#2 Updated by Luke Murphey over 7 years ago

To check:

Is the selector getting passed through
Can this be reproduced via a simple Python script?

#3 Updated by Luke Murphey over 7 years ago

I cannot match on #siteTable.

#4 Updated by Luke Murphey over 7 years ago

The issue is that I am making the selector lowercase in the SelectorField class. This is because CSS selectors ought to match without respect to case ("DIV" and "div" should match identically).

See http://stackoverflow.com/questions/1734125/is-it-possible-for-lxml-to-work-in-a-case-insensitive-manner