Project

General

Profile

Feature #1748

Add parsing of JSON fields

Added by Luke Murphey about 7 years ago. Updated about 7 years ago.

Status:
New
Priority:
Low
Assignee:
Category:
Input: Web Spider
Target version:
-
Start date:
02/16/2017
Due date:
% Done:

0%


Description

Might want to add the option to dump JSON output into a set of fields.

This would be useful for doing things like dumping pypi package information (e.g. https://pypi.python.org/pypi/pyrad/json).


Related issues

Related to Website Input - Feature #1168: Output raw data Closed 05/18/2016

History

#1 Updated by Luke Murphey about 7 years ago

Another option would be outputting the output as raw JSON. I'm not actually sure this is possible though because I need to include some things like the index and sourcetype.

I might be able to use the event_writer to make stash files to do this.

#3 Updated by Luke Murphey about 7 years ago

#4 Updated by Luke Murphey about 7 years ago

I want to see if I can just dump the JSON directly.

StashNewWriter::event_to_string() Assumes that the data is a list of fields. I strip the sourcetype using the transform "sinkhole_web_input_header". The index and source come from the stash line name so these should be good already.

#5 Updated by Luke Murphey about 7 years ago

To do this:

  1. Add the option to the modular input page
  2. Add the option to the wizard page
  3. Make the search command use the option
  4. Make sure the preview window uses the option too
  5. Add the option to the search BNF
  6. Add the option to inputs.conf.spec
  7. Add test cases

#6 Updated by Luke Murphey about 7 years ago

If I output raw data, then I don't need to selector page at all.

#7 Updated by Luke Murphey about 7 years ago

  • Target version deleted (4.1)

#8 Updated by Luke Murphey about 7 years ago

  • Priority changed from Normal to Low

Also available in: Atom PDF