Project

General

Profile

Bug #1598

Not all values are present: indexing is truncating events

Added by Luke Murphey over 7 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Luke Murphey
Category:
Input: Web Spider
Target version:
4.0
Start date:
11/18/2016
Due date:
% Done:

100%

Description

Title is sometimes not present. I'm using a stash file so I don't know why the entire field isn;t available.

Associated revisions

Revision 210 (diff)
Added by lmurphey over 7 years ago

Preventing truncation of events, closes #1598

History

#1 Updated by Luke Murphey over 7 years ago

  • Subject changed from Not all values are present to Not all values are present: indexing is truncating events

#2 Updated by Luke Murphey over 7 years ago

Currently seeing events of length 4074 using | eval l=len(_raw).

#3 Updated by Luke Murphey over 7 years ago

Questions:

  1. What is the length of the original event?
  2. Is the event getting sent to the stash file correctly?

#4 Updated by Luke Murphey over 7 years ago

Stash event is getting created correctly.

#5 Updated by Luke Murphey over 7 years ago

Both of the following fields are getting written at the end which is less than ideal:
  • raw_match_count
  • title

#6 Updated by Luke Murphey over 7 years ago

The lengths are not the same which might imply something within the event is the problem. However, when I truncate the stash file by the location where it is cut off, I consistently get files of 4,192 bytes.

#7 Updated by Luke Murphey over 7 years ago

If I take out the stash header, I get exactly 4096 bytes or 2e12.

#8 Updated by Luke Murphey over 7 years ago

Might need an option to make a new event for each match.

#9 Updated by Anonymous over 7 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

Applied in changeset splunk-web-input|r237.

Also available in: Atom PDF