Project

General

Profile

Bug #2861

High memory usage importing a large lookup file

Added by Luke Murphey over 3 years ago. Updated over 3 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
12/03/2020
Due date:
% Done:

0%


Description

3.4.6 the customer is getting issues uploading 40MB files but on 3.3.2 they are not

History

#1 Updated by Luke Murphey over 3 years ago

An 11 MB lookup doesn't show dramatic memory use. However, importing several times seems to cause memory use to increase each time.

#2 Updated by Luke Murphey over 3 years ago

Qs:
  • (No) Does the issue repro on the old version?
    • Trying 3.3.3 which has the older library
    • That worked waaaaay better (much faster and far less memory usage)
  • Exactly what version does this repro on (when did it start)?
    • 3.3.3
  • How does the import work?
  • (No) Is this limited to some particular version of Splunk?
    • Seems to be related to the app version
  • Is there a new version of the underyling library?
Obs:
  • Mem usage tops out for me at ~12 GB but it doesn't seem to complete after that

Also available in: Atom PDF