Feature #2140
Allow high volume of pings in the modular input
Start date:
11/09/2017
Due date:
% Done:
100%
Associated revisions
Updating the modular input to handle multiple threads
Reference #2140
History
#1 Updated by Luke Murphey about 7 years ago
I'm thinking I should migrate the threading functions into the base class.
What I could move:- Thread limiting check
- Threads list
- Thread limit var
- Thread cleanup function
- App config getter system
#2 Updated by Luke Murphey about 7 years ago
input_template = '''
[ping://192.168.0.%s]
hosts = 192.168.0.%s
interval = 30s
runs = 1
'''
inputs = ''
for i in range(1, 201):
inputs += input_template % (i, i)
f = file('/Users/lmurphey/Desktop/inputs.conf', 'w')
f.write(inputs)
f.close()
#3 Updated by Luke Murphey about 7 years ago
Observations:
- The current design multi-threads the inputs but not the hosts within a single input
- I see lots of logs noting that the thread limit was reached. This happens because the input goes over each input and tries to check the thread limit again.
#4 Updated by Luke Murphey about 7 years ago
According to #2135, I need to support about 8 inputs per second.
#5 Updated by Luke Murphey about 7 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
#6 Updated by Luke Murphey about 7 years ago
This input was able to handle a thread limit of 200 and was able to keep up with 200 inputs that were set to run every 30 seconds. The memory use is around 37 MB and I am seeing around 900 results per minute (15 per second).