Project

General

Profile

Bug #2888

Including lots of entries despite the option to only include the latest

Added by Luke Murphey about 2 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Start date:
03/08/2022
Due date:
% Done:

100%

History

#1 Updated by Luke Murphey about 2 years ago

Qs:
  • Does this work with other feeds?
  • Do the logs show that it is skipping some?
    • Yes, 1,466 are skipped
  • What is the parsed time?
    • Entries with 1646805098.0 are being included
    • Current time is 1646776990.032829
    • Splunk is parsing this time as the correct time
      import time
      last_entry_date = time.localtime()
      time.mktime(last_entry_date)
      
    • That returns 1646777175.0
  • What date is the input getting from the feed?
    • Latest date from feed retrieved, last_entry_date_retrieved=time.struct_time(tm_year=2022, tm_mon=3, tm_mday=8, tm_hour=10, tm_min=50, tm_sec=0, tm_wday=1, tm_yday=67, tm_isdst=0)
    • 1646765400
    • This isn't the latest time though
  • Are the entries stopping to arrive now?
    • 29,300 entries now
    • So this seems to be working now
Obs:
  • get_updated_date() gets the date
  • The date in the feed is published in pubDate
Solns:
  • The TZ is not being parsed correctly

#2 Updated by Luke Murphey about 2 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF