TODO 4.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104
  1. Things yet to do:
  2. softflowd
  3. ---------
  4. - Use strtonum()
  5. Flow tracking engine
  6. - Verify checksums (maybe. perhaps bad for accounting, good for flow tracking)
  7. - Fragment processing
  8. - We don't handle fragments right
  9. - This shouldn't be too hard or too memory intensive. We just need to
  10. keep a tree of fragment entries. Each entry would need to contain
  11. enough information to reconstruct the flow (source/dest addr, etc),
  12. but also fragment related info: IP id, list of fragment offsets. etc.
  13. - When we receive a new fragment, we add an entry to this tree (keyed
  14. by source IP, protocol, IP id)
  15. - Each new fragment matched in the tree gets its offset added to the
  16. list, until all fragments have been seen
  17. - Must be careful, as later fragments may arrive before inital one
  18. - When does accounting occur?
  19. - Upon receipt of inital fragment? (and thus for ever frag thereafter)
  20. - When we have seen all fragments? (what if we don't?)
  21. - Must limit size of tree
  22. - Must have fragment timeout (what happens then, apart from removal?)
  23. - Timeouts
  24. - Timeout for unanswered TCP connection
  25. - Ditto orphaned connections (one packet in one direction only)
  26. - Track ICMP generated by TCP/UDP session (painful, probably unecessary)
  27. - More datalink types
  28. - Improve fast-expiry of TCP session by tracking FIN sequence numbers
  29. - Multiple interface support
  30. - Requires some way to avoid duplicate recording of flows (IP ID)
  31. - Track IPsec SPIs
  32. - Track ToS / DSCP
  33. - Make counters 64 bits
  34. - We can report these directly for NetFlow v.9
  35. - For older NetFlow, report by sending multiple flows until counter < 2^32
  36. Misc features
  37. - Ability to open more than one interface (maybe)
  38. - Ability to read more than one pcap file (maybe)
  39. - Fork for ctlsock actions? (don't block mainloop)
  40. - Remote control over network (requires SSL)
  41. Performance
  42. - Profile and see where the hot spots are
  43. - Fast "new flow" test using a bloom filter
  44. - See if we can reduce per-packet overhead more
  45. - Cost of expiry remove and re-add per packet
  46. - Stop run-time malloc (maybe)
  47. - Preallocate a pool of expiry events and flow entries
  48. - keep a queue, pick/push first from head
  49. Exporter features
  50. - sflow support (www.sflow.org)
  51. - Needs XDR encoding
  52. - Ability to export to multiple hosts
  53. - Partly done, just need to keep a list of targets instead of a single one
  54. - Ability to directly write to file (maybe. If so, reuse flowd store code)
  55. - NetFlow v.9 field selection
  56. Statistics code
  57. - Collect more statistics (maybe)
  58. - Advanced packet analysis: store hash of packet payload, keep
  59. statistics on traffic similarity
  60. - Bloom filter?
  61. - Option to record histograms of
  62. - Flow lifetime and size, packet size
  63. - Flow bandwidth
  64. - Per well-known-port
  65. - How to do this quicky? Memory efficiently?
  66. - Per IP address/range
  67. - How to do this quicky? Memory efficiently?
  68. - Moving averages
  69. - Track traffic over lifetime of flow
  70. - Maintain linked list traffic counts, keyed by time interval
  71. - E.g. key by (now / 300)
  72. - Or (now - start_time) / 300 (better)
  73. - When new packet comes in:
  74. - If timestamp of HEAD of list == (now / xxx), then counter += octets
  75. - Otherwise create new traffic counter at HEAD and update it
  76. - Then trim tail if the list length is too big
  77. - Maybe store "hunks" of data, rather than individual counts in the
  78. list, as storing a single int is a huge waste of space
  79. - Maybe a rrdtool-like heirarchy of timespans
  80. - 300 seconds (5 minutes) (2400 bytes)
  81. - 360 1-minute blocks (6 hours) (2880 bytes)
  82. - 288 10-minute blocks (2 days) (2304 bytes)
  83. - 336 1-hour blocks (2 weeks) (2688 bytes)
  84. - Total 10kb worst-case per-flow (scary, probably overkill)
  85. softflowctl
  86. -----------
  87. - Extend interface
  88. - Query for specific flows (e.g. by address)
  89. - Do this in softflowd or softflowctl?
  90. - Expire/delete specific flows (maybe)
  91. - Runtime respecify timeouts
  92. - Real-time binary dump of flowtable (shm/mmap fd pass?)
  93. - ntop like view
  94. - Spiffy GUI (separate tool)