TODO 4.3 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107
  1. Things yet to do:
  2. softflowd
  3. ---------
  4. - Use strtonum()
  5. Flow tracking engine
  6. - Calculate hash over flow and use it as a key to avoid lots of
  7. cache-trashing comparisons
  8. - Verify checksums (maybe. perhaps bad for accounting, good for flow tracking)
  9. - Fragment processing
  10. - We don't handle fragments right
  11. - This shouldn't be too hard or too memory intensive. We just need to
  12. keep a tree of fragment entries. Each entry would need to contain
  13. enough information to reconstruct the flow (source/dest addr, etc),
  14. but also fragment related info: IP id, list of fragment offsets. etc.
  15. - When we receive a new fragment, we add an entry to this tree (keyed
  16. by source IP, protocol, IP id)
  17. - Each new fragment matched in the tree gets its offset added to the
  18. list, until all fragments have been seen
  19. - Must be careful, as later fragments may arrive before inital one
  20. - When does accounting occur?
  21. - Upon receipt of inital fragment? (and thus for ever frag thereafter)
  22. - When we have seen all fragments? (what if we don't?)
  23. - Must limit size of tree
  24. - Must have fragment timeout (what happens then, apart from removal?)
  25. - Timeouts
  26. - Timeout for unanswered TCP connection
  27. - Ditto orphaned connections (one packet in one direction only)
  28. - Track ICMP generated by TCP/UDP session (painful, probably unecessary)
  29. - More datalink types
  30. - Improve fast-expiry of TCP session by tracking FIN sequence numbers
  31. - Multiple interface support
  32. - Requires some way to avoid duplicate recording of flows (IP ID)
  33. - Track IPsec SPIs
  34. - Track ToS / DSCP
  35. - Make counters 64 bits
  36. - We can report these directly for NetFlow v.9
  37. - For older NetFlow, report by sending multiple flows until counter < 2^32
  38. Misc features
  39. - Ability to open more than one interface (maybe)
  40. - Ability to read more than one pcap file (maybe)
  41. - Fork for ctlsock actions? (don't block mainloop)
  42. - Remote control over network (requires SSL)
  43. Performance
  44. - Profile and see where the hot spots are
  45. - Fast "new flow" test using a bloom filter
  46. - See if we can reduce per-packet overhead more
  47. - Cost of expiry remove and re-add per packet
  48. - Stop run-time malloc (maybe)
  49. - Preallocate a pool of expiry events and flow entries
  50. - keep a queue, pick/push first from head
  51. Exporter features
  52. - sflow support (www.sflow.org)
  53. - Needs XDR encoding
  54. - Ability to export to multiple hosts
  55. - Partly done, just need to keep a list of targets instead of a single one
  56. - Ability to directly write to file (maybe. If so, reuse flowd store code)
  57. - NetFlow v.9 field selection
  58. - Get AS numbers from bgpd and fill in to Netflow packets
  59. Statistics code
  60. - Collect more statistics (maybe)
  61. - Advanced packet analysis: store hash of packet payload, keep
  62. statistics on traffic similarity
  63. - Bloom filter?
  64. - Option to record histograms of
  65. - Flow lifetime and size, packet size
  66. - Flow bandwidth
  67. - Per well-known-port
  68. - How to do this quicky? Memory efficiently?
  69. - Per IP address/range
  70. - How to do this quicky? Memory efficiently?
  71. - Moving averages
  72. - Track traffic over lifetime of flow
  73. - Maintain linked list traffic counts, keyed by time interval
  74. - E.g. key by (now / 300)
  75. - Or (now - start_time) / 300 (better)
  76. - When new packet comes in:
  77. - If timestamp of HEAD of list == (now / xxx), then counter += octets
  78. - Otherwise create new traffic counter at HEAD and update it
  79. - Then trim tail if the list length is too big
  80. - Maybe store "hunks" of data, rather than individual counts in the
  81. list, as storing a single int is a huge waste of space
  82. - Maybe a rrdtool-like heirarchy of timespans
  83. - 300 seconds (5 minutes) (2400 bytes)
  84. - 360 1-minute blocks (6 hours) (2880 bytes)
  85. - 288 10-minute blocks (2 days) (2304 bytes)
  86. - 336 1-hour blocks (2 weeks) (2688 bytes)
  87. - Total 10kb worst-case per-flow (scary, probably overkill)
  88. softflowctl
  89. -----------
  90. - Extend interface
  91. - Query for specific flows (e.g. by address)
  92. - Do this in softflowd or softflowctl?
  93. - Expire/delete specific flows (maybe)
  94. - Runtime respecify timeouts
  95. - Real-time binary dump of flowtable (shm/mmap fd pass?)
  96. - ntop like view
  97. - Spiffy GUI (separate tool)