README 4.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141
  1. INTRODUCTION
  2. ------------
  3. The vblade is the virtual EtherDrive (R) blade, a program that makes a
  4. seekable file available over an ethernet local area network (LAN) via
  5. the ATA over Ethernet (AoE) protocol.
  6. The seekable file is typically a block device like /dev/md0 but even
  7. regular files will work. Sparse files can be especially convenient.
  8. When vblade exports the block storage over AoE it becomes a storage
  9. target. Another host on the same LAN can access the storage if it has
  10. a compatible aoe kernel driver.
  11. BUILDING
  12. --------
  13. The following command should build the vblade program on a Linux-based
  14. system:
  15. make
  16. For FreeBSD systems, include an extra parameter like so:
  17. make PLATFORM=freebsd
  18. EXAMPLES
  19. --------
  20. There is a "vbladed" script that daemonizes the program and sends its
  21. output to the logger program. Make sure you have logger installed if
  22. you would like to run vblade as a daemon with the vbladed script.
  23. ecashin@kokone vblade$ echo 'I have logger' | logger
  24. ecashin@kokone vblade$ tail -3 /var/log/messages
  25. Feb 8 14:52:49 kokone -- MARK --
  26. Feb 8 15:12:49 kokone -- MARK --
  27. Feb 8 15:19:56 kokone logger: I have logger
  28. Here is a short example showing how to export a block device with a
  29. vblade. (This is a loop device backed by a sparse file, but you could
  30. use any seekable file instead of /dev/loop7.)
  31. ecashin@kokone vblade$ make
  32. cc -Wall -c -o aoe.o aoe.c
  33. cc -Wall -c -o linux.o linux.c
  34. cc -Wall -c -o ata.o ata.c
  35. cc -o vblade aoe.o linux.o ata.o
  36. ecashin@kokone vblade$ su
  37. Password:
  38. root@kokone vblade# modprobe loop
  39. root@kokone vblade# dd if=/dev/zero bs=1k count=1 seek=`expr 1024 \* 4096` of=bd
  40. -file
  41. 1+0 records in
  42. 1+0 records out
  43. 1024 bytes transferred in 0.009901 seconds (103423 bytes/sec)
  44. root@kokone vblade# losetup /dev/loop7 bd-file
  45. root@kokone vblade# ./vblade 9 0 eth0 /dev/loop7
  46. ioctl returned 0
  47. 4294968320 bytes
  48. pid 16967: e9.0, 8388610 sectors
  49. Here's how you can use the Linux aoe driver to access the storage from
  50. another host on the LAN.
  51. ecashin@kokone ecashin$ ssh makki
  52. Last login: Mon Feb 7 10:25:04 2005
  53. ecashin@makki ~$ su
  54. Password:
  55. root@makki ecashin# modprobe aoe
  56. root@makki ecashin# aoe-stat
  57. e9.0 eth1 up
  58. root@makki ecashin# mkfs -t ext3 /dev/etherd/e9.0
  59. mke2fs 1.35 (28-Feb-2004)
  60. ...
  61. Creating journal (8192 blocks): done
  62. Writing superblocks and filesystem accounting information: done
  63. This filesystem will be automatically checked every 24 mounts or
  64. 180 days, whichever comes first. Use tune2fs -c or -i to override.
  65. root@makki ecashin# mkdir /mnt/e9.0
  66. root@makki ecashin# mount /dev/etherd/e9.0 /mnt/e9.0
  67. root@makki ecashin# echo hooray > /mnt/e9.0/test.txt
  68. root@makki ecashin# cat /mnt/e9.0/test.txt
  69. hooray
  70. Remember: be as careful with these devices as you would with /dev/hda!
  71. Jumbo Frame Compatibility
  72. -------------------------
  73. Vblade can use jumbo frames provided your initiator is jumbo frame
  74. capable. There is one small configuration gotcha to consider
  75. to avoid having the vblade kernel frequently drop frames.
  76. Vblade uses a raw socket to perform AoE. The linux kernel will
  77. only buffer a certain amount of data for a raw socket. For 2.6
  78. kernels, this value is managed through /proc:
  79. root@nai aoe# grep . /proc/sys/net/core/rmem_*
  80. /proc/sys/net/core/rmem_default:128000
  81. /proc/sys/net/core/rmem_max:128000
  82. rmem_max is the max amount a user process may expand the receive
  83. buffer to -- through setsockopt(...) -- and rmem_default is, as you
  84. might expect, the default.
  85. The gotcha is that this amount to buffer does not relate
  86. to the amount of user data buffered, but the amount of
  87. real data buffered. As an example, the Intel GbE controller
  88. must be given 16KB frames to use an MTU over 8KB.
  89. For each received frame, the kernel must be able to buffer
  90. 16KB, even if the aoe frame is only 60 bytes in length.
  91. The linux aoe initiator will use 16 outstanding frames when
  92. used with vblade. A good default for ensuring frames are
  93. not dropped is to allocate 16KB for 17 frames:
  94. for f in /proc/sys/net/core/rmem_*; do echo $((17 * 16 * 1024)) >$f; done
  95. Be sure to start vblade after changing the buffering defaults
  96. as the buffer value is set when the socket is opened.
  97. AoE Initiator Compatibility
  98. ---------------------------
  99. The Linux aoe driver for the 2.6 kernel is compatible if you use
  100. aoe-2.6-7 or newer. You can use older aoe drivers but you will only
  101. be able to see one vblade per MAC address.
  102. Contrib Patches
  103. ---------------
  104. see contrib/README
  105. Kvblade
  106. -------
  107. While vblade runs as a userland process (like "ls" or "vi"), there
  108. is another program that runs inside the kernel. It is called
  109. kvblade. It is alpha software.