magic.man 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373
  1. .TH MAGIC __FSECTION__ "Public Domain"
  2. .\" install as magic.4 on USG, magic.5 on V7 or Berkeley systems.
  3. .SH NAME
  4. magic \- file command's magic number file
  5. .SH DESCRIPTION
  6. This manual page documents the format of the magic file as
  7. used by the
  8. .BR file (__CSECTION__)
  9. command, version __VERSION__.
  10. The
  11. .BR file
  12. command identifies the type of a file using,
  13. among other tests,
  14. a test for whether the file begins with a certain
  15. .IR "magic number" .
  16. The file
  17. .I __MAGIC__
  18. specifies what magic numbers are to be tested for,
  19. what message to print if a particular magic number is found,
  20. and additional information to extract from the file.
  21. .PP
  22. Each line of the file specifies a test to be performed.
  23. A test compares the data starting at a particular offset
  24. in the file with a 1-byte, 2-byte, or 4-byte numeric value or
  25. a string.
  26. If the test succeeds, a message is printed.
  27. The line consists of the following fields:
  28. .IP offset \w'message'u+2n
  29. A number specifying the offset, in bytes, into the file of the data
  30. which is to be tested.
  31. .IP type
  32. The type of the data to be tested.
  33. The possible values are:
  34. .RS
  35. .IP byte \w'message'u+2n
  36. A one-byte value.
  37. .IP short
  38. A two-byte value (on most systems) in this machine's native byte order.
  39. .IP long
  40. A four-byte value (on most systems) in this machine's native byte order.
  41. .IP string
  42. A string of bytes.
  43. The string type specification can be optionally followed
  44. by /[Bbc]*.
  45. The ``B'' flag compacts whitespace in the target, which must
  46. contain at least one whitespace character.
  47. If the magic has
  48. .I n
  49. consecutive blanks, the target needs at least
  50. .I n
  51. consecutive blanks to match.
  52. The ``b'' flag treats every blank in the target as an optional blank.
  53. Finally the ``c'' flag, specifies case insensitive matching: lowercase
  54. characters in the magic match both lower and upper case characters in the
  55. target, whereas upper case characters in the magic, only much uppercase
  56. characters in the target.
  57. .IP date
  58. A four-byte value interpreted as a UNIX date.
  59. .IP ldate
  60. A four-byte value interpreted as a UNIX-style date, but interpreted as
  61. local time rather than UTC.
  62. .IP beshort
  63. A two-byte value (on most systems) in big-endian byte order.
  64. .IP belong
  65. A four-byte value (on most systems) in big-endian byte order.
  66. .IP bedate
  67. A four-byte value (on most systems) in big-endian byte order,
  68. interpreted as a Unix date.
  69. .IP leshort
  70. A two-byte value (on most systems) in little-endian byte order.
  71. .IP lelong
  72. A four-byte value (on most systems) in little-endian byte order.
  73. .IP ledate
  74. A four-byte value (on most systems) in little-endian byte order,
  75. interpreted as a UNIX date.
  76. .IP leldate
  77. A four-byte value (on most systems) in little-endian byte order,
  78. interpreted as a UNIX-style date, but interpreted as local time rather
  79. than UTC.
  80. .IP regex
  81. A regular expression match in extended POSIX regular expression syntax
  82. (much like egrep).
  83. The type specification can be optionally followed by
  84. .B /c
  85. for case-insensitive matches.
  86. The regular expression is always
  87. tested against the first
  88. .B N
  89. lines, where
  90. .B N
  91. is the given offset, thus it
  92. is only useful for (single-byte encoded) text.
  93. .B ^
  94. and
  95. .B $
  96. will match the beginning and end of individual lines, respectively,
  97. not beginning and end of file.
  98. .IP search
  99. A literal string search starting at the given offset. It must be followed by
  100. .B /<number>
  101. which specifies how many matches shall be attempted (the range).
  102. This is suitable for searching larger binary expressions with variable
  103. offsets, using
  104. .B \e
  105. escapes for special characters.
  106. .RE
  107. .PP
  108. The numeric types may optionally be followed by
  109. .B &
  110. and a numeric value,
  111. to specify that the value is to be AND'ed with the
  112. numeric value before any comparisons are done.
  113. Prepending a
  114. .B u
  115. to the type indicates that ordered comparisons should be unsigned.
  116. .IP test
  117. The value to be compared with the value from the file.
  118. If the type is
  119. numeric, this value
  120. is specified in C form; if it is a string, it is specified as a C string
  121. with the usual escapes permitted (e.g. \en for new-line).
  122. .IP
  123. Numeric values
  124. may be preceded by a character indicating the operation to be performed.
  125. It may be
  126. .BR = ,
  127. to specify that the value from the file must equal the specified value,
  128. .BR < ,
  129. to specify that the value from the file must be less than the specified
  130. value,
  131. .BR > ,
  132. to specify that the value from the file must be greater than the specified
  133. value,
  134. .BR & ,
  135. to specify that the value from the file must have set all of the bits
  136. that are set in the specified value,
  137. .BR ^ ,
  138. to specify that the value from the file must have clear any of the bits
  139. that are set in the specified value, or
  140. .BR x ,
  141. to specify that any value will match.
  142. If the character is omitted, it is assumed to be
  143. .BR = .
  144. For all tests except
  145. .B string
  146. and
  147. .B regex,
  148. operation
  149. .BR !
  150. specifies that the line matches if the test does
  151. .B not
  152. succeed.
  153. .IP
  154. Numeric values are specified in C form; e.g.
  155. .B 13
  156. is decimal,
  157. .B 013
  158. is octal, and
  159. .B 0x13
  160. is hexadecimal.
  161. .IP
  162. For string values, the byte string from the
  163. file must match the specified byte string.
  164. The operators
  165. .BR = ,
  166. .B <
  167. and
  168. .B >
  169. (but not
  170. .BR & )
  171. can be applied to strings.
  172. The length used for matching is that of the string argument
  173. in the magic file.
  174. This means that a line can match any string, and
  175. then presumably print that string, by doing
  176. .B >\e0
  177. (because all strings are greater than the null string).
  178. .IP message
  179. The message to be printed if the comparison succeeds. If the string
  180. contains a
  181. .BR printf (3)
  182. format specification, the value from the file (with any specified masking
  183. performed) is printed using the message as the format string.
  184. .PP
  185. Some file formats contain additional information which is to be printed
  186. along with the file type or need additional tests to determine the true
  187. file type.
  188. These additional tests are introduced by one or more
  189. .B >
  190. characters preceding the offset.
  191. The number of
  192. .B >
  193. on the line indicates the level of the test; a line with no
  194. .B >
  195. at the beginning is considered to be at level 0.
  196. Tests are arranged in a tree-like hierarchy:
  197. If a the test on a line at level
  198. .IB n
  199. succeeds, all following tests at level
  200. .IB n+1
  201. are performed, and the messages printed if the tests succeed, untile a line
  202. with level
  203. .IB n
  204. (or less) appears.
  205. For more complex files, one can use empty messages to get just the
  206. "if/then" effect, in the following way:
  207. .sp
  208. .nf
  209. 0 string MZ
  210. >0x18 leshort <0x40 MS-DOS executable
  211. >0x18 leshort >0x3f extended PC executable (e.g., MS Windows)
  212. .fi
  213. .PP
  214. Offsets do not need to be constant, but can also be read from the file
  215. being examined.
  216. If the first character following the last
  217. .B >
  218. is a
  219. .B (
  220. then the string after the parenthesis is interpreted as an indirect offset.
  221. That means that the number after the parenthesis is used as an offset in
  222. the file.
  223. The value at that offset is read, and is used again as an offset
  224. in the file.
  225. Indirect offsets are of the form:
  226. .BI (( x [.[bslBSL]][+\-][ y ]).
  227. The value of
  228. .I x
  229. is used as an offset in the file. A byte, short or long is read at that offset
  230. depending on the
  231. .B [bslBSL]
  232. type specifier.
  233. The capitalized types interpret the number as a big endian
  234. value, whereas the small letter versions interpret the number as a little
  235. endian value.
  236. To that number the value of
  237. .I y
  238. is added and the result is used as an offset in the file.
  239. The default type if one is not specified is long.
  240. .PP
  241. That way variable length structures can be examined:
  242. .sp
  243. .nf
  244. # MS Windows executables are also valid MS-DOS executables
  245. 0 string MZ
  246. >0x18 leshort <0x40 MZ executable (MS-DOS)
  247. # skip the whole block below if it is not an extended executable
  248. >0x18 leshort >0x3f
  249. >>(0x3c.l) string PE\e0\e0 PE executable (MS-Windows)
  250. >>(0x3c.l) string LX\e0\e0 LX executable (OS/2)
  251. .fi
  252. .PP
  253. This strategy of examining has one drawback: You must make sure that
  254. you eventually print something, or users may get empty output (like, when
  255. there is neither PE\e0\e0 nor LE\e0\e0 in the above example)
  256. .PP
  257. If this indirect offset cannot be used as-is, there are simple calculations
  258. possible: appending
  259. .BI [+-*/%&|^]<number>
  260. inside parentheses allows one to modify
  261. the value read from the file before it is used as an offset:
  262. .sp
  263. .nf
  264. # MS Windows executables are also valid MS-DOS executables
  265. 0 string MZ
  266. # sometimes, the value at 0x18 is less that 0x40 but there's still an
  267. # extended executable, simply appended to the file
  268. >0x18 leshort <0x40
  269. >>(4.s*512) leshort 0x014c COFF executable (MS-DOS, DJGPP)
  270. >>(4.s*512) leshort !0x014c MZ executable (MS-DOS)
  271. .fi
  272. .PP
  273. Sometimes you do not know the exact offset as this depends on the length or
  274. position (when indirection was used before) of preceding fields. You can
  275. specify an offset relative to the end of the last uplevel field using
  276. .BI &
  277. as a prefix to the offset:
  278. .sp
  279. .nf
  280. 0 string MZ
  281. >0x18 leshort >0x3f
  282. >>(0x3c.l) string PE\e0\e0 PE executable (MS-Windows)
  283. # immediately following the PE signature is the CPU type
  284. >>>&0 leshort 0x14c for Intel 80386
  285. >>>&0 leshort 0x184 for DEC Alpha
  286. .fi
  287. .PP
  288. Indirect and relative offsets can be combined:
  289. .sp
  290. .nf
  291. 0 string MZ
  292. >0x18 leshort <0x40
  293. >>(4.s*512) leshort !0x014c MZ executable (MS-DOS)
  294. # if it's not COFF, go back 512 bytes and add the offset taken
  295. # from byte 2/3, which is yet another way of finding the start
  296. # of the extended executable
  297. >>>&(2.s-514) string LE LE executable (MS Windows VxD driver)
  298. .fi
  299. .PP
  300. Or the other way around:
  301. .sp
  302. .nf
  303. 0 string MZ
  304. >0x18 leshort >0x3f
  305. >>(0x3c.l) string LE\e0\e0 LE executable (MS-Windows)
  306. # at offset 0x80 (-4, since relative offsets start at the end
  307. # of the uplevel match) inside the LE header, we find the absolute
  308. # offset to the code area, where we look for a specific signature
  309. >>>(&0x7c.l+0x26) string UPX \eb, UPX compressed
  310. .fi
  311. .PP
  312. Or even both!
  313. .sp
  314. .nf
  315. 0 string MZ
  316. >0x18 leshort >0x3f
  317. >>(0x3c.l) string LE\e0\e0 LE executable (MS-Windows)
  318. # at offset 0x58 inside the LE header, we find the relative offset
  319. # to a data area where we look for a specific signature
  320. >>>&(&0x54.l-3) string UNACE \eb, ACE self-extracting archive
  321. .fi
  322. .PP
  323. Finally, if you have to deal with offset/length pairs in your file, even the
  324. second value in a parenthesed expression can be taken from the file itself,
  325. using another set of parentheses. Note that this additional indirect offset
  326. is always relative to the start of the main indirect offset.
  327. .sp
  328. .nf
  329. 0 string MZ
  330. >0x18 leshort >0x3f
  331. >>(0x3c.l) string PE\e0\e0 PE executable (MS-Windows)
  332. # search for the PE section called ".idata"...
  333. >>>&0xf4 search/0x140 .idata
  334. # ...and go to the end of it, calculated from start+length;
  335. # these are located 14 and 10 bytes after the section name
  336. >>>>(&0xe.l+(-4)) string PK\e3\e4 \eb, ZIP self-extracting archive
  337. .fi
  338. .SH BUGS
  339. The formats
  340. .IR long ,
  341. .IR belong ,
  342. .IR lelong ,
  343. .IR short ,
  344. .IR beshort ,
  345. .IR leshort ,
  346. .IR date ,
  347. .IR bedate ,
  348. and
  349. .I ledate
  350. are system-dependent; perhaps they should be specified as a number
  351. of bytes (2B, 4B, etc),
  352. since the files being recognized typically come from
  353. a system on which the lengths are invariant.
  354. .PP
  355. There is (currently) no support for specified-endian data to be used in
  356. indirect offsets.
  357. .SH SEE ALSO
  358. .BR file (__CSECTION__)
  359. \- the command that reads this file.
  360. .\"
  361. .\" From: guy@sun.uucp (Guy Harris)
  362. .\" Newsgroups: net.bugs.usg
  363. .\" Subject: /etc/magic's format isn't well documented
  364. .\" Message-ID: <2752@sun.uucp>
  365. .\" Date: 3 Sep 85 08:19:07 GMT
  366. .\" Organization: Sun Microsystems, Inc.
  367. .\" Lines: 136
  368. .\"
  369. .\" Here's a manual page for the format accepted by the "file" made by adding
  370. .\" the changes I posted to the S5R2 version.
  371. .\"
  372. .\" Modified for Ian Darwin's version of the file command.
  373. .\" @(#)$Id: magic.man,v 1.28 2005/03/17 17:34:15 christos Exp $