cpio.5.html 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482
  1. <!-- Creator : groff version 1.22.4 -->
  2. <!-- CreationDate: Tue Jul 18 07:11:06 2023 -->
  3. <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
  4. "http://www.w3.org/TR/html4/loose.dtd">
  5. <html>
  6. <head>
  7. <meta name="generator" content="groff -Thtml, see www.gnu.org">
  8. <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
  9. <meta name="Content-Style" content="text/css">
  10. <style type="text/css">
  11. p { margin-top: 0; margin-bottom: 0; vertical-align: top }
  12. pre { margin-top: 0; margin-bottom: 0; vertical-align: top }
  13. table { margin-top: 0; margin-bottom: 0; vertical-align: top }
  14. h1 { text-align: center }
  15. </style>
  16. <title></title>
  17. </head>
  18. <body>
  19. <hr>
  20. <p>CPIO(5) BSD File Formats Manual CPIO(5)</p>
  21. <p style="margin-top: 1em"><b>NAME</b></p>
  22. <p style="margin-left:6%;"><b>cpio</b> &mdash; format of
  23. cpio archive files</p>
  24. <p style="margin-top: 1em"><b>DESCRIPTION</b></p>
  25. <p style="margin-left:6%;">The <b>cpio</b> archive format
  26. collects any number of files, directories, and other file
  27. system objects (symbolic links, device nodes, etc.) into a
  28. single stream of bytes.</p>
  29. <p style="margin-left:6%; margin-top: 1em"><b>General
  30. Format</b> <br>
  31. Each file system object in a <b>cpio</b> archive comprises a
  32. header record with basic numeric metadata followed by the
  33. full pathname of the entry and the file data. The header
  34. record stores a series of integer values that generally
  35. follow the fields in <i>struct stat</i>. (See stat(2) for
  36. details.) The variants differ primarily in how they store
  37. those integers (binary, octal, or hexadecimal). The header
  38. is followed by the pathname of the entry (the length of the
  39. pathname is stored in the header) and any file data. The end
  40. of the archive is indicated by a special record with the
  41. pathname &ldquo;TRAILER!!!&rdquo;.</p>
  42. <p style="margin-left:6%; margin-top: 1em"><b>PWB
  43. format</b> <br>
  44. The PWB binary <b>cpio</b> format is the original format,
  45. when cpio was introduced as part of the Programmer&rsquo;s
  46. Work Bench system, a variant of 6th Edition UNIX. It stores
  47. numbers as 2-byte and 4-byte binary values. Each entry
  48. begins with a header in the following format:</p>
  49. <p style="margin-left:14%; margin-top: 1em">struct
  50. header_pwb_cpio { <br>
  51. short h_magic; <br>
  52. short h_dev; <br>
  53. short h_ino; <br>
  54. short h_mode; <br>
  55. short h_uid; <br>
  56. short h_gid; <br>
  57. short h_nlink; <br>
  58. short h_majmin; <br>
  59. long h_mtime; <br>
  60. short h_namesize; <br>
  61. long h_filesize; <br>
  62. };</p>
  63. <p style="margin-left:6%; margin-top: 1em">The <i>short</i>
  64. fields here are 16-bit integer values, while the <i>long</i>
  65. fields are 32 bit integers. Since PWB UNIX, like the 6th
  66. Edition UNIX it was based on, only ran on PDP-11 computers,
  67. they are in PDP-endian format, which has little-endian
  68. shorts, and big-endian longs. That is, the long integer
  69. whose hexadecimal representation is 0x12345678 would be
  70. stored in four successive bytes as 0x34, 0x12, 0x78, 0x56.
  71. The fields are as follows:</p>
  72. <p style="margin-top: 1em"><i>h_magic</i></p>
  73. <p style="margin-left:17%;">The integer value octal
  74. 070707.</p>
  75. <p style="margin-top: 1em"><i>h_dev</i>, <i>h_ino</i></p>
  76. <p style="margin-left:17%;">The device and inode numbers
  77. from the disk. These are used by programs that read
  78. <b>cpio</b> archives to determine when two entries refer to
  79. the same file. Programs that synthesize <b>cpio</b> archives
  80. should be careful to set these to distinct values for each
  81. entry.</p>
  82. <p style="margin-top: 1em"><i>h_mode</i></p>
  83. <p style="margin-left:17%; margin-top: 1em">The mode
  84. specifies both the regular permissions and the file type,
  85. and it also holds a couple of bits that are irrelevant to
  86. the cpio format, because the field is actually a raw copy of
  87. the mode field in the inode representing the file. These are
  88. the IALLOC flag, which shows that the inode entry is in use,
  89. and the ILARG flag, which shows that the file it represents
  90. is large enough to have indirect blocks pointers in the
  91. inode. The mode is decoded as follows:</p>
  92. <p style="margin-top: 1em">0100000</p>
  93. <p style="margin-left:28%; margin-top: 1em">IALLOC flag -
  94. irrelevant to cpio.</p>
  95. <p>0060000</p>
  96. <p style="margin-left:28%; margin-top: 1em">This masks the
  97. file type bits.</p>
  98. <p>0040000</p>
  99. <p style="margin-left:28%; margin-top: 1em">File type value
  100. for directories.</p>
  101. <p>0020000</p>
  102. <p style="margin-left:28%; margin-top: 1em">File type value
  103. for character special devices.</p>
  104. <p>0060000</p>
  105. <p style="margin-left:28%; margin-top: 1em">File type value
  106. for block special devices.</p>
  107. <p>0010000</p>
  108. <p style="margin-left:28%; margin-top: 1em">ILARG flag -
  109. irrelevant to cpio.</p>
  110. <p>0004000</p>
  111. <p style="margin-left:28%; margin-top: 1em">SUID bit.</p>
  112. <p>0002000</p>
  113. <p style="margin-left:28%; margin-top: 1em">SGID bit.</p>
  114. <p>0001000</p>
  115. <p style="margin-left:28%; margin-top: 1em">Sticky bit.</p>
  116. <p>0000777</p>
  117. <p style="margin-left:28%; margin-top: 1em">The lower 9
  118. bits specify read/write/execute permissions for world,
  119. group, and user following standard POSIX conventions.</p>
  120. <p style="margin-top: 1em"><i>h_uid</i>, <i>h_gid</i></p>
  121. <p style="margin-left:17%;">The numeric user id and group
  122. id of the owner.</p>
  123. <p style="margin-top: 1em"><i>h_nlink</i></p>
  124. <p style="margin-left:17%;">The number of links to this
  125. file. Directories always have a value of at least two here.
  126. Note that hardlinked files include file data with every copy
  127. in the archive.</p>
  128. <p style="margin-top: 1em"><i>h_majmin</i></p>
  129. <p style="margin-left:17%;">For block special and character
  130. special entries, this field contains the associated device
  131. number, with the major number in the high byte, and the
  132. minor number in the low byte. For all other entry types, it
  133. should be set to zero by writers and ignored by readers.</p>
  134. <p style="margin-top: 1em"><i>h_mtime</i></p>
  135. <p style="margin-left:17%;">Modification time of the file,
  136. indicated as the number of seconds since the start of the
  137. epoch, 00:00:00 UTC January 1, 1970.</p>
  138. <p style="margin-top: 1em"><i>h_namesize</i></p>
  139. <p style="margin-left:17%;">The number of bytes in the
  140. pathname that follows the header. This count includes the
  141. trailing NUL byte.</p>
  142. <p style="margin-top: 1em"><i>h_filesize</i></p>
  143. <p style="margin-left:17%;">The size of the file. Note that
  144. this archive format is limited to 16 megabyte file sizes,
  145. because PWB UNIX, like 6th Edition, only used an unsigned 24
  146. bit integer for the file size internally.</p>
  147. <p style="margin-left:6%; margin-top: 1em">The pathname
  148. immediately follows the fixed header. If <b>h_namesize</b>
  149. is odd, an additional NUL byte is added after the pathname.
  150. The file data is then appended, again with an additional NUL
  151. appended if needed to get the next header at an even
  152. offset.</p>
  153. <p style="margin-left:6%; margin-top: 1em">Hardlinked files
  154. are not given special treatment; the full file contents are
  155. included with each copy of the file.</p>
  156. <p style="margin-left:6%; margin-top: 1em"><b>New Binary
  157. Format</b> <br>
  158. The new binary <b>cpio</b> format showed up when cpio was
  159. adopted into late 7th Edition UNIX. It is exactly like the
  160. PWB binary format, described above, except for three
  161. changes:</p>
  162. <p style="margin-left:6%; margin-top: 1em">First, UNIX now
  163. ran on more than one hardware type, so the endianness of 16
  164. bit integers must be determined by observing the magic
  165. number at the start of the header. The 32 bit integers are
  166. still always stored with the most significant word first,
  167. though, so each of those two, in the struct shown above, was
  168. stored as an array of two 16 bit integers, in the
  169. traditional order. Those 16 bit integers, like all the
  170. others in the struct, were accessed using a macro that byte
  171. swapped them if necessary.</p>
  172. <p style="margin-left:6%; margin-top: 1em">Next, 7th
  173. Edition had more file types to store, and the IALLOC and
  174. ILARG flag bits were re-purposed to accommodate these. The
  175. revised use of the various bits is as follows:</p>
  176. <p style="margin-top: 1em">0170000</p>
  177. <p style="margin-left:18%; margin-top: 1em">This masks the
  178. file type bits.</p>
  179. <p>0140000</p>
  180. <p style="margin-left:18%; margin-top: 1em">File type value
  181. for sockets.</p>
  182. <p>0120000</p>
  183. <p style="margin-left:18%; margin-top: 1em">File type value
  184. for symbolic links. For symbolic links, the link body is
  185. stored as file data.</p>
  186. <p>0100000</p>
  187. <p style="margin-left:18%; margin-top: 1em">File type value
  188. for regular files.</p>
  189. <p>0060000</p>
  190. <p style="margin-left:18%; margin-top: 1em">File type value
  191. for block special devices.</p>
  192. <p>0040000</p>
  193. <p style="margin-left:18%; margin-top: 1em">File type value
  194. for directories.</p>
  195. <p>0020000</p>
  196. <p style="margin-left:18%; margin-top: 1em">File type value
  197. for character special devices.</p>
  198. <p>0010000</p>
  199. <p style="margin-left:18%; margin-top: 1em">File type value
  200. for named pipes or FIFOs.</p>
  201. <p>0004000</p>
  202. <p style="margin-left:18%; margin-top: 1em">SUID bit.</p>
  203. <p>0002000</p>
  204. <p style="margin-left:18%; margin-top: 1em">SGID bit.</p>
  205. <p>0001000</p>
  206. <p style="margin-left:18%; margin-top: 1em">Sticky bit.</p>
  207. <p>0000777</p>
  208. <p style="margin-left:18%; margin-top: 1em">The lower 9
  209. bits specify read/write/execute permissions for world,
  210. group, and user following standard POSIX conventions.</p>
  211. <p style="margin-left:6%; margin-top: 1em">Finally, the
  212. file size field now represents a signed 32 bit integer in
  213. the underlying file system, so the maximum file size has
  214. increased to 2 gigabytes.</p>
  215. <p style="margin-left:6%; margin-top: 1em">Note that there
  216. is no obvious way to tell which of the two binary formats an
  217. archive uses, other than to see which one makes more sense.
  218. The typical error scenario is that a PWB format archive
  219. unpacked as if it were in the new format will create named
  220. sockets instead of directories, and then fail to unpack
  221. files that should go in those directories. Running
  222. <i>bsdcpio -itv</i> on an unknown archive will make it
  223. obvious which it is: if it&rsquo;s PWB format, directories
  224. will be listed with an &rsquo;s&rsquo; instead of a
  225. &rsquo;d&rsquo; as the first character of the mode string,
  226. and the larger files will have a &rsquo;?&rsquo; in that
  227. position.</p>
  228. <p style="margin-left:6%; margin-top: 1em"><b>Portable
  229. ASCII Format</b> <br>
  230. Version&nbsp;2 of the Single UNIX Specification
  231. (&ldquo;SUSv2&rdquo;) standardized an ASCII variant that is
  232. portable across all platforms. It is commonly known as the
  233. &ldquo;old character&rdquo; format or as the
  234. &ldquo;odc&rdquo; format. It stores the same numeric fields
  235. as the old binary format, but represents them as 6-character
  236. or 11-character octal values.</p>
  237. <p style="margin-left:14%; margin-top: 1em">struct
  238. cpio_odc_header { <br>
  239. char c_magic[6]; <br>
  240. char c_dev[6]; <br>
  241. char c_ino[6]; <br>
  242. char c_mode[6]; <br>
  243. char c_uid[6]; <br>
  244. char c_gid[6]; <br>
  245. char c_nlink[6]; <br>
  246. char c_rdev[6]; <br>
  247. char c_mtime[11]; <br>
  248. char c_namesize[6]; <br>
  249. char c_filesize[11]; <br>
  250. };</p>
  251. <p style="margin-left:6%; margin-top: 1em">The fields are
  252. identical to those in the new binary format. The name and
  253. file body follow the fixed header. Unlike the binary
  254. formats, there is no additional padding after the pathname
  255. or file contents. If the files being archived are themselves
  256. entirely ASCII, then the resulting archive will be entirely
  257. ASCII, except for the NUL byte that terminates the name
  258. field.</p>
  259. <p style="margin-left:6%; margin-top: 1em"><b>New ASCII
  260. Format</b> <br>
  261. The &quot;new&quot; ASCII format uses 8-byte hexadecimal
  262. fields for all numbers and separates device numbers into
  263. separate fields for major and minor numbers.</p>
  264. <p style="margin-left:14%; margin-top: 1em">struct
  265. cpio_newc_header { <br>
  266. char c_magic[6]; <br>
  267. char c_ino[8]; <br>
  268. char c_mode[8]; <br>
  269. char c_uid[8]; <br>
  270. char c_gid[8]; <br>
  271. char c_nlink[8]; <br>
  272. char c_mtime[8]; <br>
  273. char c_filesize[8]; <br>
  274. char c_devmajor[8]; <br>
  275. char c_devminor[8]; <br>
  276. char c_rdevmajor[8]; <br>
  277. char c_rdevminor[8]; <br>
  278. char c_namesize[8]; <br>
  279. char c_check[8]; <br>
  280. };</p>
  281. <p style="margin-left:6%; margin-top: 1em">Except as
  282. specified below, the fields here match those specified for
  283. the new binary format above.</p>
  284. <p style="margin-top: 1em"><i>magic</i></p>
  285. <p style="margin-left:17%; margin-top: 1em">The string
  286. &ldquo;070701&rdquo;.</p>
  287. <p style="margin-top: 1em"><i>check</i></p>
  288. <p style="margin-left:17%; margin-top: 1em">This field is
  289. always set to zero by writers and ignored by readers. See
  290. the next section for more details.</p>
  291. <p style="margin-left:6%; margin-top: 1em">The pathname is
  292. followed by NUL bytes so that the total size of the fixed
  293. header plus pathname is a multiple of four. Likewise, the
  294. file data is padded to a multiple of four bytes. Note that
  295. this format supports only 4 gigabyte files (unlike the older
  296. ASCII format, which supports 8 gigabyte files).</p>
  297. <p style="margin-left:6%; margin-top: 1em">In this format,
  298. hardlinked files are handled by setting the filesize to zero
  299. for each entry except the first one that appears in the
  300. archive.</p>
  301. <p style="margin-left:6%; margin-top: 1em"><b>New CRC
  302. Format</b> <br>
  303. The CRC format is identical to the new ASCII format
  304. described in the previous section except that the magic
  305. field is set to &ldquo;070702&rdquo; and the <i>check</i>
  306. field is set to the sum of all bytes in the file data. This
  307. sum is computed treating all bytes as unsigned values and
  308. using unsigned arithmetic. Only the least-significant 32
  309. bits of the sum are stored.</p>
  310. <p style="margin-left:6%; margin-top: 1em"><b>HP
  311. variants</b> <br>
  312. The <b>cpio</b> implementation distributed with HPUX used
  313. XXXX but stored device numbers differently XXX.</p>
  314. <p style="margin-left:6%; margin-top: 1em"><b>Other
  315. Extensions and Variants</b> <br>
  316. Sun Solaris uses additional file types to store extended
  317. file data, including ACLs and extended attributes, as
  318. special entries in cpio archives.</p>
  319. <p style="margin-left:6%; margin-top: 1em">XXX Others?
  320. XXX</p>
  321. <p style="margin-top: 1em"><b>SEE ALSO</b></p>
  322. <p style="margin-left:6%;">cpio(1), tar(5)</p>
  323. <p style="margin-top: 1em"><b>STANDARDS</b></p>
  324. <p style="margin-left:6%;">The <b>cpio</b> utility is no
  325. longer a part of POSIX or the Single Unix Standard. It last
  326. appeared in Version&nbsp;2 of the Single UNIX Specification
  327. (&ldquo;SUSv2&rdquo;). It has been supplanted in subsequent
  328. standards by pax(1). The portable ASCII format is currently
  329. part of the specification for the pax(1) utility.</p>
  330. <p style="margin-top: 1em"><b>HISTORY</b></p>
  331. <p style="margin-left:6%;">The original cpio utility was
  332. written by Dick Haight while working in AT&amp;T&rsquo;s
  333. Unix Support Group. It appeared in 1977 as part of PWB/UNIX
  334. 1.0, the &ldquo;Programmer&rsquo;s Work Bench&rdquo; derived
  335. from Version&nbsp;6 AT&amp;T UNIX that was used internally
  336. at AT&amp;T. Both the new binary and old character formats
  337. were in use by 1980, according to the System III source
  338. released by SCO under their &ldquo;Ancient Unix&rdquo;
  339. license. The character format was adopted as part of IEEE
  340. Std 1003.1-1988 (&ldquo;POSIX.1&rdquo;). XXX when did
  341. &quot;newc&quot; appear? Who invented it? When did HP come
  342. out with their variant? When did Sun introduce ACLs and
  343. extended attributes? XXX</p>
  344. <p style="margin-top: 1em"><b>BUGS</b></p>
  345. <p style="margin-left:6%;">The &ldquo;CRC&rdquo; format is
  346. mis-named, as it uses a simple checksum and not a cyclic
  347. redundancy check.</p>
  348. <p style="margin-left:6%; margin-top: 1em">The binary
  349. formats are limited to 16 bits for user id, group id,
  350. device, and inode numbers. They are limited to 16 megabyte
  351. and 2 gigabyte file sizes for the older and newer variants,
  352. respectively.</p>
  353. <p style="margin-left:6%; margin-top: 1em">The old ASCII
  354. format is limited to 18 bits for the user id, group id,
  355. device, and inode numbers. It is limited to 8 gigabyte file
  356. sizes.</p>
  357. <p style="margin-left:6%; margin-top: 1em">The new ASCII
  358. format is limited to 4 gigabyte file sizes.</p>
  359. <p style="margin-left:6%; margin-top: 1em">None of the cpio
  360. formats store user or group names, which are essential when
  361. moving files between systems with dissimilar user or group
  362. numbering.</p>
  363. <p style="margin-left:6%; margin-top: 1em">Especially when
  364. writing older cpio variants, it may be necessary to map
  365. actual device/inode values to synthesized values that fit
  366. the available fields. With very large filesystems, this may
  367. be necessary even for the newer formats.</p>
  368. <p style="margin-left:6%; margin-top: 1em">BSD
  369. December&nbsp;23, 2011 BSD</p>
  370. <hr>
  371. </body>
  372. </html>