ext_ffi.html 10 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334
  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
  2. <html>
  3. <head>
  4. <title>FFI Library</title>
  5. <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
  6. <meta name="Author" content="Mike Pall">
  7. <meta name="Copyright" content="Copyright (C) 2005-2012, Mike Pall">
  8. <meta name="Language" content="en">
  9. <link rel="stylesheet" type="text/css" href="bluequad.css" media="screen">
  10. <link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print">
  11. <style type="text/css">
  12. span.codemark { position:absolute; left: 16em; color: #4040c0; }
  13. span.mark { color: #4040c0; font-family: Courier New, Courier, monospace;
  14. line-height: 1.1; }
  15. pre.mark { padding-left: 2em; }
  16. </style>
  17. </head>
  18. <body>
  19. <div id="site">
  20. <a href="http://luajit.org"><span>Lua<span id="logo">JIT</span></span></a>
  21. </div>
  22. <div id="head">
  23. <h1>FFI Library</h1>
  24. </div>
  25. <div id="nav">
  26. <ul><li>
  27. <a href="luajit.html">LuaJIT</a>
  28. <ul><li>
  29. <a href="install.html">Installation</a>
  30. </li><li>
  31. <a href="running.html">Running</a>
  32. </li></ul>
  33. </li><li>
  34. <a href="extensions.html">Extensions</a>
  35. <ul><li>
  36. <a class="current" href="ext_ffi.html">FFI Library</a>
  37. <ul><li>
  38. <a href="ext_ffi_tutorial.html">FFI Tutorial</a>
  39. </li><li>
  40. <a href="ext_ffi_api.html">ffi.* API</a>
  41. </li><li>
  42. <a href="ext_ffi_semantics.html">FFI Semantics</a>
  43. </li></ul>
  44. </li><li>
  45. <a href="ext_jit.html">jit.* Library</a>
  46. </li><li>
  47. <a href="ext_c_api.html">Lua/C API</a>
  48. </li></ul>
  49. </li><li>
  50. <a href="status.html">Status</a>
  51. <ul><li>
  52. <a href="changes.html">Changes</a>
  53. </li></ul>
  54. </li><li>
  55. <a href="faq.html">FAQ</a>
  56. </li><li>
  57. <a href="http://luajit.org/performance.html">Performance <span class="ext">&raquo;</span></a>
  58. </li><li>
  59. <a href="http://luajit.org/download.html">Download <span class="ext">&raquo;</span></a>
  60. </li><li>
  61. <a href="http://luajit.org/list.html">Mailing List <span class="ext">&raquo;</span></a>
  62. </li></ul>
  63. </div>
  64. <div id="main">
  65. <p>
  66. The FFI library allows <b>calling external C&nbsp;functions</b> and
  67. <b>using C&nbsp;data structures</b> from pure Lua code.
  68. </p>
  69. <p>
  70. The FFI library largely obviates the need to write tedious manual
  71. Lua/C bindings in C. No need to learn a separate binding language
  72. &mdash; <b>it parses plain C&nbsp;declarations!</b> These can be
  73. cut-n-pasted from C&nbsp;header files or reference manuals. It's up to
  74. the task of binding large libraries without the need for dealing with
  75. fragile binding generators.
  76. </p>
  77. <p>
  78. The FFI library is tightly integrated into LuaJIT (it's not available
  79. as a separate module). The code generated by the JIT-compiler for
  80. accesses to C&nbsp;data structures from Lua code is on par with the
  81. code a C&nbsp;compiler would generate. Calls to C&nbsp;functions can
  82. be inlined in JIT-compiled code, unlike calls to functions bound via
  83. the classic Lua/C API.
  84. </p>
  85. <p>
  86. This page gives a short introduction to the usage of the FFI library.
  87. <em>Please use the FFI sub-topics in the navigation bar to learn more.</em>
  88. </p>
  89. <h2 id="call">Motivating Example: Calling External C Functions</h2>
  90. <p>
  91. It's really easy to call an external C&nbsp;library function:
  92. </p>
  93. <pre class="code mark">
  94. <span class="codemark">&#9312;
  95. &#9313;
  96. &#9314;</span>local ffi = require("ffi")
  97. ffi.cdef[[
  98. <span style="color:#00a000;">int printf(const char *fmt, ...);</span>
  99. ]]
  100. ffi.C.printf("Hello %s!", "world")
  101. </pre>
  102. <p>
  103. So, let's pick that apart:
  104. </p>
  105. <p>
  106. <span class="mark">&#9312;</span> Load the FFI library.
  107. </p>
  108. <p>
  109. <span class="mark">&#9313;</span> Add a C&nbsp;declaration
  110. for the function. The part inside the double-brackets (in green) is
  111. just standard C&nbsp;syntax.
  112. </p>
  113. <p>
  114. <span class="mark">&#9314;</span> Call the named
  115. C&nbsp;function &mdash; Yes, it's that simple!
  116. </p>
  117. <p style="font-size: 8pt;">
  118. Actually, what goes on behind the scenes is far from simple: <span
  119. style="color:#4040c0;">&#9314;</span> makes use of the standard
  120. C&nbsp;library namespace <tt>ffi.C</tt>. Indexing this namespace with
  121. a symbol name (<tt>"printf"</tt>) automatically binds it to the
  122. standard C&nbsp;library. The result is a special kind of object which,
  123. when called, runs the <tt>printf</tt> function. The arguments passed
  124. to this function are automatically converted from Lua objects to the
  125. corresponding C&nbsp;types.
  126. </p>
  127. <p>
  128. Ok, so maybe the use of <tt>printf()</tt> wasn't such a spectacular
  129. example. You could have done that with <tt>io.write()</tt> and
  130. <tt>string.format()</tt>, too. But you get the idea ...
  131. </p>
  132. <p>
  133. So here's something to pop up a message box on Windows:
  134. </p>
  135. <pre class="code">
  136. local ffi = require("ffi")
  137. ffi.cdef[[
  138. <span style="color:#00a000;">int MessageBoxA(void *w, const char *txt, const char *cap, int type);</span>
  139. ]]
  140. ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)
  141. </pre>
  142. <p>
  143. Bing! Again, that was far too easy, no?
  144. </p>
  145. <p style="font-size: 8pt;">
  146. Compare this with the effort required to bind that function using the
  147. classic Lua/C API: create an extra C&nbsp;file, add a C&nbsp;function
  148. that retrieves and checks the argument types passed from Lua and calls
  149. the actual C&nbsp;function, add a list of module functions and their
  150. names, add a <tt>luaopen_*</tt> function and register all module
  151. functions, compile and link it into a shared library (DLL), move it to
  152. the proper path, add Lua code that loads the module aaaand ... finally
  153. call the binding function. Phew!
  154. </p>
  155. <h2 id="cdata">Motivating Example: Using C Data Structures</h2>
  156. <p>
  157. The FFI library allows you to create and access C&nbsp;data
  158. structures. Of course the main use for this is for interfacing with
  159. C&nbsp;functions. But they can be used stand-alone, too.
  160. </p>
  161. <p>
  162. Lua is built upon high-level data types. They are flexible, extensible
  163. and dynamic. That's why we all love Lua so much. Alas, this can be
  164. inefficient for certain tasks, where you'd really want a low-level
  165. data type. E.g. a large array of a fixed structure needs to be
  166. implemented with a big table holding lots of tiny tables. This imposes
  167. both a substantial memory overhead as well as a performance overhead.
  168. </p>
  169. <p>
  170. Here's a sketch of a library that operates on color images plus a
  171. simple benchmark. First, the plain Lua version:
  172. </p>
  173. <pre class="code">
  174. local floor = math.floor
  175. local function image_ramp_green(n)
  176. local img = {}
  177. local f = 255/(n-1)
  178. for i=1,n do
  179. img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
  180. end
  181. return img
  182. end
  183. local function image_to_grey(img, n)
  184. for i=1,n do
  185. local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
  186. img[i].red = y; img[i].green = y; img[i].blue = y
  187. end
  188. end
  189. local N = 400*400
  190. local img = image_ramp_green(N)
  191. for i=1,1000 do
  192. image_to_grey(img, N)
  193. end
  194. </pre>
  195. <p>
  196. This creates a table with 160.000 pixels, each of which is a table
  197. holding four number values in the range of 0-255. First an image with
  198. a green ramp is created (1D for simplicity), then the image is
  199. converted to greyscale 1000 times. Yes, that's silly, but I was in
  200. need of a simple example ...
  201. </p>
  202. <p>
  203. And here's the FFI version. The modified parts have been marked in
  204. bold:
  205. </p>
  206. <pre class="code mark">
  207. <span class="codemark">&#9312;
  208. &#9313;
  209. &#9314;
  210. &#9315;
  211. &#9314;
  212. &#9316;</span><b>local ffi = require("ffi")
  213. ffi.cdef[[
  214. </b><span style="color:#00a000;">typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;</span><b>
  215. ]]</b>
  216. local function image_ramp_green(n)
  217. <b>local img = ffi.new("rgba_pixel[?]", n)</b>
  218. local f = 255/(n-1)
  219. for i=<b>0,n-1</b> do
  220. <b>img[i].green = i*f</b>
  221. <b>img[i].alpha = 255</b>
  222. end
  223. return img
  224. end
  225. local function image_to_grey(img, n)
  226. for i=<b>0,n-1</b> do
  227. local y = <b>0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue</b>
  228. img[i].red = y; img[i].green = y; img[i].blue = y
  229. end
  230. end
  231. local N = 400*400
  232. local img = image_ramp_green(N)
  233. for i=1,1000 do
  234. image_to_grey(img, N)
  235. end
  236. </pre>
  237. <p>
  238. Ok, so that wasn't too difficult:
  239. </p>
  240. <p>
  241. <span class="mark">&#9312;</span> First, load the FFI
  242. library and declare the low-level data type. Here we choose a
  243. <tt>struct</tt> which holds four byte fields, one for each component
  244. of a 4x8&nbsp;bit RGBA pixel.
  245. </p>
  246. <p>
  247. <span class="mark">&#9313;</span> Creating the data
  248. structure with <tt>ffi.new()</tt> is straightforward &mdash; the
  249. <tt>'?'</tt> is a placeholder for the number of elements of a
  250. variable-length array.
  251. </p>
  252. <p>
  253. <span class="mark">&#9314;</span> C&nbsp;arrays are
  254. zero-based, so the indexes have to run from <tt>0</tt> to
  255. <tt>n-1</tt>. One might want to allocate one more element instead to
  256. simplify converting legacy code.
  257. </p>
  258. <p>
  259. <span class="mark">&#9315;</span> Since <tt>ffi.new()</tt>
  260. zero-fills the array by default, we only need to set the green and the
  261. alpha fields.
  262. </p>
  263. <p>
  264. <span class="mark">&#9316;</span> The calls to
  265. <tt>math.floor()</tt> can be omitted here, because floating-point
  266. numbers are already truncated towards zero when converting them to an
  267. integer. This happens implicitly when the number is stored in the
  268. fields of each pixel.
  269. </p>
  270. <p>
  271. Now let's have a look at the impact of the changes: first, memory
  272. consumption for the image is down from 22&nbsp;Megabytes to
  273. 640&nbsp;Kilobytes (400*400*4 bytes). That's a factor of 35x less! So,
  274. yes, tables do have a noticeable overhead. BTW: The original program
  275. would consume 40&nbsp;Megabytes in plain Lua (on x64).
  276. </p>
  277. <p>
  278. Next, performance: the pure Lua version runs in 9.57 seconds (52.9
  279. seconds with the Lua interpreter) and the FFI version runs in 0.48
  280. seconds on my machine (YMMV). That's a factor of 20x faster (110x
  281. faster than the Lua interpreter).
  282. </p>
  283. <p style="font-size: 8pt;">
  284. The avid reader may notice that converting the pure Lua version over
  285. to use array indexes for the colors (<tt>[1]</tt> instead of
  286. <tt>.red</tt>, <tt>[2]</tt> instead of <tt>.green</tt> etc.) ought to
  287. be more compact and faster. This is certainly true (by a factor of
  288. ~1.7x). Switching to a struct-of-arrays would help, too.
  289. </p>
  290. <p style="font-size: 8pt;">
  291. However the resulting code would be less idiomatic and rather
  292. error-prone. And it still doesn't get even close to the performance of
  293. the FFI version of the code. Also, high-level data structures cannot
  294. be easily passed to other C&nbsp;functions, especially I/O functions,
  295. without undue conversion penalties.
  296. </p>
  297. <br class="flush">
  298. </div>
  299. <div id="foot">
  300. <hr class="hide">
  301. Copyright &copy; 2005-2012 Mike Pall
  302. <span class="noprint">
  303. &middot;
  304. <a href="contact.html">Contact</a>
  305. </span>
  306. </div>
  307. </body>
  308. </html>