c
/
LuaJIT
mirror of https://github.com/LuaJIT/LuaJIT.git


			
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486
							<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>FFI Tutorial</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Author" content="Mike Pall">
<meta name="Copyright" content="Copyright (C) 2005-2011, Mike Pall">
<meta name="Language" content="en">
<link rel="stylesheet" type="text/css" href="bluequad.css" media="screen">
<link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print">
<style type="text/css">
span.codemark { position:absolute; left: 16em; color: #4040c0; }
span.mark { color: #4040c0; font-family: Courier New, Courier, monospace;
  line-height: 1.1; }
pre.mark { padding-left: 2em; }
table.idiomtable { line-height: 1.2; }
table.idiomtable tt { font-size: 100%; }
tr.idiomhead td { font-weight: bold; }
td.idiomc { width: 12em; }
td.idiomlua { width: 14em; }
td.idiomlua b { font-weight: normal; color: #2142bf; }
</style>
</head>
<body>
<div id="site">
<a href="http://luajit.org"><span>Lua<span id="logo">JIT</span></span></a>
</div>
<div id="head">
<h1>FFI Tutorial</h1>
</div>
<div id="nav">
<ul><li>
<a href="luajit.html">LuaJIT</a>
<ul><li>
<a href="install.html">Installation</a>
</li><li>
<a href="running.html">Running</a>
</li></ul>
</li><li>
<a href="extensions.html">Extensions</a>
<ul><li>
<a href="ext_ffi.html">FFI Library</a>
<ul><li>
<a class="current" href="ext_ffi_tutorial.html">FFI Tutorial</a>
</li><li>
<a href="ext_ffi_api.html">ffi.* API</a>
</li><li>
<a href="ext_ffi_semantics.html">FFI Semantics</a>
</li></ul>
</li><li>
<a href="ext_jit.html">jit.* Library</a>
</li><li>
<a href="ext_c_api.html">Lua/C API</a>
</li></ul>
</li><li>
<a href="status.html">Status</a>
<ul><li>
<a href="changes.html">Changes</a>
</li></ul>
</li><li>
<a href="faq.html">FAQ</a>
</li><li>
<a href="http://luajit.org/performance.html">Performance <span class="ext">&raquo;</span></a>
</li><li>
<a href="http://luajit.org/download.html">Download <span class="ext">&raquo;</span></a>
</li></ul>
</div>
<div id="main">
<p>
This page is intended to give you an overview of the features of the FFI
library by presenting a few use cases and guidelines.
</p>
<p>
This page makes no attempt to explain all of the FFI library, though.
You'll want to have a look at the <a href="ext_ffi_api.html">ffi.* API
function reference</a> and the <a href="ext_ffi_semantics.html">FFI
semantics</a> to learn more.
</p>

<h2 id="load">Loading the FFI Library</h2>
<p>
The FFI library is built into LuaJIT by default, but it's not loaded
and initialized by default. The suggested way to use the FFI library
is to add the following to the start of every Lua file that needs one
of its functions:
</p>
<pre class="code">
local ffi = require("ffi")
</pre>
<p>
Please note this doesn't define an <tt>ffi</tt> variable in the table
of globals &mdash; you really need to use the local variable. The
<tt>require</tt> function ensures the library is only loaded once.
</p>

<h2 id="sleep">Accessing Standard System Functions</h2>
<p>
The following code explains how to access standard system functions.
We slowly print two lines of dots by sleeping for 10&nbsp;milliseconds
after each dot:
</p>
<pre class="code mark">
<span class="codemark">&nbsp;
&#9312;


&#9313;
&#9314;
&#9315;


&#9316;


&#9317;</span>local ffi = require("ffi")
ffi.cdef[[
<span style="color:#00a000;">void Sleep(int ms);
int poll(struct pollfd *fds, unsigned long nfds, int timeout);</span>
]]

local sleep
if ffi.os == "Windows" then
  function sleep(s)
    ffi.C.Sleep(s*1000)
  end
else
  function sleep(s)
    ffi.C.poll(nil, 0, s*1000)
  end
end

for i=1,160 do
  io.write("."); io.flush()
  sleep(0.01)
end
io.write("\n")
</pre>
<p>
Here's the step-by-step explanation:
</p>
<p>
<span class="mark">&#9312;</span> This defines the
C&nbsp;library functions we're going to use. The part inside the
double-brackets (in green) is just standard C&nbsp;syntax. You can
usually get this info from the C&nbsp;header files or the
documentation provided by each C&nbsp;library or C&nbsp;compiler.
</p>
<p>
<span class="mark">&#9313;</span> The difficulty we're
facing here, is that there are different standards to choose from.
Windows has a simple <tt>Sleep()</tt> function. On other systems there
are a variety of functions available to achieve sub-second sleeps, but
with no clear consensus. Thankfully <tt>poll()</tt> can be used for
this task, too, and it's present on most non-Windows systems. The
check for <tt>ffi.os</tt> makes sure we use the Windows-specific
function only on Windows systems.
</p>
<p>
<span class="mark">&#9314;</span> Here we're wrapping the
call to the C&nbsp;function in a Lua function. This isn't strictly
necessary, but it's helpful to deal with system-specific issues only
in one part of the code. The way we're wrapping it ensures the check
for the OS is only done during initialization and not for every call.
</p>
<p>
<span class="mark">&#9315;</span> A more subtle point is
that we defined our <tt>sleep()</tt> function (for the sake of this
example) as taking the number of seconds, but accepting fractional
seconds. Multiplying this by 1000 gets us milliseconds, but that still
leaves it a Lua number, which is a floating-point value. Alas, the
<tt>Sleep()</tt> function only accepts an integer value. Luckily for
us, the FFI library automatically performs the conversion when calling
the function (truncating the FP value towards zero, like in C).
</p>
<p style="font-size: 8pt;">
Some readers will notice that <tt>Sleep()</tt> is part of
<tt>KERNEL32.DLL</tt> and is also a <tt>stdcall</tt> function. So how
can this possibly work? The FFI library provides the <tt>ffi.C</tt>
default C&nbsp;library namespace, which allows calling functions from
the default set of libraries, like a C&nbsp;compiler would. Also, the
FFI library automatically detects <tt>stdcall</tt> functions, so you
don't need to declare them as such.
</p>
<p>
<span class="mark">&#9316;</span> The <tt>poll()</tt>
function takes a couple more arguments we're not going to use. You can
simply use <tt>nil</tt> to pass a <tt>NULL</tt> pointer and <tt>0</tt>
for the <tt>nfds</tt> parameter. Please note that the
number&nbsp;<tt>0</tt> <em>does not convert to a pointer value</em>,
unlike in C++. You really have to pass pointers to pointer arguments
and numbers to number arguments.
</p>
<p style="font-size: 8pt;">
The page on <a href="ext_ffi_semantics.html">FFI semantics</a> has all
of the gory details about
<a href="ext_ffi_semantics.html#convert">conversions between Lua
objects and C&nbsp;types</a>. For the most part you don't have to deal
with this, as it's performed automatically and it's carefully designed
to bridge the semantic differences between Lua and C.
</p>
<p>
<span class="mark">&#9317;</span> Now that we have defined
our own <tt>sleep()</tt> function, we can just call it from plain Lua
code. That wasn't so bad, huh? Turning these boring animated dots into
a fascinating best-selling game is left as an exercise for the reader.
:-)
</p>

<h2 id="zlib">Accessing the zlib Compression Library</h2>
<p>
The following code shows how to access the <a
href="http://zlib.net/">zlib</a> compression library from Lua code.
We'll define two convenience wrapper functions that take a string and
compress or uncompress it to another string:
</p>
<pre class="code mark">
<span class="codemark">&nbsp;
&#9312;


&#9313;


&#9314;

&#9315;


&#9316;


&#9317;


&#9318;</span>local ffi = require("ffi")
ffi.cdef[[
<span style="color:#00a000;">unsigned long compressBound(unsigned long sourceLen);
int compress2(uint8_t *dest, unsigned long *destLen,
	      const uint8_t *source, unsigned long sourceLen, int level);
int uncompress(uint8_t *dest, unsigned long *destLen,
	       const uint8_t *source, unsigned long sourceLen);</span>
]]
local zlib = ffi.load(ffi.os == "Windows" and "zlib1" or "z")

local function compress(txt)
  local n = zlib.compressBound(#txt)
  local buf = ffi.new("uint8_t[?]", n)
  local buflen = ffi.new("unsigned long[1]", n)
  local res = zlib.compress2(buf, buflen, txt, #txt, 9)
  assert(res == 0)
  return ffi.string(buf, buflen[0])
end

local function uncompress(comp, n)
  local buf = ffi.new("uint8_t[?]", n)
  local buflen = ffi.new("unsigned long[1]", n)
  local res = zlib.uncompress(buf, buflen, comp, #comp)
  assert(res == 0)
  return ffi.string(buf, buflen[0])
end

-- Simple test code.
local txt = string.rep("abcd", 1000)
print("Uncompressed size: ", #txt)
local c = compress(txt)
print("Compressed size: ", #c)
local txt2 = uncompress(c, #txt)
assert(txt2 == txt)
</pre>
<p>
Here's the step-by-step explanation:
</p>
<p>
<span class="mark">&#9312;</span> This defines some of the
C&nbsp;functions provided by zlib. For the sake of this example, some
type indirections have been reduced and it uses the pre-defined
fixed-size integer types, while still adhering to the zlib API/ABI.
</p>
<p>
<span class="mark">&#9313;</span> This loads the zlib shared
library. On POSIX systems it's named <tt>libz.so</tt> and usually
comes pre-installed. Since <tt>ffi.load()</tt> automatically adds any
missing standard prefixes/suffixes, we can simply load the
<tt>"z"</tt> library. On Windows it's named <tt>zlib1.dll</tt> and
you'll have to download it first from the
<a href="http://zlib.net/"><span class="ext">&raquo;</span>&nbsp;zlib site</a>. The check for
<tt>ffi.os</tt> makes sure we pass the right name to
<tt>ffi.load()</tt>.
</p>
<p>
<span class="mark">&#9314;</span> First, the maximum size of
the compression buffer is obtained by calling the
<tt>zlib.compressBound</tt> function with the length of the
uncompressed string. The next line allocates a byte buffer of this
size. The <tt>[?]</tt> in the type specification indicates a
variable-length array (VLA). The actual number of elements of this
array is given as the 2nd argument to <tt>ffi.new()</tt>.
</p>
<p>
<span class="mark">&#9315;</span> This may look strange at
first, but have a look at the declaration of the <tt>compress2</tt>
function from zlib: the destination length is defined as a pointer!
This is because you pass in the maximum buffer size and get back the
actual length that was used.
</p>
<p>
In C you'd pass in the address of a local variable
(<tt>&amp;buflen</tt>). But since there's no address-of operator in
Lua, we'll just pass in a one-element array. Conveniently it can be
initialized with the maximum buffer size in one step. Calling the
actual <tt>zlib.compress2</tt> function is then straightforward.
</p>
<p>
<span class="mark">&#9316;</span> We want to return the
compressed data as a Lua string, so we'll use <tt>ffi.string()</tt>.
It needs a pointer to the start of the data and the actual length. The
length has been returned in the <tt>buflen</tt> array, so we'll just
get it from there.
</p>
<p style="font-size: 8pt;">
Note that since the function returns now, the <tt>buf</tt> and
<tt>buflen</tt> variables will eventually be garbage collected. This
is fine, because <tt>ffi.string()</tt> has copied the contents to a
newly created (interned) Lua string. If you plan to call this function
lots of times, consider reusing the buffers and/or handing back the
results in buffers instead of strings. This will reduce the overhead
for garbage collection and string interning.
</p>
<p>
<span class="mark">&#9317;</span> The <tt>uncompress</tt>
functions does the exact opposite of the <tt>compress</tt> function.
The compressed data doesn't include the size of the original string,
so this needs to be passed in. Otherwise no surprises here.
</p>
<p>
<span class="mark">&#9318;</span> The code, that makes use
of the functions we just defined, is just plain Lua code. It doesn't
need to know anything about the LuaJIT FFI &mdash; the convenience
wrapper functions completely hide it.
</p>
<p>
One major advantage of the LuaJIT FFI is that you are now able to
write those wrappers <em>in Lua</em>. And at a fraction of the time it
would cost you to create an extra C&nbsp;module using the Lua/C API.
Many of the simpler C&nbsp;functions can probably be used directly
from your Lua code, without any wrappers.
</p>
<p style="font-size: 8pt;">
Side note: the zlib API uses the <tt>long</tt> type for passing
lengths and sizes around. But all those zlib functions actually only
deal with 32&nbsp;bit values. This is an unfortunate choice for a
public API, but may be explained by zlib's history &mdash; we'll just
have to deal with it.
</p>
<p style="font-size: 8pt;">
First, you should know that a <tt>long</tt> is a 64&nbsp;bit type e.g.
on POSIX/x64 systems, but a 32&nbsp;bit type on Windows/x64 and on
32&nbsp;bit systems. Thus a <tt>long</tt> result can be either a plain
Lua number or a boxed 64&nbsp;bit integer cdata object, depending on
the target system.
</p>
<p style="font-size: 8pt;">
Ok, so the <tt>ffi.*</tt> functions generally accept cdata objects
wherever you'd want to use a number. That's why we get a away with
passing <tt>n</tt> to <tt>ffi.string()</tt> above. But other Lua
library functions or modules don't know how to deal with this. So for
maximum portability one needs to use <tt>tonumber()</tt> on returned
<tt>long</tt> results before passing them on. Otherwise the
application might work on some systems, but would fail in a POSIX/x64
environment.
</p>

<h2 id="idioms">Translating C&nbsp;Idioms</h2>
<p>
Here's a list of common C&nbsp;idioms and their translation to the
LuaJIT FFI:
</p>
<table class="idiomtable">
<tr class="idiomhead">
<td class="idiomdesc">Idiom</td>
<td class="idiomc">C&nbsp;code</td>
<td class="idiomlua">Lua code</td>
</tr>
<tr class="odd separate">
<td class="idiomdesc">Pointer dereference<br><tt>int *p;</tt></td><td class="idiomc"><tt>x = *p;<br>*p = y;</tt></td><td class="idiomlua"><tt>x = <b>p[0]</b><br><b>p[0]</b> = y</tt></td></tr>
<tr class="even">
<td class="idiomdesc">Pointer indexing<br><tt>int i, *p;</tt></td><td class="idiomc"><tt>x = p[i];<br>p[i+1] = y;</tt></td><td class="idiomlua"><tt>x = p[i]<br>p[i+1] = y</tt></td></tr>
<tr class="odd">
<td class="idiomdesc">Array indexing<br><tt>int i, a[];</tt></td><td class="idiomc"><tt>x = a[i];<br>a[i+1] = y;</tt></td><td class="idiomlua"><tt>x = a[i]<br>a[i+1] = y</tt></td></tr>
<tr class="even separate">
<td class="idiomdesc"><tt>struct</tt>/<tt>union</tt> dereference<br><tt>struct foo s;</tt></td><td class="idiomc"><tt>x = s.field;<br>s.field = y;</tt></td><td class="idiomlua"><tt>x = s.field<br>s.field = y</tt></td></tr>
<tr class="odd">
<td class="idiomdesc"><tt>struct</tt>/<tt>union</tt> pointer deref.<br><tt>struct foo *sp;</tt></td><td class="idiomc"><tt>x = sp->field;<br>sp->field = y;</tt></td><td class="idiomlua"><tt>x = <b>s.field</b><br><b>s.field</b> = y</tt></td></tr>
<tr class="even separate">
<td class="idiomdesc">Pointer arithmetic<br><tt>int i, *p;</tt></td><td class="idiomc"><tt>x = p + i;<br>y = p - i;</tt></td><td class="idiomlua"><tt>x = p + i<br>y = p - i</tt></td></tr>
<tr class="odd">
<td class="idiomdesc">Pointer difference<br><tt>int *p1, *p2;</tt></td><td class="idiomc"><tt>x = p1 - p2;</tt></td><td class="idiomlua"><tt>x = p1 - p2</tt></td></tr>
<tr class="even">
<td class="idiomdesc">Array element pointer<br><tt>int i, a[];</tt></td><td class="idiomc"><tt>x = &amp;a[i];</tt></td><td class="idiomlua"><tt>x = <b>a+i</b></tt></td></tr>
<tr class="odd">
<td class="idiomdesc">Cast pointer to address<br><tt>int *p;</tt></td><td class="idiomc"><tt>x = (intptr_t)p;</tt></td><td class="idiomlua"><tt>x = <b>tonumber(<br>&nbsp;ffi.cast("intptr_t",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;p))</b></tt></td></tr>
<tr class="even separate">
<td class="idiomdesc">Functions with outargs<br><tt>void foo(int *inoutlen);</tt></td><td class="idiomc"><tt>int len = x;<br>foo(&amp;len);<br>y = len;</tt></td><td class="idiomlua"><tt><b>local len =<br>&nbsp;&nbsp;ffi.new("int[1]", x)<br>foo(len)<br>y = len[0]</b></tt></td></tr>
<tr class="odd">
<td class="idiomdesc"><a href="ext_ffi_semantics.html#convert_vararg">Vararg conversions</a><br><tt>int printf(char *fmt, ...);</tt></td><td class="idiomc"><tt>printf("%g", 1.0);<br>printf("%d", 1);<br>&nbsp;</tt></td><td class="idiomlua"><tt>printf("%g", 1);<br>printf("%d",<br>&nbsp;&nbsp;<b>ffi.new("int", 1)</b>)</tt></td></tr>
</table>

<h2 id="cache">To Cache or Not to Cache</h2>
<p>
It's a common Lua idiom to cache library functions in local variables
or upvalues, e.g.:
</p>
<pre class="code">
local byte, char = string.byte, string.char
local function foo(x)
  return char(byte(x)+1)
end
</pre>
<p>
This replaces several hash-table lookups with a (faster) direct use of
a local or an upvalue. This is less important with LuaJIT, since the
JIT compiler optimizes hash-table lookups a lot and is even able to
hoist most of them out of the inner loops. It can't eliminate
<em>all</em> of them, though, and it saves some typing for often-used
functions. So there's still a place for this, even with LuaJIT.
</p>
<p>
The situation is a bit different with C&nbsp;function calls via the
FFI library. The JIT compiler has special logic to eliminate <em>all
of the lookup overhead</em> for functions resolved from a
<a href="ext_ffi_semantics.html#clib">C&nbsp;library namespace</a>!
Thus it's not helpful and actually counter-productive to cache
individual C&nbsp;functions like this:
</p>
<pre class="code">
local <b>funca</b>, <b>funcb</b> = ffi.C.funcb, ffi.C.funcb -- <span style="color:#c00000;">Not helpful!</span>
local function foo(x, n)
  for i=1,n do <b>funcb</b>(<b>funca</b>(x, i), 1) end
end
</pre>
<p>
This turns them into indirect calls and generates bigger and slower
machine code. Instead you'll want to cache the namespace itself and
rely on the JIT compiler to eliminate the lookups:
</p>
<pre class="code">
local <b>C</b> = ffi.C          -- <span style="color:#00a000;">Instead use this!</span>
local function foo(x, n)
  for i=1,n do <b>C.funcb</b>(<b>C.funca</b>(x, i), 1) end
end
</pre>
<p>
This generates both shorter and faster code. So <b>don't cache
C&nbsp;functions</b>, but <b>do</b> cache namespaces! Most often the
namespace is already in a local variable at an outer scope, e.g. from
<tt>local&nbsp;lib&nbsp;=&nbsp;ffi.load(...)</tt>. Note that copying
it to a local variable in the function scope is unnecessary.
</p>
<br class="flush">
</div>
<div id="foot">
<hr class="hide">
Copyright &copy; 2005-2011 Mike Pall
<span class="noprint">
&middot;
<a href="contact.html">Contact</a>
</span>
</div>
</body>
</html>