Lua

The FFI library allows calling external C functions and using C data structures from pure Lua code.

The FFI library largely obviates the need to write tedious manual Lua/C bindings in C. No need to learn a separate binding language — it parses plain C declarations! These can be cut-n-pasted from C header files or reference manuals. It's up to the task of binding large libraries without the need for dealing with fragile binding generators.

The FFI library is tightly integrated into LuaJIT (it's not available as a separate module). The code generated by the JIT-compiler for accesses to C data structures from Lua code is on par with the code a C compiler would generate. Calls to C functions can be inlined in JIT-compiled code, unlike calls to functions bound via the classic Lua/C API.

This page gives a short introduction to the usage of the FFI library. Please use the FFI sub-topics in the navigation bar to learn more.

Motivating Example: Calling External C Functions

It's really easy to call an external C library function: 

①
②


③local ffi = require("ffi")
ffi.cdef[[
int printf(const char *fmt, ...);
]]
ffi.C.printf("Hello %s!", "world")

So, let's pick that apart:

① Load the FFI library.

② Add a C declaration for the function. The part inside the double-brackets (in green) is just standard C syntax.

③ Call the named C function — Yes, it's that simple!

Actually, what goes on behind the scenes is far from simple: ③ makes use of the standard C library namespace ffi.C. Indexing this namespace with a symbol name ("printf") automatically binds it to the standard C library. The result is a special kind of object which, when called, runs the printf function. The arguments passed to this function are automatically converted from Lua objects to the corresponding C types.

Ok, so maybe the use of printf() wasn't such a spectacular example. You could have done that with io.write() and string.format(), too. But you get the idea ...

So here's something to pop up a message box on Windows: 

local ffi = require("ffi")
ffi.cdef[[
int MessageBoxA(void *w, const char *txt, const char *cap, int type);
]]
ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)

Bing! Again, that was far too easy, no?

Compare this with the effort required to bind that function using the classic Lua/C API: create an extra C file, add a C function that retrieves and checks the argument types passed from Lua and calls the actual C function, add a list of module functions and their names, add a luaopen_* function and register all module functions, compile and link it into a shared library (DLL), move it to the proper path, add Lua code that loads the module aaaand ... finally call the binding function. Phew!

Motivating Example: Using C Data Structures

The FFI library allows you to create and access C data structures. Of course the main use for this is for interfacing with C functions. But they can be used stand-alone, too.

Lua is built upon high-level data types. They are flexible, extensible and dynamic. That's why we all love Lua so much. Alas, this can be inefficient for certain tasks, where you'd really want a low-level data type. E.g. a large array of a fixed structure needs to be implemented with a big table holding lots of tiny tables. This imposes both a substantial memory overhead as well as a performance overhead.

Here's a sketch of a library that operates on color images plus a simple benchmark. First, the plain Lua version: 

local floor = math.floor

local function image_ramp_green(n)
  local img = {}
  local f = 255/(n-1)
  for i=1,n do
    img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
  end
  return img
end

local function image_to_grey(img, n)
  for i=1,n do
    local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
    img[i].red = y; img[i].green = y; img[i].blue = y
  end
end

local N = 400*400
local img = image_ramp_green(N)
for i=1,1000 do
  image_to_grey(img, N)
end

This creates a table with 160.000 pixels, each of which is a table holding four number values in the range of 0-255. First an image with a green ramp is created (1D for simplicity), then the image is converted to greyscale 1000 times. Yes, that's silly, but I was in need of a simple example ...

And here's the FFI version. The modified parts have been marked in bold: 

①





②

③
④






③
⑤local ffi = require("ffi")
ffi.cdef[[
typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;
]]

local function image_ramp_green(n)
  local img = ffi.new("rgba_pixel[?]", n)
  local f = 255/(n-1)
  for i=0,n-1 do
    img[i].green = i*f
    img[i].alpha = 255
  end
  return img
end

local function image_to_grey(img, n)
  for i=0,n-1 do
    local y = 0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue
    img[i].red = y; img[i].green = y; img[i].blue = y
  end
end

local N = 400*400
local img = image_ramp_green(N)
for i=1,1000 do
  image_to_grey(img, N)
end

Ok, so that wasn't too difficult:

① First, load the FFI library and declare the low-level data type. Here we choose a struct which holds four byte fields, one for each component of a 4x8 bit RGBA pixel.

② Creating the data structure with ffi.new() is straightforward — the '?' is a placeholder for the number of elements of a variable-length array.

③ C arrays are zero-based, so the indexes have to run from 0 to n-1. One might want to allocate one more element instead to simplify converting legacy code.

④ Since ffi.new() zero-fills the array by default, we only need to set the green and the alpha fields.

⑤ The calls to math.floor() can be omitted here, because floating-point numbers are already truncated towards zero when converting them to an integer. This happens implicitly when the number is stored in the fields of each pixel.

Now let's have a look at the impact of the changes: first, memory consumption for the image is down from 22 Megabytes to 640 Kilobytes (400*400*4 bytes). That's a factor of 35x less! So, yes, tables do have a noticeable overhead. BTW: The original program would consume 40 Megabytes in plain Lua (on x64).

Next, performance: the pure Lua version runs in 9.57 seconds (52.9 seconds with the Lua interpreter) and the FFI version runs in 0.48 seconds on my machine (YMMV). That's a factor of 20x faster (110x faster than the Lua interpreter).

The avid reader may notice that converting the pure Lua version over to use array indexes for the colors ([1] instead of .red, [2] instead of .green etc.) ought to be more compact and faster. This is certainly true (by a factor of ~1.7x). Switching to a struct-of-arrays would help, too.

However the resulting code would be less idiomatic and rather error-prone. And it still doesn't get even close to the performance of the FFI version of the code. Also, high-level data structures cannot be easily passed to other C functions, especially I/O functions, without undue conversion penalties.