Nanogest low-level C API
Forewarning
Nanogest should typically be used through the high-level APIs supplied for each supported platform. The high-level APIs take care of camera device enumeration, configuration and video capture transparently, and provide easy-to-use error handling.
The low-level API is also more likely to change in incompatible ways in the future.
In other words, do not use the low-level API unless you have discussed it with us and we have agreed that it was a good idea for you to do so.
Installation
For each platform, Nanogest includes binaries to be linked with your application. In addition to the high-level APIs, these binaries also expose the common low-level C interface, defined in nanogest.h.
Build instructions are platform dependent and are outside the scope of this document. Please contact us if you need further assistance.
Principle of Operation
Nanogest can be used in two distinct modes of operations, selected by
the background_thread
argument of nanogest_init()
.
Background Mode
In Background Mode background_thread=1
, Nanogest processes
video data on a background thread that it creates and manages, and
Nanogest reports any gesture it detects to the application
asynchronously.
The application may use Nanogest on the main application thread, as Nanogest functions return nearly immediately (except when noted otherwise).
Synchronous Mode
In Synchronous Mode background_thread=0
, the application is responsible for pushing
processing to a secondary thread. Nanogest functions may take a
significant amount of time before returning.
Synopsis
Background Mode
#include "nanogest.h" void your_allocate_camera_buffer(uint8_t **frame, size_t *len, uint32_t width, uint32_t height, enum nanogest_pixel_format pixel_format); void your_capture_frame(uint8_t *frame, uint32_t len); double your_timeofday(void); static void on_gesture(struct nanogest *ngest, enum nanogest_kind kind, double time_s, void *user_data) { printf("gesture %d\n", kind); } int main() { struct nanogest *ngest = nanogest_new(); nanogest_init(ngest, 640, 480, NANOGEST_PIXEL_FORMAT_NV21, 0, 1, 1, NULL); nanogest_set_result_callback(ngest, on_gesture, NULL); uint8_t *frame = NULL; uint32_t len = 0; your_allocate_camera_buffer(&frame, &len, 640, 480, NANOGEST_PIXEL_FORMAT_NV21); while (1) { your_capture_frame(frame, len); nanogest_submit_frame(ngest, frame, len, your_timeofday()); } nanogest_delete(ngest); return 0; }
Synchronous Mode
#include "nanogest.h" void your_allocate_camera_buffer(uint8_t **frame, size_t *len, uint32_t width, uint32_t height, enum nanogest_pixel_format pixel_format); void your_capture_frame(uint8_t *frame, uint32_t len); double your_timeofday(void); int main() { struct nanogest *ngest = nanogest_new(); nanogest_init(ngest, 640, 480, NANOGEST_PIXEL_FORMAT_NV21, 0, 0, 0, NULL); nanogest_set_result_callback(ngest, on_gesture, NULL); uint8_t *frame = NULL; uint32_t len = 0; your_allocate_camera_buffer(&frame, &len, 640, 480, NANOGEST_PIXEL_FORMAT_NV21); while (1) { your_capture_frame(frame, len); nanogest_submit_frame(ngest, frame, len, your_timeofday()); enum nanogest_kind kind = nanogest_result_fetch(ngest); if (kind != NANOGEST_NONE) { printf("gesture %d\n", kind); } } nanogest_delete(ngest); return 0; }
Initialization
struct nanogest
struct nanogest;
An opaque structure that contains Nanogest configuration and state.
Nanogest has no global state stored outside of
struct nanogest
. Multiple instances of Nanogest can be used at
a time.
A single instance of Nanogest can be accessed safely from multiple
threads, if (and only if) nanogest_init()
is called with
the argument background_thread
set to 1
.
nanogest_new()
struct nanogest *nanogest_new();
Allocates and returns a zero-initialized instance of
struct nanogest
.
The instance needs to be further initialized with nanogest_init()
.
nanogest_delete()
void nanogest_delete(struct nanogest *ngest);
Frees all resources associated with the Nanogest instance
ngest
, including the struct nanogest
pointed to by
ngest
.
Background Mode
In Background Mode, blocks until the background thread has finished processing the current frame.
enum nanogest_pixel_format
enum nanogest_pixel_format { NANOGEST_PIXEL_FORMAT_NV12 = 1, NANOGEST_PIXEL_FORMAT_420YpCbCr8BiPlanar = NANOGEST_PIXEL_FORMAT_NV12, NANOGEST_PIXEL_FORMAT_NV21 = 2, NANOGEST_PIXEL_FORMAT_I420 = 3, };
This enum identifies each pixel format supported by Nanogest. If you need support for pixel formats not listed here, please contact us.
NANOGEST_PIXEL_FORMAT_NV12
see its definition at FOURCC.NANOGEST_PIXEL_FORMAT_420YpCbCr8BiPlanar
corresponds to the iOS format kCVPixelFormatType_420YpCbCr8BiPlanarVideoRangeNANOGEST_PIXEL_FORMAT_NV21
is the default preview ImageFormat.NV21 on Android; see its definition at FOURCC.NANOGEST_PIXEL_FORMAT_I420
see its definition at FOURCC.
nanogest_init()
void nanogest_init(struct nanogest *ngest, uint32_t width, uint32_t height, enum nanogest_pixel_format pix_format, uint32_t rotation_counterclockwise_degrees, int background_thread, int copy_on_submit, void *reserved);
For best performance, the video frames should be 640x480 pixels. Other resolutions will likely result in degraded operation.
ngest
is the Nanogest instance,
width
is the width in pixels of the video frames,
height
is the height in pixels of the video frames,
pix_format
is the pixel format of the video (see
enum nanogest_pixel_format
),
rotation_counterclockwise_degrees
is the rotation of the
device
(see nanogest_change_orientation()
),
background_thread
determines whether Nanogest uses a background thread,
copy_on_submit
requests that frames be copied to memory owned
by the Nanogest instance when submitted (see
nanogest_submit_frame()
),
reserved
should be NULL
.
Gesture Processing
enum nanogest_kind
enum nanogest_kind { NANOGEST_NONE = 0, NANOGEST_SWIPE_LEFT = 1, NANOGEST_SWIPE_RIGHT = 2, NANOGEST_SWIPE_UP = 3, NANOGEST_SWIPE_DOWN = 4, NANOGEST_HELLO = 5, };
The enum nanogest_kind
identifies each gesture recognized by Nanogest.
The gestures are always to be understood with respect to the user. Nanogest takes into account the current orientation of the device automatically.
NANOGEST_SWIPE_LEFT
indicates the user performed a swipe gesture to her left.NANOGEST_SWIPE_RIGHT
indicates the user performed a swipe gesture to her right.NANOGEST_SWIPE_UP
indicates the user performed a swipe gesture toward the ceiling.NANOGEST_SWIPE_DOWN
indicates the user performed a swipe gesture toward the floor.NANOGEST_HELLO
indicates the user performed a "hello" gesture, a rapid back-and-worth wave of the hand. UsemayAcceptOneHelloGesture:
to enable "hello" gestures.
nanogest_submit_frame()
void nanogest_submit_frame(struct nanogest *ngest, const uint8_t *frame, uint32_t len, double time_seconds);
Submit a frame for processing in the format specified by the
nanogest_init()
parameters width
, height
and
pixel_format
.
ngest
is the Nanogest instance,
frame
is a pointer to a buffer of byte size len
that
contains the video frame data in the format specified to
nanogest_init()
,
time_seconds
is the date of when this frame was recorded (it
should be in seconds since Epoch, January 1st, 1970).
Background Mode
In Background Mode, this function returns quickly, before the frame
is fully processed. The caller may poll with
nanogest_result_ready()
, or block on
nanogest_result_wait()
, or rely on the result callback to be
called (if any).
In any case, you should ensure that nanogest_submit_frame()
has finished processing a frame before submitting the next or the new
frame may be dropped and ignored.
If the nanogest_init()
parameter copy_on_submit
was set
to 1
, nanogest_submit_frame()
copies the content of
frame
into a private buffer, allowing the application to
release or reuse frame
immediately.
If the nanogest_init()
parameter copy_on_submit
was set
to 0
, Nanogest retains a reference to frame
until the
frame is done processing.
Synchronous Mode
In Synchronous Mode, nanogest_submit_frame()
blocks until
Nanogest has fully processed the frame.
nanogest_result_wait()
void nanogest_result_wait(struct nanogest *ngest);
Background Mode
In Background Mode, block until any submitted frame is fully processed.
Synchronous Mode
In Synchronous Mode, this is a no-op.
nanogest_result_ready()
int nanogest_result_ready(struct nanogest *ngest);
Returns a non-zero value if the Nanogest instance ngest
is
done processing any submitted frame.
nanogest_result_fetch()
enum nanogest_kind nanogest_result_fetch(struct nanogest *ngest);
Can be used in both Background Mode (when no result callback has been set) and Synchronous Mode to retrieve the latest detected gesture.
Needs to be called once for each submitted frame to guarantee not missing any detected gesture.
nanogest_result_callback
typedef void (*nanogest_result_callback)(struct nanogest *ngest, enum nanogest_kind kind, double time_s, void *user_data);
ngest
is the Nanogest instance that generated the gesture,
kind
is the detected gesture and is never NANOGEST_NONE
(see
enum nanogest_kind
),
time_s
is the date of the frame that triggered the detection
in seconds (as passed to
nanogest_submit_frame()
),
user_data
is the user_data
that was passed to
nanogest_set_result_callback()
.
Callbacks are called on the background processing thread and should do as little work as possible. For instance, a callback might just notify the main application thread.
Callbacks should not call any Nanogest function on ngest
or
risk a deadlock.
nanogest_set_result_callback()
void nanogest_set_result_callback(struct nanogest *ngest, nanogest_result_callback cb, void *user_data);
Background Mode
In Background Mode, can be used instead of polling with
nanogest_result_ready()
or blocking with
nanogest_result_wait()
.
ngest
is the Nanogest instance,
cb
is the callback to register,
user_data
will be passed to cb
.
Synchronous Mode
Ignored in Synchronous Mode, it is a no-op.
Configuration
nanogest_accept_one_hello()
void nanogest_accept_one_hello(struct nanogest *ngest, int enabled);
If enabled
is 1
, allow the next gesture to be a "hello"
gesture, or any other gesture. Once any gesture is detected,
Nanogest will revert to the normal behavior of only accepting
non-hello gestures.
If enabled
is 0
, only accept non-hello
gestures. This is the default behavior.
Because "hello" is a slow gesture that can only be detected over a few hundred milliseconds, Nanogest does not accept "hello" gestures by default. Use "hello" gestures only when needed.
nanogest_change_orientation()
void nanogest_change_orientation(struct nanogest *ngest, uint32_t rotation_counterclockwise_degrees);
Gestures detected by Nanogest are always to be understood with
respect to the user, enum nanogest_kind
values are always
corrected for device orientation.
Therefore, Nanogest must be informed of any orientation change with this function.
ngest
is the Nanogest instance,
rotation_counterclockwise_degrees
is the rotation of the
device counterclockwise and in degrees with respect to the "natural"
orientation of the video frame data (when rendered in left-to-right,
top-to-bottom raster order).
Background Mode
In Background Mode, this function should be called after any submitted frame is done processing, or it may block.