Nanogest low-level C API

Forewarning

Nanogest should typically be used through the high-level APIs supplied for each supported platform. The high-level APIs take care of camera device enumeration, configuration and video capture transparently, and provide easy-to-use error handling.

The low-level API is also more likely to change in incompatible ways in the future.

In other words, do not use the low-level API unless you have discussed it with us and we have agreed that it was a good idea for you to do so.

Installation

For each platform, Nanogest includes binaries to be linked with your application. In addition to the high-level APIs, these binaries also expose the common low-level C interface, defined in nanogest.h.

Build instructions are platform dependent and are outside the scope of this document. Please contact us if you need further assistance.

Principle of Operation

Nanogest can be used in two distinct modes of operations, selected by the background_thread argument of nanogest_init().

Background Mode

In Background Mode background_thread=1, Nanogest processes video data on a background thread that it creates and manages, and Nanogest reports any gesture it detects to the application asynchronously.

The application may use Nanogest on the main application thread, as Nanogest functions return nearly immediately (except when noted otherwise).

Synchronous Mode

In Synchronous Mode background_thread=0, the application is responsible for pushing processing to a secondary thread. Nanogest functions may take a significant amount of time before returning.

Synopsis

Background Mode

#include "nanogest.h" void your_allocate_camera_buffer(uint8_t **frame, size_t *len, uint32_t width, uint32_t height, enum nanogest_pixel_format pixel_format); void your_capture_frame(uint8_t *frame, uint32_t len); double your_timeofday(void); static void on_gesture(struct nanogest *ngest, enum nanogest_kind kind, double time_s, void *user_data) { printf("gesture %d\n", kind); } int main() { struct nanogest *ngest = nanogest_new(); nanogest_init(ngest, 640, 480, NANOGEST_PIXEL_FORMAT_NV21, 0, 1, 1, NULL); nanogest_set_result_callback(ngest, on_gesture, NULL); uint8_t *frame = NULL; uint32_t len = 0; your_allocate_camera_buffer(&frame, &len, 640, 480, NANOGEST_PIXEL_FORMAT_NV21); while (1) { your_capture_frame(frame, len); nanogest_submit_frame(ngest, frame, len, your_timeofday()); } nanogest_delete(ngest); return 0; }

Synchronous Mode

#include "nanogest.h" void your_allocate_camera_buffer(uint8_t **frame, size_t *len, uint32_t width, uint32_t height, enum nanogest_pixel_format pixel_format); void your_capture_frame(uint8_t *frame, uint32_t len); double your_timeofday(void); int main() { struct nanogest *ngest = nanogest_new(); nanogest_init(ngest, 640, 480, NANOGEST_PIXEL_FORMAT_NV21, 0, 0, 0, NULL); nanogest_set_result_callback(ngest, on_gesture, NULL); uint8_t *frame = NULL; uint32_t len = 0; your_allocate_camera_buffer(&frame, &len, 640, 480, NANOGEST_PIXEL_FORMAT_NV21); while (1) { your_capture_frame(frame, len); nanogest_submit_frame(ngest, frame, len, your_timeofday()); enum nanogest_kind kind = nanogest_result_fetch(ngest); if (kind != NANOGEST_NONE) { printf("gesture %d\n", kind); } } nanogest_delete(ngest); return 0; }

Initialization

struct nanogest

struct nanogest;

An opaque structure that contains Nanogest configuration and state.

Nanogest has no global state stored outside of struct nanogest. Multiple instances of Nanogest can be used at a time.

A single instance of Nanogest can be accessed safely from multiple threads, if (and only if) nanogest_init() is called with the argument background_thread set to 1.

nanogest_new()

struct nanogest *nanogest_new();

Allocates and returns a zero-initialized instance of struct nanogest. The instance needs to be further initialized with nanogest_init().

nanogest_delete()

void nanogest_delete(struct nanogest *ngest);

Frees all resources associated with the Nanogest instance ngest, including the struct nanogest pointed to by ngest.

Background Mode

In Background Mode, blocks until the background thread has finished processing the current frame.

enum nanogest_pixel_format

enum nanogest_pixel_format { NANOGEST_PIXEL_FORMAT_NV12 = 1, NANOGEST_PIXEL_FORMAT_420YpCbCr8BiPlanar = NANOGEST_PIXEL_FORMAT_NV12, NANOGEST_PIXEL_FORMAT_NV21 = 2, NANOGEST_PIXEL_FORMAT_I420 = 3, };

This enum identifies each pixel format supported by Nanogest. If you need support for pixel formats not listed here, please contact us.

nanogest_init()

void nanogest_init(struct nanogest *ngest, uint32_t width, uint32_t height, enum nanogest_pixel_format pix_format, uint32_t rotation_counterclockwise_degrees, int background_thread, int copy_on_submit, void *reserved);

For best performance, the video frames should be 640x480 pixels. Other resolutions will likely result in degraded operation.

ngest is the Nanogest instance, width is the width in pixels of the video frames, height is the height in pixels of the video frames, pix_format is the pixel format of the video (see enum nanogest_pixel_format), rotation_counterclockwise_degrees is the rotation of the device (see nanogest_change_orientation()), background_thread determines whether Nanogest uses a background thread, copy_on_submit requests that frames be copied to memory owned by the Nanogest instance when submitted (see nanogest_submit_frame()), reserved should be NULL.

Gesture Processing

enum nanogest_kind

enum nanogest_kind { NANOGEST_NONE = 0, NANOGEST_SWIPE_LEFT = 1, NANOGEST_SWIPE_RIGHT = 2, NANOGEST_SWIPE_UP = 3, NANOGEST_SWIPE_DOWN = 4, NANOGEST_HELLO = 5, };

The enum nanogest_kind identifies each gesture recognized by Nanogest.

The gestures are always to be understood with respect to the user. Nanogest takes into account the current orientation of the device automatically.

nanogest_submit_frame()

void nanogest_submit_frame(struct nanogest *ngest, const uint8_t *frame, uint32_t len, double time_seconds);

Submit a frame for processing in the format specified by the nanogest_init() parameters width, height and pixel_format.

ngest is the Nanogest instance, frame is a pointer to a buffer of byte size len that contains the video frame data in the format specified to nanogest_init(), time_seconds is the date of when this frame was recorded (it should be in seconds since Epoch, January 1st, 1970).

Background Mode

In Background Mode, this function returns quickly, before the frame is fully processed. The caller may poll with nanogest_result_ready(), or block on nanogest_result_wait(), or rely on the result callback to be called (if any).

In any case, you should ensure that nanogest_submit_frame() has finished processing a frame before submitting the next or the new frame may be dropped and ignored.

If the nanogest_init() parameter copy_on_submit was set to 1, nanogest_submit_frame() copies the content of frame into a private buffer, allowing the application to release or reuse frame immediately.

If the nanogest_init() parameter copy_on_submit was set to 0, Nanogest retains a reference to frame until the frame is done processing.

Synchronous Mode

In Synchronous Mode, nanogest_submit_frame() blocks until Nanogest has fully processed the frame.

nanogest_result_wait()

void nanogest_result_wait(struct nanogest *ngest);

Background Mode

In Background Mode, block until any submitted frame is fully processed.

Synchronous Mode

In Synchronous Mode, this is a no-op.

nanogest_result_ready()

int nanogest_result_ready(struct nanogest *ngest);

Returns a non-zero value if the Nanogest instance ngest is done processing any submitted frame.

nanogest_result_fetch()

enum nanogest_kind nanogest_result_fetch(struct nanogest *ngest);

Can be used in both Background Mode (when no result callback has been set) and Synchronous Mode to retrieve the latest detected gesture.

Needs to be called once for each submitted frame to guarantee not missing any detected gesture.

nanogest_result_callback

typedef void (*nanogest_result_callback)(struct nanogest *ngest, enum nanogest_kind kind, double time_s, void *user_data);

ngest is the Nanogest instance that generated the gesture, kind is the detected gesture and is never NANOGEST_NONE (see enum nanogest_kind), time_s is the date of the frame that triggered the detection in seconds (as passed to nanogest_submit_frame()), user_data is the user_data that was passed to nanogest_set_result_callback().

Callbacks are called on the background processing thread and should do as little work as possible. For instance, a callback might just notify the main application thread.

Callbacks should not call any Nanogest function on ngest or risk a deadlock.

nanogest_set_result_callback()

void nanogest_set_result_callback(struct nanogest *ngest, nanogest_result_callback cb, void *user_data);

Background Mode

In Background Mode, can be used instead of polling with nanogest_result_ready() or blocking with nanogest_result_wait().

ngest is the Nanogest instance, cb is the callback to register, user_data will be passed to cb.

Synchronous Mode

Ignored in Synchronous Mode, it is a no-op.

Configuration

nanogest_accept_one_hello()

void nanogest_accept_one_hello(struct nanogest *ngest, int enabled);

If enabled is 1, allow the next gesture to be a "hello" gesture, or any other gesture. Once any gesture is detected, Nanogest will revert to the normal behavior of only accepting non-hello gestures.

If enabled is 0, only accept non-hello gestures. This is the default behavior.

Because "hello" is a slow gesture that can only be detected over a few hundred milliseconds, Nanogest does not accept "hello" gestures by default. Use "hello" gestures only when needed.

nanogest_change_orientation()

void nanogest_change_orientation(struct nanogest *ngest, uint32_t rotation_counterclockwise_degrees);

Gestures detected by Nanogest are always to be understood with respect to the user, enum nanogest_kind values are always corrected for device orientation.

Therefore, Nanogest must be informed of any orientation change with this function.

ngest is the Nanogest instance, rotation_counterclockwise_degrees is the rotation of the device counterclockwise and in degrees with respect to the "natural" orientation of the video frame data (when rendered in left-to-right, top-to-bottom raster order).

Background Mode

In Background Mode, this function should be called after any submitted frame is done processing, or it may block.