An Embeddable Datastore Engine

Tweet

Follow @Vedis

This article provides an overview and roadmap to the C/C++ interface to Vedis.

Early versions of Vedis were very easy to learn since they only supported 8 C/C++ interfaces. But as the Vedis engine has grown in capability, new C/C++ interfaces have been added so that now there are over 55 distinct APIs. This can be overwhelming to a new programmer. Fortunately, most of the C/C++ interfaces in Vedis are very specialized and never need to be used. Despite having so many entry points, the core API is still relatively simple and easy to code to. This article aims to provide all of the background information needed to easily understand how the Vedis database engine works.

A separate document, The Vedis C/C++ Interface, provides detailed specifications for all of the various C/C++ APIs for Vedis. Once the reader understands the basic principles of operation for Vedis, that document should be used as a reference guide. This article is intended as introduction only and is neither a complete nor authoritative reference for the Vedis API.

1.0 Core Objects And Interfaces

The principal task of a datastore engine is to store and retrieve records. Vedis support both structured and raw data storage.

Structured data storage is presented to clients via the command execution interface. Basically, you execute one or more commands ala Redis (i.e. SET key value; HSET ds key value, GET key) via vedis_exec() and you extract the execution result (The return value of the command) via vedis_exec_result(). Refer to the following page for the list of built-in commands and the download page for some C/C++ samples.

Raw data storage is presented to clients via the Key/Value store interfaces. Vedis is a standard key/value store similar to Berkeley DB, Tokyo Cabinet, LevelDB, etc. but with a rich feature set including support for transactions (ACID), concurrent reader, etc. Under the KV store, both keys and values are treated as simple arrays of bytes, so content can be anything from ASCII strings, binary blob and even disk files. The KV store layer is presented to clients via a set of interfaces, these includes: vedis_kv_store(), vedis_kv_append(), vedis_kv_fetch_callback(), vedis_kv_append_fmt(), etc.

In order to accomplish this purpose, the developer needs to know about two objects:

• The Datastore Engine Handle: vedis

• The Vedis Object Value: vedis_value

The datastore engine handle and optionally, the object value are controlled by a small set of C/C++ interface routine listed below.

• vedis_open()
  Command Executions Interfaces

• vedis_exec()
• vedis_exec_fmt()
• vedis_exec_result()
  Key/Value Store Interface

• vedis_kv_store()

• vedis_kv_append()

• vedis_kv_store_fmt()

• vedis_kv_append_fmt()

• vedis_kv_delete()

• vedis_kv_fetch()

• vedis_kv_fetch_callback()

  Optionally, the Transaction Manager
• vedis_begin()
• vedis_commit()

• vedis_rollback()
  Closing the datastore

• vedis_close()

The dozens of the C/C++ interface routines and two objects listed above form the core functionality of Vedis. The developer who understands them will have a good foundation for using Vedis.

2.0 Typical Usage of the Core C/C++ Interfaces

An application that wants to use Vedis will typically use the following interfaces with their optional components such as the transaction manager and the cursors interfaces. Note that an application may switch between the Key/Value store and the command execution interfaces without any problem.

vedis_open()	This routine opens a connection to a Vedis database file and returns a datastore handle. This is often the first Vedis API call that an application makes and is a prerequisite for most other Vedis APIs. Many Vedis interfaces require a pointer to the datastore handle as their first parameter and can be thought of as methods on the database handle. This routine is the constructor for the datastore handle.
vedis_exec() vedis_exec_fmt() vedis_exec_result()	Command Execution Interfaces Vedis is shipped with over 70 commands similar to the Redis counterpart. Use the following set of interfaces to execute and extract the return value of the desired command.
vedis_kv_store() vedis_kv_append() vedis_kv_store_fmt() vedis_kv_append_fmt() vedis_kv_delete() vedis_kv_fetch() vedis_kv_fetch_callback()	Key/Value Store Interfaces Under the Key/Value store, both keys and values are treated as simple arrays of bytes, so content can anything from ASCII strings, binary blob and even disk files. These set of interfaces allow clients to store and retrieve raw database records efficiently regardless of the underlying KV storage engine.
vedis_begin() vedis_commit() vedis_rollback()	Manual Transaction Manager These set of interfaces allow the host application to manually start a write-transaction. Note that Vedis is smart enough and will automatically start a write-transaction in the background when needed and so call to these routines is usually not necessary.
vedis_close()	This routine is the destructor for the vedis handle obtained by a prior successful call to vedis_open().  Each datastore connection must be closed in order to avoid memory leaks and malformed database image.

3.0 In-Process Extending

Vedis includes interfaces that can be used to extend its functionality. Such routines includes vedis_regsister_command().

The vedis_regsister_command() interface is used to install a foreign command (typically a C/C++ function) that can be invoked from vedis_exec(). The new command implementation typically makes use of the following additional interfaces.

All of the built-in commands of Vedis such as GET, SET, HSET, SADD, SLEN, MD5, etc. are created using exactly this interface.

4.0 Others Interfaces

This article only mentions the foundational Vedis interfaces. The Vedis library includes many other APIs implementing useful features that are not described here. A complete list of functions that form the Vedis application programming interface is found at the C/C++ Interface Specification. Refer to that document for complete and authoritative information about all Vedis interfaces.