Overview

LiteStore is a lightweight, self-contained, RESTful, multi-format NoSQL document store server written in Nim and powered by a SQLite backend for storage. It aims to be a very simple and lightweight backend ideal for prototyping and testing REST APIs and single-page applications.

Rationale

If you ever wanted to build a simple single-page application in your favorite framework, just to try something out or as a prototype, you inevitably had to answer the question “What backend should I use?”

Sure, setting up a simple REST service using Sinatra or Express.js is not very hard, but if you want to distribute it, that backend will become a prerequisite for your app: you’ll either distribute it with it, or install it beforehand on any machine you want to try your app on. Which is a shame, really, because single-page-applications are meant to be running anywhere provided that they can access their backend.

LiteStore aims to solve this problem. When you use LiteStore as the backend for your app, you only need to take two files with you, at all times:

And yes, you can even store the code of your client-side application inside the datastore itself, along with your application data.

Key Features

Despite being fairly small and self-contained, LiteStore comes with many useful features that are essential for many use cases.

Multi-format Documents

LiteStore can be used to store documents in virtually any format, as long as you specify an appropriate content type for them. Textual documents are stored as-is, while binary documents are base64-encoded (not the best thing in the world, but definitely the easiest and most portal option).

Document Tagging

You can add custom tags to documents to easily categorize them and retrieve them. Some system tags are also added automatically to identify the document content type, format and collection.

Enhanced Querying of JSON documents

By leveraging the SQLite JSON1 extension and implementing custom query string parsing, LiteStore provides enhanced filtering, ordering, and custom field selection of JSON documents. Additionally, you can also configure custom indexes specifying JSON fields for extra speed!

Full-text Search

By leveraging SQLite FTS4 extension and implementing an enhanced algorithm for result rankings, LiteStore provides full-text search for all textual documents out-of-the-box.

RESTful HTTP API

Every operation can be performed on the data store using a simple but powerful RESTful HTTP API, perfect for client-side, single-page applications.

JWT-based Authorization

LiteStore can be configure validate JWT tokens and configure access to specific resources based on specific OAuth2 scopes.

Middleware

By leveraging the duktape library, you can create your own middleware functions in JavaScript to perform additional tasks (validation, logging, data aggregation…) before accessing data.

Multiple Data Stores

LiteStore can be configured to manage more than one SQLite file through the same process. At run time, it will be possible to access data stored in each store but also add and remove stores.

Nim API

If you want, you can use LiteStore as a Nim library and perform data store operations from your own Nim program.

Command-line API

Every operation can also be performed from command line, using the litestore execute command.

Directory Bulk Import/Export

To make serving a single-page application from LiteStore even easier and faster, you can automatically import (and export) the contents of a directory recursively.

Directory Mounting and Mirroring

After importing the contents of a directory into a LiteStore data store, you can mount it on LiteStore and mirror all data store changes to the filesystem. Incidentally, that’s how most of the LiteStore Admin test app was built .

Use Cases

While LiteStore may not be the best choice for large data-intensive applications, it definitely shines when used for rapid prototyping and as a backend for small/lightweight applications.

Lightweight Document Store

LiteStore is, first and foremost a document store. Although it uses a relational database (SQlite) as a storage medium, it is a NoSQL document store accessible via a RESTful API.

As a document store, LiteStore provides the following features

SPA Prototyping Backend and Lightweight File Server

LiteStore can be used as a backend for prototyping single-page application. You can use it as a simple RESTful service layer to manage your SPA data, but also as a web-server to serve your application files: all you have to do is import a phisical directory using the litestore import command and LiteStore will loads all the directory contents recursively as documents.

Together with LiteStore’s ability to mirror changes to the filesystem, you could even develop your single-page application live using LiteStore REST API to save and retrieve files. The LiteStore Admin SPA is an example of single-page application developed almost entirely with LiteStore.

Lightweight RESTFul Virtual Filesystem

By using the import, export and delete commands, you can easily load and unload data to and from a physical directory.

You could even use datastore files as file containers, with the advantage that (when served by LiteStore) you can also:

A note on datastore file size…

Do not expect datastore files to occupy the same physical space as the original files on disk! Binary files are stored in datastore files as base64-encoded strings. This greatly simplifies storage and portability, but also makes stored binary contents roughly 33% larger than the original files.

Additionally, extra space is used to store tag information and the full-text index, so… basically a datastore file is always larger than the total sizes of all the original files combined.

Personal Wiki/CMS/App Backend

LiteStore is the ideal backend for personal apps. You could create a wiki app or a simple CMS as a Javascript single-page application, and distribute it as a single datastore file.

Your app could then be served on any desktop system able to run LiteStore (e.g. OSX, Windows, Linux, …even on a Raspberry Pi).

Static Site Backend

LiteStore can be configured to run in read-only mode, so that only GET, HEAD, or OPTIONS request are accepted by the server. This makes it ideal as a backend for static web site generated with something like nanoc or Jekyll.

Architecture

LiteStore is entirely developed using the Nim programming language. It also embeds the SQLite C library (statically linked), compiled with the FTS4 extension and a custom ranking function to provide full-text search support.

The litestore executable file is entirely self-contained and is used to start/stop the LiteStore server, as well as perform maintenance and bulk operations via dedicated CLI commands.

System Decomposition

The following diagram illustrates the main LiteStore components.

LiteStore Architecture

At the lowest level, the SQLite Library is used to manage data access and storage. The functionality provided by the SQLite Library is used by the main Core module, which exposes the main high-level procedures to manage LiteStore-specific artifacts (documents and tags).

The Core module is then used by the two main interfaces exposed to users:

Database Schema

The database schema of LiteStore data store file is very simple, as shown in the following diagram:

LiteStore Database

info Table

The info table currently contains just two INT columns used to keep track of:

documents Table

The documents table is the most important table of the data store, as it contains all the documents stored in it. The following information is stored for each document:

system_documents Table

The system_documents table has a structure similar to the documents table, but it is used for well-known system documents that are used to provide additional functionalities, such as authorization or custom resources.

Unlike ordinary documents, system documents: * cannot be accessed via the HTTP or Nim API, they can only be imported, exported, or deleted with the corresponding commands. * are not searchable. * cannot be tagged.

The following information is stored for each system document:

tags Table

The tags table is used to store the associations between tags and documents. Tags can be added by users or add automatically by the system when a document is imported into the data store.

searchdata Table

This table is used as full-text index for searchable documents.

Getting Started

Downloading Pre-built Binaries

The easiest way to get LiteStore is by downloading one of the prebuilt binaries from the [Github Release Page][release]:

Running a Docker Image as a Container

Official Docker images are available on Docker Hub.

Just pull the latest version:

docker pull h3rald/litestore:v1.13.0

then start a container to run the image on port 9500:

docker run -p 9500:9500 h3rald/litestore:v1.13.0 -a:0.0.0.0

Tip

The Dockerfile used to create tbe image is available in root of tbe LiteStore Github repository.

Installing using Nimble

If you already have Nim installed on your computer, you can simply run

nimble install litestore

Running the Administration App

A simple but functional Administration App is available to manage LiteStore, create documents interactively, view and search content, etc.

To get the app up and running (assuming that you have the litestore executable in your path):

  1. Extract the default data.db file included in the LiteStore release package. This file is a LiteStore data store file containing the sample app.
  2. Go to the local directory in which you downloaded the data.db file.
  3. Run litestore -s:data.db
  4. Go to localhost:9500/docs/admin/index.html.

Usage

Command Line Syntax

litestore [ command ] [ option1, option2, … ]

Commands

Options

Examples

Starting the HTTP Server

Importing a directory

Import a directory called admin:

litestore import -d:admin

Importing system documents from a directory

Import all documents stored in a directory called system as system documents:

litestore import -d:system --system

Importing a directory and tagging files

Import all documents stored in a directory called media (including subdirectories):

+ media
  + cars
  | + _tags
  | + Lamborghini.jpg
  | + VW.jpg
  | ` BMW.jpg
  + planes
  | + _tags
  | + 767.jpg
  | + F-16.jpg
  | ` B-1.jpg
  ` trains
    + TGV.jpg
    ` Eurostar.jpg

litestore import -d:media --import-tags

Every _tags file contains a list of tags, one per line, which are applied to all imported documents from the same directory. In the example above all cars and planes images will be tagged on import. The trains images however will not as there is no _tags file in the trains directory.

The individual _tags files are also imported. When the --import-tags option is not set the _tags files are ignored and not imported.

Excluding files from full text search

Import all documents stored in a directory called media (including subdirectories):

+ media
  + _css
  | + style1.css
  | ` style2.css
  + cars
  | + _index.html
  | + Lamborghini.md
  | + VW.md
  | ` BMW.md
  + planes
  | + _index.html
  | + 767.md
  | + F-16.md
  | ` B-1.md
  ` trains
    + _index.html
    * _script.js
    + TGV.md
    ` Eurostar.md

litestore import -d:media --not-searchable

All documents are imported but the files whose name starts with underscore and files inside a folder which name starts with underscore are excluded from full text search. The idea is that these files have special meaning for the application: * they should be accessible via regular URLs (unlike system files) * but they content should not be searchable.

Exporting a directory

Export all documents tagged with $dir:admin to a local directory called admin:

litestore export -d:admin

Deleting documents within a directory

Delete all documents tagged with $dir:admin:

litestore delete -d:admin

Performing maintenance operations

Executing operations

Authorization

LiteStore can be configured to automatically validate JWT tokens and authorize authenticated users on specific resources (and specific resource verbs even) based on their OAuth2 scopes specified in the token itself.

auth.json vs. config.json

As of version 1.8.0, it is recommended to use the LiteStore configuration file to configure authorization. This specialized auth.json configuration file format will however be maintained for compatibility reasons.

To configure authorization, create an auth.json file like the following:

{
  "access": {
    "/info": {
      "GET": ["admin:server"]
    }, 
    "/docs/*": {
      "POST": ["admin:server"],
      "PATCH": ["admin:server"],
      "PUT": ["admin:server"],
      "DELETE": ["admin:server"]
    },
    "/docs/wiki/*": {
      "POST": ["admin:wiki"],
      "PUT": ["admin:wiki"],
      "PATCH": ["admin:wiki"],
      "DELETE": ["admin:wiki"]
    }
  },
  "signature": "\n-----BEGIN CERTIFICATE-----\n<certificate text goes here>\n-----END CERTIFICATE-----\n"
}

The access property is a dictionary of endpoints to which only users that have one of the specified scopes can access.

For example, in this case only users with the admin:server scope will be able to access /info, and any /docs/ document.

However, users with the admin:wiki scope will be able to access documents located under the /docs/wiki/ folder.

Finally, specify the public signature to be used to validate JWT tokens using the signature property. Typically, its value should be set to the first value of the x.509 certificate chain specified in the JSON Web Key Set of your API.

signature vs. jwks_uri

As of version 1.13.0, it is recommended to use the jwks_uri property in a LiteStore configuration file instead of the signature property.

To use this configuration at runtime, specify it through the --auth option, like this:

litestore -\-auth:auth.json

Once enabled, LiteStore will return HTTP 401 error codes if an invalid token or no token is included in the HTTP Authorization header of the request accessing the resource or HTTP 403 error codes in case an authenticated user does not have a valid scope to access a specified resource.

Configuration File

As of version 1.8.0, you can specify a configuration file containing settings, middleware and authorization configuration using the --config or -c command line option:

litestore -c:config.json

A typical configuration file looks like this:

{
  "settings": {
    "log": "debug",
    "port": 9200
  },
  "stores": {
    "logs": {
      "file": "logs.db",
      "config": {
        "resources": {
          "/docs/*": {
            "GET": {
              "auth": ["admin:server"]
            },
            "POST": {
              "allowed": false
            },
            "PUT": {
              "allowed": false
            },
            "PATCH": {
              "allowed": false
            },
            "DELETE": {
              "allowed": false
            }
          }
        }
      }
    }
  },
  "resources": {
    "/docs/vehicles/*": {
      "GET": {
        "middleware": ["validate", "log"]
      },
      "HEAD": {
        "middleware": ["validate", "log"]
      },
      "POST": {
        "allowed": false
      },
      "PATCH": {
        "auth": ["admin:vehicles"],
        "middleware": ["validate", "log"]
      },
      "PUT": {
        "auth": ["admin:vehicles"],
        "middleware": ["validate", "log"]
      },
      "DELETE": {
        "auth": ["admin:vehicles"],
        "middleware": ["validate", "log"]
      }
    }
  }
}

At present, it contains a settings, a resources, and a signature section.

settings

This section contains some of the most common command-line options, i.e.:

If a configuration file is specified and some of these settings are configured, they will be recognized as if they were specified via command line. However, if you also specify the same settings via command line, the command line settings will take precedence over the settings defined in the configuration file.

stores

This section is used to defined additional stores to be managed by LiteStore by specifying the SQLite file to open and optionally the store configuration.

In this case, the logs store is configured as an additional store.

resources

This section can contain any number of resource paths, like /docs/, /info/, /docs/vehicles/AA456CC or /docs/logs/*. If a wildcard is specified after a resource or folder path, the rules defined within that section will match any document within the specified path. So for examople /docs/vehicles/* will match both /docs/vehicles/AB547QV and /docs/vehicles/BB326CZ, but not /docs/vehicles/.

Within each resource path, you can specify different HTTP methods (all uppercase) and within each method any of the following properties:

signature

This section must be set to a valid certificate used validate JWT tokens. Note that the certificate must follow a specific format and start with the appropriate begin/end blocks.

jwks_uri

As of version 1.13.0, this property can be set to a URI pointing to a valid JSON Web Key Sets file. If this property is specified, it will be used instead of signature to perform signature verification of JWKS tokens.

How JWKS data is managed

If this property is set, LiteStore will attempt to download the specified JWKS file on startup. This file will be catched to a store-name_jwks.json file (e.g. data_jwks.json) and its contents stored in memory.

Middleware

As of version 1.8.0, you can define your own custom middleware using JavaScript executed on the server side.

LiteStore embeds the duktape lightweight JavaScript engine and therefore you can use all functionalities exposed by duktape in your code, plus access some LiteStore-specific properties and method through some special global object.

Although writing extremely complex logic in a JavaScript handler may not be appropriate, it can be useful for certain use cases, such as: * validating data before it is saved * manipulate data before it is saved * aggregating different types of data not accessible via a single query * perform additional operation when accessing data, such as logging who requested it

How middleware works

Potentially, each resource could have one or more middleware functions associated to it. Association is done through the LiteStore configuration file in a similar way as authentication is configured:

{
  "settings": {
    "middleware": "test/middleware"
  },
  "resources": {
    "/docs/vehicles/*": {
      "GET": {
        "middleware": ["validate", "log"]
      },
      "PUT": {
        "auth": ["admin:vehicles"],
        "middleware": ["validate", "log"]
      }
    }
  }
}

This simple configuration file shows how to configure middleware to be executed when a resources it requested via GET or PUT. In both cases, first the validate middleware function is executed, and then the log. These functions must reside in separate files named validate.js and log.js respectively, and placed into a folder (test/middleware in this case) referenced via the middleware setting (which is also exposed as a command line option, with -w as shorthand option).

Middleware functions are executed sequentially until one of them explicitly stops the execution chain or the execution completes (by requesting the original resource).

Considering the previous configuration example, if a PUT request is made to an item under /docs/vehicles:

  1. The validate middleware function is executed.
  2. The log middleware function is executed.
  3. The request is processed as normal.

Note that, for example, the validate middleware function may cause the execution to stop before it reaches the log middleware, thereby effectively implementing server-side validation.

Creating a JavaScript Middleware Function

Let’s say you want to keep records of Italian vehicles identified by their number plate, which is in the following format:

\[two-uppercase-letters\][three-digits][two-uppercase-letters]

For example: AB467DX (OK, in reality there’s a space between each set of digits/letters, but spaces in identifiers are ugly, so let’s remove them!)

Let’s also say that Italian vehicle data will be managed within a folder called vehicles, therefore vehicles will be accessible at URLs similar to the following:

To make sure that valid IDs are used, we can create a file called vehicles.js and write the following code:

(function() {
  var id = $req.path.replace(/^\/docs\//, "");
  var valid = /[A-Z]{2}[0-9]{3}[A-Z]{2}/;
  if (!id.match(valid)) {
    $res.content = {
      error: "Invalid number plate"
    };
    $res.code = 400;
    return true;
  }
  $ctx.existing = !!($store.get("docs", id).code == 200);
})();

Note that middleware must be coded in the form of an IIFE. In this case, the function: * retrieves the ID of the vehicle. * checks if it’s valid * if it’s invalid, prepares a 400 - Bad Request response and stops the execution of additional middleware (and ultimately the request itself) by returning true. * otherwise, it checks whether the vehicle already exists and stores this information on a context, so that it will be accessible to other middleware functions down the execution chain.

Passing data to another middleware

Although you can technically add additional properties to the $req and $res objects, you should use $ctx instead. $ctx is a global objects.

In the validate middleware described in the previous section, the $ctx.existing property was set. This property can then be accessed and/or modified in additional middleware down the execution chain.

Consider for example the following, very basic log middleware function:

(function(){
    var doc = {
        user: $req.jwt.claims && $req.jwt.claims.sub || null,
        agent: $req.headers['user-agent'],
        language: $req.headers['accept-language'] && $req.headers['accept-language'].replace(/,.+$/, ''),
        path: $req.path,
        existing: !!$ctx.existing,
        method: $req.method,
        timestamp: Date.now()
    }
    $store.post('docs', 'logs', JSON.stringify(doc), 'application/json');
}())

This middleware function simply logs the current request to another folder within the data store, and gathers some stats and whether the request was performed on an existing object or not. In this case, $ctx.existing should be set by another middleware up the chain (validate).

Global JavaScript Objects

When creating JavaScript handlers for middleware, you can use some special $-prefixed global objects to access the HTTP request to the resource, the HTTP response, and also access other LiteStore resources.

$ctx

An empty object that can be used to temporarily store data to pass across different middleware handlers.

$req

The current HTTP request sent to access the current resource.

method: string
The HTTP method used by the request, all uppercase (GET, POST, DELETE, PUT, PATCH, OPTIOONS, or HEAD).
jwt: object
An object containing a parsed JWT token, if present. It exposes two properties:
  • headers, an object typically containing the alg (algorithm) and typ (type) keys.
  • claims, an object containing the claims included in the token (see the IANA JSON Web Token Claims Registry for a list of possible claims).
headers: object
An object containing the request headers, as keys and values.
protocol: string
The request protocol and version.
hostname: string
The hostname target of the request.
port: number
The port used for the request.
path: string
The path to the resource requested.
query: string
The contents of the request query string.
content: string
When applicable, the content that was sent as body of the request.

$res

The HTTP response to return to the client.

code: number
The HTTP return code, by default set to `200`.
content: string
The response content, by default set to `""`.
headers: object
The response headers, by default set to: 

{
  "Access-Control-Allow-Origin": "*",
  "Access-Control-Allow-Headers": "Authorization, Content-Type",
  "Server": "LiteStore/<version>",
  "Content-Type": "application/json",
  "Content-Length": "<Automatically set to the length of the content property.>"
}

$store

Simple synchronous API to access LiteStore resources in a RESTful way, mimicking HTTP methods.

All methods return a response object containing two String properties, code and content.

function get(resource: string, id: string, parameters: string): object
Retrieves the specified resource(s).

Examples:

  • $store.get('docs', 'test-folder/test.json')
  • $store.get('docs', '', 'search=test&limit=20&offset=0')

function post(resource: string, folder: string, body: string, contentType: string): object
Creates a new resource.

Examples:

  • $store.post('docs', 'test-folder', 'test!', 'text/plain')
  • $store.post('docs', '', '{"a": 1}', ?application/json')

function put(resource: string, id: string, body: string, contentType: string): object
Creates or updates a specific resource.

Examples:

  • $store.put('docs', 'test-folder/test1.txt', 'Another Test.', 'text/plain')
  • $store.put('docs', 'test.json', '{"a": 2}', 'application/json')

function patch(resource: string, id: string, body: string): object
Patches one or more fields of an existing resource.

Examples:

  • $store.patch('docs', 'test-folder/test1.txt', '{"op":"add", "path":"/tags/3", "value":"test1"}')

function delete(resource: string, id: string): object
Deletes a specific resource.

Examples:

  • $store.delete('docs', 'test-folder/test1.txt')
  • $store.delete('docs', 'test.json')

function head(resource: string, id: string): object
Retrieves the metadata of one or more resources, without retrieving their contents.

Examples:

  • $store.head('docs', 'test-folder/test1.txt')
  • $store.head('docs')

$http

Simple synchronous API to perform HTTP requests.

All methods return a response object containing the following properties: * code (string) * content (string) * headers (object)

function get(url: string, headers: object): object
Executes a GET request.

Example:

  • $http.get('https://reqres.in/api/users', {})

function post(url: string, headers: object body: string): object
Executes a POST request.

Example:

  • $http.post(https://reqres.in/api/users', {'Content-Type': 'application/json'}, '{"name": "Test", "job": "Tester"}')

function put(url: string, headers: object body: string): object
Executes a PUT request.

Example:

  • $http.put(https://reqres.in/api/users/2', {'Content-Type': 'application/json'}, '{"name": "Test", "job": "Tester"}')

function patch(url: string, headers: object body: string): object
Executes a PATCH request.

Example:

  • $http.patch(https://reqres.in/api/users/2', {'Content-Type': 'application/json'}, '{"name": "Test", "job": "Tester"}')

function delete(url: string, headers: object): object
Executes a DELETE request.

Example:

  • $http.delete('https://reqres.in/api/users/2', {})

function head(url: string, headers: object): object
Executes a HEAD request.

Example:

  • $http.head('https://reqres.in/api/users', {})

System Documents

As of version 1.8.0, it is possible to import, export, or delete system documents besides ordinary documents. Such documents are different from ordinary documents, because:

At present, only the following system documents are recognized by LiteStore:

Importing, exporting and deleting System Documents

You can import, export, and delete system documents with the respective commands, but you must specify the --system command line flag.

For example, suppose you have a sysdocs folder containing the following file hierarchy:

To import all the documents stored within the sysdocs folder, you must run the following command:

litestore -d:sysdocs --system import

Similarly, the export and delete commands can be used to export and delete system documents respectively, always specifying the --system flag.

How LiteStore uses System Documents

While at development time you may want to be able to edit your system documents and therefore keep them outside your data store as ordinary text files (and load them using the --auth, --config and -\s-middleware options), in production you may want to ship them within the data store along with your application data.

At run time, LiteStore will attempt to retrieve settings/middleware/authorization configuration using the following order of precedence (first listed have higher precedence):

  1. Command line options
  2. Configuration files specified via command line options
  3. Configuration files loaded as system documents

Multiple Data Stores

As of version 1.9.0, it is possible to configure LiteStore to manage several different SQLite database files, or stores. Essentially, besides the master store it is possible to create, delete or access additional stores at run time though the new /stores resource.

Although folders already provide some partitioning for documents, in certain situations you may want to physically separate your data into multiple files, for example when:

Although all stores can be accessed by the same process using the /stores resource (which can essentially forward requests to be executed on a specific file), each store can have its own configuration file stored as a system document, its own authentication and its own middleware.

Configuring additional stores

If you know the details of each store at development/configuration time, you can configure them in the stores section of the LiteStore configuration file, like this:

{
  "settings": {
    "log": "debug",
    "port": 9200
  },
  "stores": {
    "test1": {
      "file": "test1.db",
      "config": null
    },
    "test2": {
      "file": "test2.db",
      "config": null
    },
    "test3": {
      "file": "test3.db",
      "config": null
    }
  },
  "resources": {
    "/docs/vehicles/*": {
      "GET": {
        "middleware": ["validate", "log"]
      },
      "HEAD": {
        "middleware": ["validate", "log"]
      },
      "POST": {
        "allowed": false
      },
      "PATCH": {
        "auth": ["admin:vehicles"],
        "middleware": ["validate", "log"]
      },
      "PUT": {
        "auth": ["admin:vehicles"],
        "middleware": ["validate", "log"]
      },
      "DELETE": {
        "auth": ["admin:vehicles"],
        "middleware": ["validate", "log"]
      }
    },
    "/docs/logs/*": {
      "GET": {
        "auth": ["admin:server"]
      },
      "POST": {
        "allowed": false
      },
      "PUT": {
        "allowed": false
      },
      "PATCH": {
        "allowed": false
      },
      "DELETE": {
        "allowed": false
      }
    }
  }
}

When LiteStore is executed, the three additional stores will be created and initialized, their configuration (if any) will be saved as a system document and they will be immediately accessible.

Alternatively, you can add or remove stores dynamically at run time by executing POST and DELETE requests to the /stores resource, optionally specifying configuration settings as the request body.

Administration App

A simple, slightly dated, but fully-functional administration app can be used to manage LiteStore data stores. This app can simply be imported into a data store file and then run via LiteStore.

Obtaining and Running the Administration App

There are two ways to get the Administration App:

Using a Pre-populated Data Store File

If you are using the data.db data store file distributed with the pre-built LiteStore binaries, you don’t have to do anything: the Administration App is already bundled in the datastore file. You simply have to run LiteStore and access the app at the following address: http://localhost:9500/docs/admin/index.html.

Importing the admin directory

If you are using your own data store file, you can still import the Administration App in it by downloading the LiteStore source files or cloning the LiteStore repository from Github and running LiteStore with the following arguments and options from the top-level directory of the LiteStore repository:

litestore -d:admin import

Once the import is completed, you can run litestore and access the app at the following address: http://localhost:9500/docs/admin/index.html.

Main Functionalities

The LiteStore Administration App is a single-page application built using the Mithril Javascript framework and other open source software like Bootstrap and the ACE Editor.

It can be used to easily access and explore any LiteStore data store (provided that it has been loaded in it) and access most of LiteStore functionalities via its HTTP API.

View Data Store Information

When first loaded, the app loads the Info page by default. This pages contains useful information on the currently-loaded LiteStore data store, some stats on documents and tags, and links to access documents by tag.

Info Page

Read LiteStore Documentation

The Guide section of the Administration App contains the official LiteStore User Guide, conveniently split into separate pages by section.

Note

If the data store is loaded in read/write mode (default), you’ll see some Edit buttons on the Guide pages. Clicking them will open the corresponding Markdown source document for editing – Such links have been very useful to update the docs in-place!

Guide

Display Documents by Tag

By clicking any of the tag links on the Info page or by accessing the Tags menu in the navigation bar you can browse all documents tagged with a specific tag. Currently, it is only possible to specify only one single tag at a time although the API allows several tags to be specified at once.

Display documents by tag

Search Documents

You can search for documents using the search box in the navigation bar. The Administration App lets you search through all the textual/searchable documents loaded in the currently open data store.

Search

View, Preview, Delete, Create and Edit Documents

You can view, edit and delete any document loaded in the data store using the bundled ACE Editor component. Additionally:

Document

HTTP API Reference

LiteStore provides a RESTful HTTP API. At present, the only two resources exposed are the following:

Accessing LiteStore Resources

To access a LiteStore resource, use URLs composed in the following way:

http:<hostname>:<port>/v<version>/<resource>/<id>

Example: localhost:9500/v2/docs/admin/index.html

Remarks

  • If the version is omitted, the latest version of the LiteStore API will be used by default. The previous URL can be therefore written as http://localhost:9500/docs/admin/index.html.
  • In a somewhat non-standard way, IDs of LiteStore documents can contain slashes. The main reason behind this is that this makes it possible to implement a virtual file system with LiteStore easily, and serve up web pages as you would from an ordinary filesystem.

info (LiteStore Information)

This resource can be queried to obtain basic information and statistics on the LiteStore server.

OPTIONS info

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/info
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

GET info

Returns the following server statistics:

Example
$ curl -i http://127.0.0.1:9500/info
HTTP/1.1 200 OK
Content-Length: 965
Content-Type: application/json
Access-Control-Allow-Headers: Authorization, Content-Type
Access-Control-Allow-Origin: http://127.0.0.1:9500
Server: LiteStore/1.9.0

{
  "version": "LiteStore v1.9.0",
  "datastore_version": 2,
  "api_version": 7,
  "size": "6.98 MB",
  "read_only": false,
  "log_level": "warn",
  "directory": "admin",
  "mount": true,
  "additional_stores": [],
  "auth": false,
  "total_documents": 68,
  "total_tags": 18,
  "tags": [
    {
      "$dir:admin": 68
    },
    {
      "$format:binary": 8
    },
    {
      "$format:text": 60
    },
    {
      "$subtype:css": 3
    },
    {
      "$subtype:html": 2
    }
  ]
}

dir (LiteStore Directory)

This resource can be queried to retrieve files within the served directory (specied via -d).

OPTIONS dir

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/dir
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.0

GET dir/:id

Returns the content of a file within the served directory (if it exists).

Example
$ curl -i http://127.0.0.1:9500/dir/test.txt
HTTP/1.1 200 OK
Content-Length: 25
Content-Type: text/plain
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

This is a test text file.

docs (LiteStore Documents)

A document is the main resource type managed by LiteStore. Any LiteStore document can be represented as a JSON object exposing the following properties:

JSON Documents

Documents with content type “application/json” are special: their data property is not set to a string like for all other textual and binary documents, but a real, non-escaped JSON object. This little quirk makes JSON documents different from other documents, but also makes things so much easier when you just want to use LiteStore as a simple JSON document store.

Example Document

{
    "id": "test_document",
    "data": "This is a test document",
    "created": "2015-02-07T10:36:09Z",
    "modified": null,
    "tags": ["$type:text", "$subtype:plain", "$format:text", "another_tag"]
}

OPTIONS docs

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS 'http://127.0.0.1:9500/docs'
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: HEAD,GET,OPTIONS,POST
Allow: HEAD,GET,OPTIONS,POST
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.1.0

OPTIONS docs/:id

Returns the allowed HTTP verbs for this resource.

Example
curl -i -X OPTIONS 'http://127.0.0.1:9500/docs/test' 
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: HEAD,GET,OPTIONS
Allow: HEAD,GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.1.0

OPTIONS docs/:folder/

API v2 Required

This method has been introduced in version 2 of the LiteStore API.

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS 'http://127.0.0.1:9500/docs/test/'
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: HEAD,GET,OPTIONS,POST
Allow: HEAD,GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.1.0

POST docs

Creates a new document with a randomly-generated ID.

Example
$ curl -i -X POST -d 'A document with a randomly-generated ID.' 'http://127.0.0.1:9500/docs' -\-header "Content-Type:text/plain"
HTTP/1.1 201 Created
Content-Length: 197
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

{"id": "555f93e82190e77500000000", "data": "A document with a randomly-generated ID.", "created": "2015-05-22T08:39:04Z", "modified": null, "tags": ["$type:text", "$subtype:plain", "$format:text"]}

POST docs/:folder/

Creates a new document with a randomly-generated ID under the specified folder path.

Example
$ curl -i -X POST -d 'A document with a randomly-generated ID.' 'http://127.0.0.1:9500/docs/test/' --header "Content-Type:text/plain"
HTTP/1.1 201 Created
Content-Length: 197
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

{"id": "test/555f93e82230f77500000000", "data": "A document with a randomly-generated ID.", "created": "2015-05-22T08:39:04Z", "modified": null, "tags": ["$type:text", "$subtype:plain", "$format:text"]}

HEAD docs

Retrieves all headers related to the docs resource and no content (this is probably not that useful, but at least it should make REST purists happy).

Example
$ curl -i -X HEAD 'http://127.0.0.1:9500/docs'
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

HEAD docs/:folder/

Retrieves all headers related to the a folder and no content. Useful to check whether a folder exists or not.

Example
$ curl -i -X HEAD 'http://localhost:9500/docs/admin/images/'
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.1.0

HEAD docs/:id

Retrieves all headers related to the a document and no content. Useful to check whether a document exists or not.

Example
$ curl -i -X HEAD 'http://127.0.0.1:9500/docs/test'
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

GET docs

Retrieves a list of documents in JSON format. Several query string options are supported to query documents.

content option

If set to false, do not retrieve document data.

Example: http://127.0.0.1:9500/docs/?contents=false

limit and offset options

Provide a way to implement pagination:

Example: http://127.0.0.1:9500/docs/?limit=10&offset=20

search option

Search for the specified string.

Example: http://127.0.0.1:9500/docs/?search=Something

Tip

If search is specified, each result will contain a highlight property with a highlighted search snippet, and a rank property identified the rank of the result within the search. Results will also be automatically ordered by descending rank.

like option

If this option is specified, retrieves a single document which id matches the specified pattern. The value give to like option is not used so the search pattern is specified as the regular document id as a part of the path. The regular SQL wildcards % and _ can be used. To escape them use \ encoded in the URL as %5C.

Example: http://127.0.0.1:9500/docs/%/api%5C_%.md?like=1&raw=true&contents=false&search=API%20v7%Required

If finds the first markdown file which name starts with api_ and which contains string API v7 Required.

Tip

Only the first matching document is returned even if there was more than one which id matched the pattern. To get the subsequent documents use offset option and increase its value from 1 up.

tags option

Retrieve only documents with matching tag(s).

Example: http://127.0.0.1:9500/docs/?tags=tag1,tag2

filter option

API v3 Required

This query string option has been introduced in version 3 of the LiteStore API.

Retrieve only JSON documents matching the specified filter expression.

Filter expressions can be composed by one or more clauses joined together through or or and operators. Each clause must be composed exactly by:

API v5 Required

Support for the like operator has been added in version 5 of the LiteStore API.

Limitations

  • Parenthesis are not supported.
  • Up to 10 or clauses and 10 and clauses are supported.
  • Paths can only contain keys that contain only numbers, letters and underscores.

Examples:

select option

API v3 Required

This query string option has been introduced in version 3 of the LiteStore API.

Retrieve JSON documents containing only the specified fields. Fields must be specified by comma-separated path/alias expression.

Example: http://127.0.0.1:9500/docs/?select=$.name.first%20as%20FirstName,$.age%20as%20Age

created-after option

Retrieve only documents created after a the specified timestamp.

Example: http://127.0.0.1:9500/?created-after=1537695677

API v4 Required

This query string option has been introduced in version 4 of the LiteStore API.

created-before option

Retrieve only documents created before a the specified timestamp.

Example: http://127.0.0.1:9500/?created-before=1537695677

API v4 Required

This query string option has been introduced in version 4 of the LiteStore API.

modified-after option

Retrieve only documents modified after a the specified timestamp.

Example: http://127.0.0.1:9500/?modified-after=1537695677

API v4 Required

This query string option has been introduced in version 4 of the LiteStore API.

modified-before option

Retrieve only documents modified before a the specified timestamp.

Example: http://127.0.0.1:9500/?modified-before=1537695677

API v4 Required

This query string option has been introduced in version 4 of the LiteStore API.

sort option

Sort by created, modified, id or a JSON path to a field (prepend - for DESC and + for ASC).

API v3 Required for JSON path support

Support for JSON paths requires version 3 of the LiteStore API.

Examples:

Query String Options
Example
$ curl -i 'http://localhost:9500/docs?search=Use%20Cases&limit=10
&offset=0&tags=$subtype:x-markdown'
HTTP/1.1 200 OK
Content-Length: 1960
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

{
  "search": "Use Cases",
  "tags": [
    "$subtype:x-markdown"
  ],
  "limit": 10,
  "total": 3,
  "execution_time": 0.01843700000000001,
  "results": [
    {
      "id": "admin/md/use-cases.md",
      "created": "2015-09-19T01:37:59Z",
      "modified": null,
      "highlight": "### <strong>Use</strong> <strong>Cases</strong>\u000A\u000AWhile LiteStore may not be the best choice for large data-intensive applications, it definitely shines when <strong>used</strong> for rapid prototyping and as a backend for small/lightweight<strong>&hellip;</strong>",
      "rank": "99.5820018475243",
      "tags": [
        "$type:text",
        "$subtype:x-markdown",
        "$format:text",
        "$dir:admin"
      ]
    },
    {
      "id": "admin/md/architecture.md",
      "created": "2015-09-19T01:37:59Z",
      "modified": null,
      "highlight": "<strong>&hellip;</strong>public unique document identifier, <strong>used</strong> to access the document via the HTTP API.\u000A* **data** &ndash; The contents of the document (or their base64-encoded representation in <strong>case</strong> of binary documents<strong>&hellip;</strong>",
      "rank": "39.492608737092",
      "tags": [
        "$type:text",
        "$subtype:x-markdown",
        "$format:text",
        "$dir:admin"
      ]
    },
    {
      "id": "admin/md/overview.md",
      "created": "2015-09-19T01:37:59Z",
      "modified": null,
      "highlight": "<strong>&hellip;</strong>contained, LiteStore comes with many <strong>useful</strong> features that are essential for many <strong>use</strong> <strong>cases</strong>.\u000A\u000A#### [](class:fa-file-text-o) Multi-format Documents\u000A\u000ALiteStore can be <strong>used</strong> to store documents in<strong>&hellip;</strong>",
      "rank": "39.4926086158248",
      "tags": [
        "$type:text",
        "$subtype:x-markdown",
        "$format:text",
        "$dir:admin"
      ]
    }
  ]
}

GET docs/:folder/

API v2 Required

This method has been introduced in version 2 of the LiteStore API.

Retrieves a list of documents in JSON format starting with the specified folder path (it must end with ‘/’).

Supported query options

The same query options of the docs resource are supported.

Example
$ curl -i -X GET 'http://localhost:9500/docs/admin/images/?contents=false'
HTTP/1.1 200 OK
Content-Length: 2066
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.1.0

{
  "folder": "admin/images/",
  "total": 2,
  "execution_time": 0.014684,
  "results": [
    {
      "id": "admin/images/app_document.png",
      "created": "2016-02-06T01:11:30Z",
      "modified": null,
      "tags": [
        "$type:image",
        "$subtype:png",
        "$format:binary",
        "$dir:admin"
      ]
    },
    {
      "id": "admin/images/app_guide.png",
      "created": "2016-02-06T01:11:30Z",
      "modified": null,
      "tags": [
        "$type:image",
        "$subtype:png",
        "$format:binary",
        "$dir:admin"
      ]
    }
  ]
}

GET docs/:id

Retrieves the specified document. By default the response is returned in the document’s content type; however, it is possible to retrieve the raw document (including metadata) in JSON format by setting the raw query string option to true.

Example: original content type
$ curl -i 'http://127.0.0.1:9500/docs/test'
HTTP/1.1 200 OK
Content-Length: 24
Content-Type: text/plain
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

This is a test document.
Example: raw format
$ curl -i 'http://127.0.0.1:9500/docs/test?raw=true'
HTTP/1.1 200 OK
Content-Length: 191
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

{"id": "test", "data": "This is a test document.", "created": "2015-09-19T08:07:43Z", "modified": null, "tags": ["$type:text", "$subtype:plain", "$format:text"]}

PUT docs/:id

Updates an existing document or creates a new document with the specified ID.

Example
$ curl -i -X PUT -d 'This is a test document.' 'http://127.0.0.1:9500/docs/test' --header "Content-Type:text/plain"
HTTP/1.1 201 Created
Content-Length: 161
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

{"id": "test", "data": "This is a test document.", "created": "2015-05-22T08:40:00Z", "modified": null, "tags": ["$type:text", "$subtype:plain", "$format:text"]}

PATCH docs/:id

Adds, removes, replaces or tests the specified document for tags or data. Operations must be specified using the JSONPatch format.

Tip

If you plan on patching tags, always retrieve document tags first before applying a patch, to know the order tags have been added to the document.

Limitations

  • Only add, remove, replace and test operations are supported.
  • It is only possible to patch data and/or tags of a document.
  • It is only possible to patch the data of JSON documents.
Example: Patching Tags
$ curl -i -X PATCH 'http://localhost:9500/docs/test.json' --header "Content-Type:application/json" -d '[{"op":"add", "path":"/tags/3", "value":"test1"},{"op":"add", "path":"/tags/4", "value":"test2"},{"op":"add", "path":"/tags/5", "value":"test3"}]'
HTTP/1.1 200 OK
Content-Length: 187
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

{"id": "test.json", "data": {"test": true}, "created": "2015-09-20T09:06:25Z", "modified": null, "tags": ["$type:application", "$subtype:json", "$format:text", "test1", "test2", "test3"]}
Example: Patching Data

Given the following document:

{
  "id": "test",
  "data": {
    "name": {
      "first": "Tom",
      "last": "Paris"
    },
    "rank": "lieutenant"
  },
  "created": "2018-02-25T14:33:14Z",
  "modified": null,
  "tags": [
    "$type:application",
    "$subtype:json",
    "$format:text"
  ]
}

The following PATCH operation can be applied to change its data.

$ curl -i -X PATCH 'http://localhost:9500/docs/test' --header "Content-Type:application/json" -d '[{"op": "replace", "path": "/data/name", "value": "Seven of Nine"}, {"op": "remove", "path": "/data/rank"}]'
HTTP/1.1 200 OK
server: LiteStore/1.3.0
access-control-allow-origin: *
content-type: application/json
access-control-allow-headers: Content-Type
Content-Length: 172

{"id":"test","data":{"name":"Seven of Nine"},"created":"2018-02-25T14:33:14Z","modified":"2018-02-25T14:35:52Z","tags":["$type:application","$subtype:json","$format:text"]}

DELETE docs/:id

Deletes the specified document.

Example
$ curl -i -X DELETE 'http://127.0.0.1:9500/docs/test'
HTTP/1.1 204 No Content
Content-Length: 0
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.0.3

assets (LiteStore Assets)

API v8 Required

This resource has been introduced in version 5 of the LiteStore API.

Assets represents another way to query LiteStore Documents. All documents can also be retrieved via /assets/ instead of docs, but when doing so: * a non-raw version of the document will always be returned * when querying a folder without specifying a document ID, LiteStore will attempt to retrieve an index.html or index.htm file within that folder

This is particularly useful when your documents have been imported from the filesystem and you need to effectively serve files.

OPTIONS assets

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS 'http://127.0.0.1:9500/assets'
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

OPTIONS assets/:id

Returns the allowed HTTP verbs for this resource.

Example
curl -i -X OPTIONS 'http://127.0.0.1:9500/assets/test.json' 
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

OPTIONS assets/:folder/

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS 'http://127.0.0.1:9500/assets/test/'
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

GET assets

Retrieves the file assets/index.html or assets/index.htm if not found.

GET assets/:folder/

Retrieves the file assets/:folder/index.html or assets/:folder/index.htm if not found.

GET assets/:id

Retrieves the specified document. The response is returned in the document’s content type (even for JSON files).

Example
$ curl -i 'http://127.0.0.1:9500/docs/test.txt'
HTTP/1.1 200 OK
Content-Length: 24
Content-Type: text/plain
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

This is a test document.

tags (LiteStore Tags)

This resource can be queried to retrieve the total of documents associated to a tag, or a list of tags matching a string.

API v4 Required

This resource has been introduced in version 4 of the LiteStore API.

OPTIONS tags

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/tags
HTTP/1.1 200 OK
server: LiteStore/1.5.0
access-control-allow-origin: http://localhost:9500
access-control-allow-headers: Content-Type
allow: GET,OPTIONS
access-control-allow-methods: GET,OPTIONS
content-length: 0

OPTIONS tags/:id

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/tags/$type:text
HTTP/1.1 200 OK
server: LiteStore/1.5.0
access-control-allow-origin: http://localhost:9500
access-control-allow-headers: Content-Type
allow: GET,OPTIONS
access-control-allow-methods: GET,OPTIONS
Content-Length: 0

GET tags

Retrieves all tags and the total of their associated documents.

like option

If this option is specified, retrieves all tags matching the specified string.

Wildcards

You can use asterisks (*) as wildcards.

limit and offset options

Provide a way to implement pagination:

Example
$ curl -i http://localhost:9500/tags/?like=%24type:%2A
HTTP/1.1 200 OK
server: LiteStore/1.5.0
access-control-allow-origin: http://localhost:9500
content-type: application/json
vary: Origin
access-control-allow-headers: Content-Type
Content-Length: 290

{
  "like": "$type:*",
  "total": 3,
  "execution_time": 0.0008190000000000003,
  "results": [
    {
      "id": "$type:application",
      "documents": 43
    },
    {
      "id": "$type:image",
      "documents": 10
    },
    {
      "id": "$type:text",
      "documents": 32
    }
  ]
}

GET tags/:id

Retrieves the specified tag and corresponding document total.

Example
$ curl -i http://localhost:9500/tags/%24type%3Atext
HTTP/1.1 200 OK
server: LiteStore/1.5.0
access-control-allow-origin: http://localhost:9500
content-type: application/json
vary: Origin
access-control-allow-headers: Content-Type
Content-Length: 34

{"id":"$type:text","documents":32}

indexes (LiteStore Indexes)

API v5 Required

This resource has been introduced in version 5 of the LiteStore API.

LiteStore Indexes are special indexes used to optimize the performance of queries on JSON documents.

OPTIONS indexes

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/indexes
HTTP/1.1 200 OK
server: LiteStore/1.7.0
access-control-allow-origin: http://localhost:9500
access-control-allow-headers: Content-Type
allow: GET,OPTIONS
access-control-allow-methods: GET,OPTIONS
content-length: 0

OPTIONS indexes/:id

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/indexes/name
HTTP/1.1 200 OK
server: LiteStore/1.7.0
access-control-allow-origin: http://localhost:9500
access-control-allow-headers: Content-Type
allow: GET,OPTIONS,PUT,DELETE
access-control-allow-methods: GET,OPTIONS,PUT,DELETE
Content-Length: 0

GET indexes

Retrieves all indexes and their respective JSON fields.

like option

If this option is specified, retrieves all indexes matching the specified string.

Wildcards

You can use asterisks (*) as wildcards.

limit and offset options

Provide a way to implement pagination:

Example
$ curl -i http://localhost:9500/indexes/?like=%2Aname%2A
HTTP/1.1 200 OK
server: LiteStore/1.7.0
access-control-allow-origin: http://localhost:9500
content-type: application/json
vary: Origin
access-control-allow-headers: Content-Type
Content-Length: 244

{
  "like": "*name*",
  "total": 2,
  "execution_time": 0.0006140000000000001,
  "results": [
    {
      "id": "name",
      "field": "$.name"
    },
    {
      "id": "document.name",
      "field": "$.document.name"
    }
  ]
}

GET indexes/:id

Retrieves the specified index and corresponding JSON field.

Example
$ curl -i http://localhost:9500/indexes/name
HTTP/1.1 200 OK
server: LiteStore/1.7.0
access-control-allow-origin: http://localhost:9500
content-type: application/json
vary: Origin
access-control-allow-headers: Content-Type
Content-Length: 30

{"id":"name","field":"$.name"}

PUT indexes/:id

Creates a new index with the specified ID.

Note that: * Index IDs can only contain letters, numbers, and underscores. * Index fields must be valid paths to JSON fields.

No updates

It is not possible to update an existing index. Delete it and re-create it instead.

Example
$ curl -i -X PUT -d '{"field": "$.name"}' 'http://127.0.0.1:9500/indexes/name' --header "Content-Type:application/json"
HTTP/1.1 201 Created
Content-Length: 31
Content-Type: application/json
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: http://localhost:9500
Server: LiteStore/1.7.0

{"id":"name", "field":"$.name"}

DELETE indexes/:id

Deletes the specified index.

Example
$ curl -i -X DELETE 'http://127.0.0.1:9500/indexes/name'
HTTP/1.1 204 No Content
Content-Length: 0
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: http://localhost:9500
Server: LiteStore/1.7.0

stores (LiteStore Stores)

API v7 Required

This resource has been introduced in version 7 of the LiteStore API.

As of version 1.9.0, it is possible for a single LiteStore process to manage multiple data store files. These additional stores can be accessed but also added or removed at run time using this resource.

OPTIONS stores

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/stores
HTTP/1.1 200 OK
server: LiteStore/1.9.0
access-control-allow-origin: http://localhost:9500
access-control-allow-headers: Content-Type
allow: GET,OPTIONS
access-control-allow-methods: GET,OPTIONS
content-length: 0

OPTIONS stores/:id

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS http://127.0.0.1:9500/stores/test1
HTTP/1.1 200 OK
server: LiteStore/1.9.0
access-control-allow-origin: http://localhost:9500
access-control-allow-headers: Content-Type
allow: GET,OPTIONS,PUT,DELETE
access-control-allow-methods: GET,OPTIONS,PUT,DELETE
Content-Length: 0

GET stores

Retrieves information on all available stores (file name and configuration).

Example
$ curl -i http://localhost:9500/stores
HTTP/1.1 200 OK
content-type: application/json
access-control-allow-origin: http://127.0.0.1:9500
access-control-allow-headers: Authorization, Content-Type
vary: Origin
server: LiteStore/1.9.0
Content-Length: 2346

{
  "total": 4,
  "execution_time": 0.0,
  "results": [
    {
      "id": "test1",
      "file": "test1.db",
      "config": null
    },
    {
      "id": "test2",
      "file": "test2.db",
      "config": null
    },
    {
      "id": "test3",
      "file": "test3.db",
      "config": null
    },
    {
      "id": "master",
      "file": "data.db",
      "config": {
        "settings": {
          "log": "debug",
          "port": 9200
        },
        "stores": {
          "test1": {
            "file": "test1.db",
            "config": null
          },
          "test2": {
            "file": "test2.db",
            "config": null
          },
          "test3": {
            "file": "test3.db",
            "config": null
          }
        },
        "resources": {
          "/docs/vehicles/*": {
            "GET": {
              "middleware": [
                "validate",
                "log"
              ]
            },
            "HEAD": {
              "middleware": [
                "validate",
                "log"
              ]
            },
            "POST": {
              "allowed": false
            },
            "PATCH": {
              "auth": [
                "admin:vehicles"
              ],
              "middleware": [
                "validate",
                "log"
              ]
            },
            "PUT": {
              "auth": [
                "admin:vehicles"
              ],
              "middleware": [
                "validate",
                "log"
              ]
            },
            "DELETE": {
              "auth": [
                "admin:vehicles"
              ],
              "middleware": [
                "validate",
                "log"
              ]
            }
          },
          "/docs/logs/*": {
            "GET": {
              "auth": [
                "admin:server"
              ]
            },
            "POST": {
              "allowed": false
            },
            "PUT": {
              "allowed": false
            },
            "PATCH": {
              "allowed": false
            },
            "DELETE": {
              "allowed": false
            }
          }
        },
        "signature": "\n-----BEGIN CERTIFICATE-----\n<certificate text goes here>\n-----END CERTIFICATE-----\n"
      }
    }
  ]
}

GET stores/:id

Retrieves information on the specified store.

Example
HTTP/1.1 200 OK
content-type: application/json
access-control-allow-origin: http://127.0.0.1:9500
access-control-allow-headers: Authorization, Content-Type
vary: Origin
server: LiteStore/1.9.0
Content-Length: 46

{"id":"test1","file":"test1.db","config":null}

PUT stores/:id

Adds a new stores with the specified ID. If a file called \.db does not exist already, it will be created in the current working directory and initialized as a LiteStore store.

Note that: * Index IDs can only contain letters, numbers, and underscores. * The body must be present and contain the store configuration (or null).

No updates

It is not possible to update an existing store. Remove it and re-add it instead.

Example
curl -i -X PUT -d "null" "http://127.0.0.1:9500/stores/test3" --header "Content-Type:application/json"
HTTP/1.1 201 Created
content-type: application/json
access-control-allow-origin: http://127.0.0.1:9500
access-control-allow-headers: Authorization, Content-Type
vary: Origin
server: LiteStore/1.9.0
Content-Length: 46

{"id":"test3","file":"test3.db","config":null}

DELETE stores/:id

Removes the specified store. Although it will no longer be accessible via LiteStore, the corresponding file will not be deleted from the filesystem.

Example
$ curl -i -X DELETE "http://127.0.0.1:9200/stores/test3"
HTTP/1.1 204 No Content
vary: Origin
access-control-allow-origin: http://127.0.0.1:9200
access-control-allow-headers: Authorization, Content-Type
content-length: 0
server: LiteStore/1.9.0
Content-Length: 0

* stores/:id/*

Forward the request to the specified store. Essentially, the path fragment after the store ID will be forwarded as a standard request to the specified store.

Examples

Retrieve all tags from store vehicles: $ curl -i http://localhost:9500/stores/vehicles/tags/

Delete the document with ID AA457DB from store vehicles:

$ curl -i -X DELETE "http://127.0.0.1:9200/stores/vehicles/docs/AA457DB"

assets (LiteStore Assets)

API v8 Required

This resource has been introduced in version 5 of the LiteStore API.

Assets represents another way to query LiteStore Documents. All documents can also be retrieved via /assets/ instead of docs, but when doing so: * a non-raw version of the document will always be returned * when querying a folder without specifying a document ID, LiteStore will attempt to retrieve an index.html or index.htm file within that folder

This is particularly useful when your documents have been imported from the filesystem and you need to effectively serve files.

OPTIONS assets

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS 'http://127.0.0.1:9500/assets'
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

OPTIONS assets/:id

Returns the allowed HTTP verbs for this resource.

Example
curl -i -X OPTIONS 'http://127.0.0.1:9500/assets/test.json' 
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

OPTIONS assets/:folder/

Returns the allowed HTTP verbs for this resource.

Example
$ curl -i -X OPTIONS 'http://127.0.0.1:9500/assets/test/'
HTTP/1.1 200 OK
Content-Length: 0
Access-Control-Allow-Methods: GET,OPTIONS
Allow: GET,OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

GET assets

Retrieves the file assets/index.html or assets/index.htm if not found.

GET assets/:folder/

Retrieves the file assets/:folder/index.html or assets/:folder/index.htm if not found.

GET assets/:id

Retrieves the specified document. The response is returned in the document’s content type (even for JSON files).

Example
$ curl -i 'http://127.0.0.1:9500/docs/test.txt'
HTTP/1.1 200 OK
Content-Length: 24
Content-Type: text/plain
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Origin: *
Server: LiteStore/1.12.0

This is a test document.

Nim API Reference

Besides exposing an HTTP API, LiteStore also provides a basic Nim API to use it as a library within other Nim projects.

Data Types

The following data types are used by the LiteStore Nim API procs

LSResponse

An HTTP Response.

LSResponse* = tuple[
  code: HttpCode,
  content: string,
  headers: HttpHeaders]

QueryOptions

The set of options SQL-like to be used to compose a LiteStore query.

QueryOptions* = object
  select*: seq[string]
  single*:bool         
  limit*: int           
  offset*: int           
  orderby*: string      
  tags*: string
  folder*: string
  search*: string

Example: Jester Web Framework Integration

The following code example shows how to use the procs provided by LiteStore Nim API to expose LiteStore HTTP routes using the Jester web framework for Nim:

import 
  jester,
  litestore, 
  asyncdispatch, 
  re, 
  strtabs, 
  asyncnet

litestore.setup()

routes:

  # Just a simple, unrelated Jester route
  get "/": 
    resp "Hello, World!"

  # Remapping LiteStore routes on Jester
  get re"^\/litestore\/(docs|info)\/?(.*)":
    let r = get(request.matches[0], 
                request.matches[1], 
                request.params, 
                request.headers)
    resp(r.code, r.content, r.headers["Content-Type"]) 

  post re"^\/litestore\/docs\/?(.*)":
    let r = post("docs", 
                 request.matches[0], 
                 request.body, 
                 request.headers)
    resp(r.code, r.content, r.headers["Content-Type"]) 

  put re"^\/litestore\/docs\/?(.*)":
    let r = put("docs", 
                request.matches[0], 
                request.body, 
                request.headers)
    resp(r.code, r.content, r.headers["Content-Type"]) 

  patch re"^\/litestore\/docs\/?(.*)":
    let r = patch("docs", 
                  request.matches[0], 
                  request.body, 
                  request.headers)
    resp(r.code, r.content, r.headers["Content-Type"]) 

  delete re"^\/litestore\/docs\/?(.*)":
    let r = delete("docs", 
                   request.matches[0], 
                   request.headers)
    resp(r.code, r.content) 

  head re"^\/litestore\/docs\/?(.*)":
    let r = head("docs", 
                 request.matches[0], 
                 request.headers)
    headers = newStringTable()
    for key, value in r.headers.pairs:
      headers[key] = value
    await response.sendHeaders(r.code, headers)
    response.client.close()

  options re"^\/litestore\/docs\/?(.*)":
    let r = options("docs", 
                    request.matches[0], 
                    request.headers)
    headers = newStringTable()
    for key, value in r.headers.pairs:
      headers[key] = value
    await response.sendHeaders(r.code, headers)
    response.client.close()

runForever()

High-Level Nim API

The following procs map 1:1 to the corresponding HTTP methods. This API is recommended for most uses, as every method triggers exactly the same logic as the corresponding HTTP request.

get

 proc get*(resource, id: string, params = newStringTable(), headers = newHttpHeaders()): LSResponse

Retrieves a resource.

Parameters
resource
The resource to perform the operation on info or docs.
id
The identifier of the resource (set to an empty string if not needed).
params
The parameters to pass to the operation (see the corresponding HTTP querystring parameters).
headers
An HttpHeaders object containing the headers of the request. Use the Content-Type header to specify the content type of the body parameter.

post

 proc post*(resource, id, body: string, headers = newHttpHeaders()): LSResponse

Creates a new resource.

Parameters
resource
The resource to perform the operation on info or docs.
folder
The folder that will contain the resource (set to an empty string if not needed).
body
The request body.
headers
An HttpHeaders object containing the headers of the request. Use the Content-Type header to specify the content type of the body parameter.

put

 proc put*(resource, id, body: string, headers = newHttpHeaders()): LSResponse

Modifies an existing resource.

Parameters
resource
The resource to perform the operation on info or docs.
id
The identifier of the resource (set to an empty string if not needed).
body
The request body.
headers
An HttpHeaders object containing the headers of the request. Use the Content-Type header to specify the content type of the body parameter.

patch

 proc patch*(resource, id, body: string, headers = newHttpHeaders()): LSResponse

Modifies the tags of an existing resource (or the data of a JSON document).

Parameters
resource
The resource to perform the operation on info or docs.
id
The identifier of the resource (set to an empty string if not needed).
body
The request body.
headers
An HttpHeaders object containing the headers of the request. Use the Content-Type header to specify the content type of the body parameter.

delete

 proc delete*(resource, id: string, headers = newHttpHeaders()): LSResponse

Deletes an existing resource.

Parameters
resource
The resource to perform the operation on info or docs.
id
The identifier of the resource (set to an empty string if not needed).
headers
An HttpHeaders object containing the headers of the request. Use the Content-Type header to specify the content type of the body parameter.

head

 proc head*(resource, id: string, headers = newHttpHeaders()): LSResponse

Checks whether a resource exists or not.

Parameters
resource
The resource to perform the operation on info or docs.
id
The identifier of the resource (set to an empty string if not needed).
headers
An HttpHeaders object containing the headers of the request. Use the Content-Type header to specify the content type of the body parameter.

options

 proc options*(resource, id: string, headers = newHttpHeaders()): LSResponse

Checks what HTTP methods are supported by the specified resource.

Parameters
resource
The resource to perform the operation on info or docs.
id
The identifier of the resource (set to an empty string if not needed).
headers
An HttpHeaders object containing the headers of the request. Use the Content-Type header to specify the content type of the body parameter.

Low-Level Nim API

The following procs implement low-level operations on the LiteStore data store. Unlike the high-level API, the procs exposed by this API do not perform the same validation checks as the corresponding HTTP operations, and they are meant to be used with caution in more advanced use cases.

getInfo

 proc getInfo*(): LSResponse

Retrieves information about the currently-loaded data store file.

getRawDocuments

 proc getRawDocuments*(options = newQueryOptions()): LSResponse

Retrieves multiple documents in JSON format based on the specified options.

Parameters
options
A QueryOptions object representing a query to execute on a document.

getDocument

 proc getDocument*(id: string, options = newQueryOptions()): LSResponse

Retrieves a single document.

Parameters
id
The identifier of the resource (set to an empty string if not needed).
options
A QueryOptions object representing a query to execute on a document.

getRawDocument

 proc getRawDocument*(id: string, options = newQueryOptions()): LSResponse

Retrieves a single document in JSON format.

Parameters
id
The identifier of the resource (set to an empty string if not needed).
options
A QueryOptions object representing a query to execute on a document.

deleteDocument

 proc deleteDocument*(id: string): LSResponse

Deletes the specified document.

Parameters
id
The identifier of the resource (set to an empty string if not needed).

postDocument

 proc postDocument*(body, ct: string, folder=""): LSResponse

Creates a new document in the specified folder.

Parameters
body
The request body.
ct
Determines the content type of the body parameter.
folder
A logical folder where the document will be saved.

putDocument

 proc putDocument*(id, body, ct: string): LSResponse

Modifies an existing document.

Parameters
id
The identifier of the resource (set to an empty string if not needed).
body
The request body.
ct
Determines the content type of the body parameter.

patchDocument

 proc patchDocument*(id, body: string): LSResponse

Modifies the tags of an existing document (or the body of a JSON document).

Parameters
id
The identifier of the resource (set to an empty string if not needed).
body
The request body.

Credits

LiteStore was created and is maintained by Fabio Cevasco, with contributions from:

Special thanks to the following individuals, communities and organizations that made LiteStore possible: