{ "cells": [ { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Aerospike Document API for JSON Documents\n", "\n", "This notebook illustrates with code examples the Aerospike Document API, which is used to store JSON documents in the Aerospike Database and perform operations on the stored documents.\n", "\n", "This notebook requires the Aerospike Database running locally with Java kernel and Aerospike Java Client. To create a Docker container that satisfies the requirements and holds a copy of Aerospike notebooks, visit the [Aerospike Notebooks Repo](https://github.com/aerospike-examples/interactive-notebooks)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction\n", "The Aerispike Document library is available in [this github repository](https://github.com/aerospike/aerospike-document-lib). It is described in blog posts [this](https://medium.com/aerospike-developer-blog/aerospike-document-api-fd8870b4106c) and [this](https://medium.com/aerospike-developer-blog/aerospike-document-api-jsonpath-queries-bd6260b2d076).\n", " \n", "This goal of this notebook is to demonstrate the Document API and JSONPath capabilities. \n", "\n", "The main topics in this notebook include:\n", "- Document API methods and code examples\n", "- JSONPath overview and code examples\n", "- Additional JSONPath capabilities and code examples" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisites\n", "This tutorial assumes familiarity with the following topics:\n", "- [Aerospike Notebooks - Readme and Tips](../readme_tips.ipynb)\n", "- [Hello World](hello_world.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Ensure database is running\n", "This notebook requires that the Aerospike Database is running." ] }, { "cell_type": "code", "execution_count": 93, "metadata": {}, "outputs": [], "source": [ "import io.github.spencerpark.ijava.IJava;\n", "import io.github.spencerpark.jupyter.kernel.magic.common.Shell;\n", "IJava.getKernelInstance().getMagics().registerMagics(Shell.class);\n", "%sh asd" ] }, { "cell_type": "markdown", "metadata": { "hide_input": false }, "source": [ "### Download and Install Additional Components\n", "Aerospike Java client 6.1.6 and the document api library 2.0.0." ] }, { "cell_type": "code", "execution_count": 94, "metadata": {}, "outputs": [], "source": [ "%%loadFromPOM\n", "\n", " \n", " com.aerospike\n", " aerospike-client\n", " 6.1.6\n", " \n", " \n", " com.aerospike\n", " aerospike-document-api\n", " 2.0.0\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Imports" ] }, { "cell_type": "code", "execution_count": 95, "metadata": {}, "outputs": [], "source": [ "import java.io.IOException;\n", "import java.nio.charset.StandardCharsets;\n", "import java.nio.file.Files;\n", "import java.nio.file.Paths;\n", "import java.util.stream.Stream;\n", "import java.util.stream.Collectors;\n", "import com.fasterxml.jackson.databind.JsonNode;\n", "import com.fasterxml.jackson.databind.JsonNode;\n", "import com.jayway.jsonpath.JsonPath;\n", "\n", "import com.aerospike.client.AerospikeException;\n", "import com.aerospike.client.AerospikeClient;\n", "import com.aerospike.client.Key;\n", "import com.aerospike.client.BatchRecord;\n", "import com.aerospike.documentapi.batch.BatchOperation;\n", "import com.aerospike.documentapi.batch.GetBatchOperation;\n", "import com.aerospike.documentapi.batch.PutBatchOperation;\n", "import com.aerospike.documentapi.batch.AppendBatchOperation;\n", "import com.aerospike.documentapi.batch.DeleteBatchOperation;\n", "\n", "import com.aerospike.documentapi.*;\n", "import com.aerospike.documentapi.util.JsonConverters;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define Convenience Functions\n", "Define convenience functions `readJSONFromAFile`, `PrintDocuments`, and `truncateTestData`." ] }, { "cell_type": "code", "execution_count": 96, "metadata": {}, "outputs": [], "source": [ "// define a convenience function to get the document bin from all records and print\n", "void PrintDocuments() {\n", " List batchOpsList = new ArrayList<>();\n", " batchOpsList.add(new GetBatchOperation(key1, Collections.singletonList(documentBinName), \"$\"));\n", " batchOpsList.add(new GetBatchOperation(key2, Collections.singletonList(documentBinName), \"$\"));\n", " batchOpsList.add(new GetBatchOperation(key3, Collections.singletonList(documentBinName), \"$\"));\n", " List results = documentClient.batchPerform(batchOpsList, true);\n", " results.stream().forEach(s -> System.out.println(s.record.bins.get(\"documentBin\")));\n", "}\n", "\n", "// convenience function to read JSON doc from a file\n", "String readJSONFromAFile(String filePath) throws IOException {\n", " StringBuilder contentBuilder = new StringBuilder();\n", " Stream stream = Files.lines(Paths.get(filePath), StandardCharsets.UTF_8);\n", " stream.forEach(s -> contentBuilder.append(s).append(\"\\n\"));\n", " return contentBuilder.toString();\n", "}\n", "\n", "// convenience function to truncate test data\n", "void truncateTestData() {\n", " try {\n", " aerospikeClient.truncate(null, NAMESPACE, SET, null);\n", " }\n", " catch (AerospikeException e) {\n", " // ignore\n", " }\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Document API\n", "\n", "The Aerospike Document API provides CRUD operations at arbitrary points within a JSON document as described in [this](https://medium.com/aerospike-developer-blog/aerospike-document-api-fd8870b4106c) blog post.\n", "\n", "## Interface \n", "The document API is as represented in the interface [IAerospikeDocumentClient](https://github.com/aerospike/aerospike-document-lib/blob/main/src/main/java/com/aerospike/documentapi/IAerospikeDocumentClient.java), which is shown below for convenience. The API has the CRUD methods `get`, `put`, `append`, and `delete`, and allows a JSONPath argument to specify parts of the document simply and expressively to apply these methods. Starting with release 2.0, a powerful batch functionality is also available through the `batchPerform` method that allows multiple operations to be performed in a single request, where each operation is get/put/append/delete on a record. \n", "\n", "
\n",
    "    /**\n",
    "     * Retrieve an object matched by JSON path.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binName  name of a bin storing json.\n",
    "     * @param jsonPath JSON path matching the required elements.\n",
    "     * @return object matched by jsonPath.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    Object get(Key key, String binName, String jsonPath);\n",
    "\n",
    "    /**\n",
    "     * Retrieve a map of objects matched by JSON path.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binNames names of bins storing json (all bins with the same document structure).\n",
    "     * @param jsonPath JSON path matching the required elements.\n",
    "     * @return A map of objects matched by jsonPath with bin names as keys.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    Map get(Key key, Collection binNames, String jsonPath);\n",
    "\n",
    "    /**\n",
    "     * Put a JSON document.\n",
    "     *\n",
    "     * @param key        Aerospike Key.\n",
    "     * @param binName    name of a bin to store json.\n",
    "     * @param jsonObject JSON object (document) to put.\n",
    "     */\n",
    "    void put(Key key, String binName, JsonNode jsonObject);\n",
    "\n",
    "    /**\n",
    "     * Put an object at a particular path in JSON document.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binName  name of a bin storing json.\n",
    "     * @param jsonPath A JSON path to put the given JSON object in.\n",
    "     * @param object   An object to put in the given JSON path.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    void put(Key key, String binName, String jsonPath, Object object);\n",
    "\n",
    "    /**\n",
    "     * Put an object at a particular path in JSON document.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binNames names of bins storing json (all bins with the same document structure).\n",
    "     * @param jsonPath JSON path to put the given JSON object in.\n",
    "     * @param object   the object to be put at the given JSON path.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    void put(Key key, Collection binNames, String jsonPath, Object object);\n",
    "\n",
    "    /**\n",
    "     * Append an object to a collection at a particular path in JSON document.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binName  name of a bin storing json.\n",
    "     * @param jsonPath JSON path that includes a collection to append the given JSON object to.\n",
    "     * @param object   the object to be appended at the given JSON path.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    void append(Key key, String binName, String jsonPath, Object object);\n",
    "\n",
    "    /**\n",
    "     * Append an object to a collection at a particular path in JSON document.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binNames names of bins storing json (all bins with the same document structure).\n",
    "     * @param jsonPath JSON path that includes a collection to append the given JSON object to.\n",
    "     * @param object   the object to be appended at the given JSON path.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    void append(Key key, Collection binNames, String jsonPath, Object object);\n",
    "\n",
    "    /**\n",
    "     * Delete an object at a particular path in JSON document.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binName  name of a bin storing json.\n",
    "     * @param jsonPath JSON path for the object deletion.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    void delete(Key key, String binName, String jsonPath);\n",
    "\n",
    "    /**\n",
    "     * Delete an object at a particular path in JSON document.\n",
    "     *\n",
    "     * @param key      Aerospike Key.\n",
    "     * @param binNames names of bins storing json (all bins with the same document structure).\n",
    "     * @param jsonPath JSON path for the object deletion.\n",
    "     * @throws DocumentApiException if there was an error.\n",
    "     */\n",
    "    void delete(Key key, Collection binNames, String jsonPath);\n",
    "\n",
    "    /**\n",
    "     * Perform batch operations.\n",
    "     *\n",
    "     * 

Operations order is preserved only for those 1-step operations\n", " * (with JSONPath that contains only array and/or map elements)\n", " * that have unique Aerospike keys within a batch.

\n", " *

Every 2-step operation (with JSONPath containing wildcards, recursive descent, filters, functions, scripts)\n", " * should have unique Aerospike key within a batch.\n", " *\n", " * @param batchOperations a list of batch operations to apply.\n", " * @param parallel whether batch processing stream operations should run in parallel.\n", " * @return a list of corresponding {@link BatchRecord} results.\n", " * @throws DocumentApiException if there was an error.\n", " * @throws IllegalArgumentException if the batch has multiple two-step operations with the same key.\n", " */\n", " List batchPerform(List batchOperations, boolean parallel);\n", "}\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example JSON Document\n", "The example JSON document used is stored in file `doc_api_example_tommyleejones.json`, and has this json:\n", "\n", "
\n",
    "{\n",
    "  \"forenames\": [\n",
    "    \"Tommy\",\n",
    "    \"Lee\"\n",
    "  ],\n",
    "  \"surname\": \"Jones\",\n",
    "  \"date_of_birth\": {\n",
    "    \"day\": 15,\n",
    "    \"month\": 9,\n",
    "    \"year\": 1946\n",
    "  },\n",
    "  \"selected_filmography\":{\n",
    "    \"2012\":[\"Lincoln\",\"Men In Black 3\"],\n",
    "    \"2007\":[\"No Country For Old Men\"],\n",
    "    \"2002\":[\"Men in Black 2\"],\n",
    "    \"1997\":[\"Men in Black\",\"Volcano\"],\n",
    "    \"1994\":[\"Natural Born Killers\",\"Cobb\"],\n",
    "    \"1991\":[\"JFK\"],\n",
    "    \"1980\":[\"Coal Miner's Daughter\",\"Barn Burning\"]\n",
    "  },\n",
    "  \"imdb_rank\":{\n",
    "    \"source\":\"https://www.imdb.com/list/ls050274118/\",\n",
    "    \"rank\":51\n",
    "  },\n",
    "  \"best_films_ranked\": [\n",
    "    {\n",
    "      \"source\": \"http://www.rottentomatoes.com\",\n",
    "      \"films\": [\"The Fugitive\",\"No Country For Old Men\",\"Men In Black\",\"Coal Miner's Daughter\",\"Lincoln\"]\n",
    "    },\n",
    "    {\n",
    "      \"source\":\"https://medium.com/the-greatest-films-according-to-me/10-greatest-films-of-tommy-lee-jones-97426103e3d6\",\n",
    "      \"films\":[\"The Three Burials of Melquiades Estrada\",\"The Homesman\",\"No Country for Old Men\",\"In the Valley of Elah\",\"Coal Miner's Daughter\"]\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Store JSON Document" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Iniialize Document Client\n", "Create a document client using an Aerospike client." ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Initialized Aerospike client and connected to the cluster.\n", "Initialized document client from the Aerospike client.\n" ] } ], "source": [ "final String NAMESPACE = \"test\";\n", "final String SET = \"document-api\";\n", "\n", "AerospikeClient aerospikeClient = new AerospikeClient(\"localhost\", 3000);\n", "System.out.println(\"Initialized Aerospike client and connected to the cluster.\");\n", "AerospikeDocumentClient documentClient = new AerospikeDocumentClient(aerospikeClient);\n", "System.out.println(\"Initialized document client from the Aerospike client.\");;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Read JSON Document\n", "Using the convenience function `readJSONFromAFile` defined above, read a json file." ] }, { "cell_type": "code", "execution_count": 98, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Read JSON doc from file.\n", "JSON doc: {\"forenames\":[\"Tommy\",\"Lee\"],\"surname\":\"Jones\",\"date_of_birth\":{\"day\":15,\"month\":9,\"year\":1946},\"selected_filmography\":{\"2012\":[\"Lincoln\",\"Men In Black 3\"],\"2007\":[\"No Country For Old Men\"],\"2002\":[\"Men in Black 2\"],\"1997\":[\"Men in Black\",\"Volcano\"],\"1994\":[\"Natural Born Killers\",\"Cobb\"],\"1991\":[\"JFK\"],\"1980\":[\"Coal Miner's Daughter\",\"Barn Burning\"]},\"imdb_rank\":{\"source\":\"https://www.imdb.com/list/ls050274118/\",\"rank\":51},\"best_films_ranked\":[{\"source\":\"http://www.rottentomatoes.com\",\"films\":[\"The Fugitive\",\"No Country For Old Men\",\"Men In Black\",\"Coal Miner's Daughter\",\"Lincoln\"]},{\"source\":\"https://medium.com/the-greatest-films-according-to-me/10-greatest-films-of-tommy-lee-jones-97426103e3d6\",\"films\":[\"The Three Burials of Melquiades Estrada\",\"The Homesman\",\"No Country for Old Men\",\"In the Valley of Elah\",\"Coal Miner's Daughter\"]}]}\n" ] } ], "source": [ "// Read the json document into a string.\n", "String jsonString = readJSONFromAFile(\"doc_api_example_tommyleejones.json\");\n", "System.out.println(\"Read JSON doc from file.\");\n", "\n", "// Convert JSON string to a JsonNode\n", "JsonNode jsonNode = JsonConverters.convertStringToJsonNode(jsonString);\n", "System.out.format(\"JSON doc: %s\\n\", jsonNode);;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Add JSON Document to Database" ] }, { "cell_type": "code", "execution_count": 99, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Stored JSON doc in database.\n" ] } ], "source": [ "// Construct a key\n", "Key tommyLeeJonesDBKey = new Key(NAMESPACE, SET, \"tommy-lee-jones.json\");\n", "\n", "// Add the document to database\n", "String documentBinName = \"documentBin\";\n", "documentClient.put(tommyLeeJonesDBKey, documentBinName, jsonNode);\n", "System.out.println(\"Stored JSON doc in database.\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Read" ] }, { "cell_type": "code", "execution_count": 100, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "The Fugitive" ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// indexed array elements\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$.best_films_ranked[0].films[0]\" );" ] }, { "cell_type": "code", "execution_count": 101, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[Coal Miner's Daughter, Barn Burning]" ] }, "execution_count": 101, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// an array element denoted by a simple path\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$.selected_filmography.1980\");" ] }, { "cell_type": "code", "execution_count": 102, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{1997=[Men in Black, Volcano], 2007=[No Country For Old Men], 1994=[Natural Born Killers, Cobb], 2002=[Men in Black 2], 1991=[JFK], 1980=[Coal Miner's Daughter, Barn Burning], 2012=[Lincoln, Men In Black 3]}" ] }, "execution_count": 102, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// complex json path: wild-card\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$.selected_filmography.*\");" ] }, { "cell_type": "code", "execution_count": 103, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[\"http:\\/\\/www.rottentomatoes.com\",\"https:\\/\\/medium.com\\/the-greatest-films-according-to-me\\/10-greatest-films-of-tommy-lee-jones-97426103e3d6\",\"https:\\/\\/www.imdb.com\\/list\\/ls050274118\\/\"]" ] }, "execution_count": 103, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// complex json path: matching elements in a subtree\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$..source\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Update" ] }, { "cell_type": "code", "execution_count": 104, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[Ad Astra]" ] }, "execution_count": 104, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// add year 2019 and films\n", "List _2019Films = new Vector();\n", "_2019Films.add(\"Ad Astra\");\n", "documentClient.put(tommyLeeJonesDBKey, documentBinName, \"$.selected_filmography.2019\", _2019Films);\n", "// read the update\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$.selected_filmography.2019\" );" ] }, { "cell_type": "code", "execution_count": 105, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "45" ] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// update the imdb rank\n", "documentClient.put(tommyLeeJonesDBKey, documentBinName, \"$.imdb_rank.rank\", 45);\n", "// read the update\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$.imdb_rank.rank\" );" ] }, { "cell_type": "code", "execution_count": 106, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[The Fugitive, No Country For Old Men, Men In Black, Coal Miner's Daughter, Lincoln, Rolling Thunder, The Three Burials]" ] }, "execution_count": 106, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// append films to best_films_ranked\n", "documentClient.append(tommyLeeJonesDBKey, documentBinName, \"$.best_films_ranked[0].films\", \"Rolling Thunder\");\n", "documentClient.append(tommyLeeJonesDBKey, documentBinName, \"$.best_films_ranked[0].films\", \"The Three Burials\");\n", "// read the updates\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$.best_films_ranked[0].films\" );" ] }, { "cell_type": "code", "execution_count": 107, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{\"1997\":[\"Men in Black\",\"Volcano\"],\"2019\":[\"Ad Astra\"],\"2007\":[\"No Country For Old Men\"],\"1994\":[\"Natural Born Killers\",\"Cobb\"],\"2002\":[\"Men in Black 2\"],\"1991\":[\"JFK\"],\"1980\":[\"Coal Miner's Daughter\",\"Barn Burning\"],\"2012\":[\"Lincoln\",\"Men In Black 3\"],\"2020\":[\"The Comeback Trail\",\"Wander\"]}]" ] }, "execution_count": 107, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// put a json-node object\n", "String jsonString = \"[\\\"The Comeback Trail\\\",\\\"Wander\\\"]\";\n", "//String jsonString = \"{\\\"a\\\":[\\\"The Comeback Trail\\\",\\\"Wander\\\"]}\";\n", "//String jsonString = \"{\\\"last-updated\\\": 2020,\\\"updated-by\\\": \\\"A.Fan\\\"}\";\n", "// convert JSON string to a JsonNode\n", "JsonNode jsonNode = JsonConverters.convertStringToJsonNode(jsonString);\n", "//put the json-node\n", "documentClient.put(tommyLeeJonesDBKey, documentBinName, \"$.selected_filmography.2020\", jsonNode);\n", "// read the update\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$..selected_filmography\" );" ] }, { "cell_type": "code", "execution_count": 108, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{\"last-updated\":2020,\"updated-by\":\"A.Fan\"}" ] }, "execution_count": 108, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// put a json-node object\n", "String jsonString = \"{\\\"last-updated\\\": 2020,\\\"updated-by\\\": \\\"A.Fan\\\"}\";\n", "// convert JSON string to a JsonNode\n", "JsonNode jsonNode = JsonConverters.convertStringToJsonNode(jsonString);\n", "//put the json-node\n", "documentClient.put(tommyLeeJonesDBKey, documentBinName, \"$.notes\", jsonNode);\n", "// read the update\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$.notes\" );" ] }, { "cell_type": "code", "execution_count": 109, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[[\"Men in Black\",\"Volcano\",\"TEST\"],[\"Ad Astra\",\"TEST\"],[\"No Country For Old Men\",\"TEST\"],[\"Natural Born Killers\",\"Cobb\",\"TEST\"],[\"Men in Black 2\",\"TEST\"],[\"JFK\",\"TEST\"],[\"Coal Miner's Daughter\",\"Barn Burning\",\"TEST\"],[\"Lincoln\",\"Men In Black 3\",\"TEST\"],[\"The Comeback Trail\",\"Wander\",\"TEST\"]]" ] }, "execution_count": 109, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// complex json path: wild-card\n", "documentClient.append(tommyLeeJonesDBKey, documentBinName, \"$..selected_filmography[*]\", \"TEST\");\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$..selected_filmography[*]\" );" ] }, { "cell_type": "code", "execution_count": 110, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[[\"The Fugitive\",\"No Country For Old Men\",\"Men In Black\",\"Coal Miner's Daughter\",\"Lincoln\",\"Rolling Thunder\",\"The Three Burials\",\"TEST\"],[\"The Three Burials of Melquiades Estrada\",\"The Homesman\",\"No Country for Old Men\",\"In the Valley of Elah\",\"Coal Miner's Daughter\",\"TEST\"]]" ] }, "execution_count": 110, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// complex json path: matching elements in a subtree\n", "//append to all arrays\n", "documentClient.append(tommyLeeJonesDBKey, documentBinName, \"$..films\", \"TEST\");\n", "// read the update\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$..films\" );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Delete" ] }, { "cell_type": "code", "execution_count": 111, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{\"1997\":[\"Men in Black\",\"Volcano\"],\"2019\":[\"Ad Astra\",\"TEST\"],\"2007\":[\"No Country For Old Men\",\"TEST\"],\"1994\":[\"Natural Born Killers\",\"Cobb\"],\"2002\":[\"Men in Black 2\",\"TEST\"],\"1991\":[\"JFK\",\"TEST\"],\"1980\":[\"Coal Miner's Daughter\",\"Barn Burning\"],\"2012\":[\"Lincoln\",\"Men In Black 3\"],\"2020\":[\"The Comeback Trail\",\"Wander\"]}]" ] }, "execution_count": 111, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// complex json path: wild-card\n", "documentClient.delete(tommyLeeJonesDBKey, documentBinName, \"$.selected_filmography.[*][2]\");\n", "documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$..selected_filmography\" );" ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before delete: \n", "{best_films_ranked=[{films=[The Fugitive, No Country For Old Men, Men In Black, Coal Miner's Daughter, Lincoln, Rolling Thunder, The Three Burials, TEST], source=http://www.rottentomatoes.com}, {films=[The Three Burials of Melquiades Estrada, The Homesman, No Country for Old Men, In the Valley of Elah, Coal Miner's Daughter, TEST], source=https://medium.com/the-greatest-films-according-to-me/10-greatest-films-of-tommy-lee-jones-97426103e3d6}], date_of_birth={month=9, day=15, year=1946}, forenames=[Tommy, Lee], imdb_rank={rank=45, source=https://www.imdb.com/list/ls050274118/}, notes={last-updated=2020, updated-by=A.Fan}, selected_filmography={1997=[Men in Black, Volcano], 2019=[Ad Astra, TEST], 2007=[No Country For Old Men, TEST], 1994=[Natural Born Killers, Cobb], 2002=[Men in Black 2, TEST], 1991=[JFK, TEST], 1980=[Coal Miner's Daughter, Barn Burning], 2012=[Lincoln, Men In Black 3], 2020=[The Comeback Trail, Wander]}, surname=Jones}\n", "After delete: \n", "{}\n" ] } ], "source": [ "// print state before deletion\n", "Object docObject = documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$\");\n", "System.out.format(\"Before delete: \\n%s\\n\", docObject);\n", "// delete all descendants of the document root\n", "documentClient.delete(tommyLeeJonesDBKey, documentBinName, \"$..*\");\n", "// print state after deletion\n", "docObject = documentClient.get(tommyLeeJonesDBKey, documentBinName, \"$\");\n", "System.out.format(\"After delete: \\n%s\\n\", docObject);;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Batch Operations\n", "In Document API versions 2.0+, batch operations are supported: Document operations involving multiple records can be performed in a single batch request. An operation in a batch can be get, put, append, or delete, and specifies the operand using a record key, document bin(s), and JSON path. \n", "\n", "#### Simple and Complex JSON Paths in Operations\n", "A JSON path that resolves in a single element in the target document is considered a simple JSON path with a `1-step` operation because the operation can be performed in a single server request. Examples of simple JSON paths are any path resolving to a single element such as a scalar type, an object, or an array.\n", "\n", "A JSON path that resolves in multiple elements in the target document is considered a complex JSON path requiring a `2-step` operation because the operation takes two steps: the first step to resolve all operand elements within the document, and the second step to perform individual operations on the operand elements. Examples of complex JSON paths are paths resolving to multiple elements such as wild-cards (\\*), all instances in a subtree (..attr), filters (?(@.attr condition), and functions (such as @.length).\n", "\n", "#### Constraints on 2-Step Operations\n", "Multiple operations on the same operands are not allowed with complex JSON paths that must be processed in two steps. Specifically, a batch of records that has the same record appearing more than once is not allowed. The reason being that such operations can yield different results depending on the execution sequence of the multiple client-server interactions. \n", "\n", "Note also that the document api allows the same operation to be performed on multiple document bins within a record. However, a batch request may not have the same bin appearing more than once. Such redundant appearance of a bin will amount to performing the same operation multiple times, which does not make sense.\n", "\n", "#### Parallel execution\n", "A batch request can be submitted with a `parallel` flag. If this flag is set to false, operations on records on a single node are performed sequentially in a single thread. Otherwise they may be executed in parallel. Sequential execution must be specified if the batch operations depend on the specific order of execution. For example, first append an element to an array, and then retrieve the length of the new array.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Batch Operations to Add Multiple Document Records\n", "Below we add multiple documents in a batch operation." ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{arr=[11, 12], id=1, obs={t1=101, t2=102}}\n", "{arr=[21, 22], id=2, obs={t1=201, t2=202}}\n", "{arr=[31, 32], id=3, obs={t1=301, t2=302}}\n" ] } ], "source": [ "final String SET = \"document_api_batch\";\n", "// Insert three documents as separate records with keys 1, 2, and 3\n", "Key key1 = new Key(NAMESPACE, SET, 1);\n", "Key key2 = new Key(NAMESPACE, SET, 2);\n", "Key key3 = new Key(NAMESPACE, SET, 3);\n", "// Create the documents.\n", "// 1 \n", "String jsonString = \"{\\\"id\\\": 1, \\\"obs\\\": {\\\"t1\\\": 101, \\\"t2\\\": 102}, \\\"arr\\\": [11, 12] }\";\n", "JsonNode jsonNode = JsonConverters.convertStringToJsonNode(jsonString);\n", "documentClient.put(key1, documentBinName, jsonNode);\n", "// 2\n", "jsonString = \"{\\\"id\\\": 2, \\\"obs\\\": {\\\"t1\\\": 201, \\\"t2\\\": 202}, \\\"arr\\\": [21, 22] }\";\n", "jsonNode = JsonConverters.convertStringToJsonNode(jsonString);\n", "documentClient.put(key2, documentBinName, jsonNode);\n", "// 3\n", "jsonString = \"{\\\"id\\\": 3, \\\"obs\\\": {\\\"t1\\\": 301, \\\"t2\\\": 302}, \\\"arr\\\": [31, 32] }\";\n", "jsonNode = JsonConverters.convertStringToJsonNode(jsonString);\n", "documentClient.put(key3, documentBinName, jsonNode);\n", "\n", "PrintDocuments();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Batch Operations with Simple JSON Paths, Non-Unique Records\n", "Below is an example of a batch request where multiple operations are performed on different records, and two operations are performed on one record:\n", "- key1: \n", " - put a new element t3=103 in obs object\n", " - key1: delete the second element in arr array\n", "- key2: put 0 in the first position of arr array\n", "- key3: append 33 to arr array" ] }, { "cell_type": "code", "execution_count": 114, "metadata": {}, "outputs": [], "source": [ "// batch operations\n", "// Insert\n", "BatchOperation op1 = new PutBatchOperation(\n", " key1, \n", " Collections.singletonList(documentBinName),\n", " \"$.obs.t3\",\n", " \"103\"\n", ");\n", "\n", "// Update\n", "BatchOperation op2 = new PutBatchOperation(\n", " key2,\n", " Collections.singletonList(documentBinName),\n", " \"$.arr[0]\",\n", " 0\n", ");\n", "\n", "// Append\n", "BatchOperation op3 = new AppendBatchOperation(\n", " key3,\n", " Collections.singletonList(documentBinName),\n", " \"$.arr\",\n", " 33\n", ");\n", "\n", "// Delete\n", "BatchOperation op4 = new DeleteBatchOperation(\n", " key1,\n", " Collections.singletonList(documentBinName),\n", " \"$.arr[1]\"\n", ");\n", "\n", "// Collecting operations and running\n", "List batchOpsList = new ArrayList<>();\n", "batchOpsList.add(op1);\n", "batchOpsList.add(op2);\n", "batchOpsList.add(op3);\n", "batchOpsList.add(op4);\n", "\n", "List results = documentClient.batchPerform(batchOpsList, true);" ] }, { "cell_type": "code", "execution_count": 115, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{arr=[11], id=1, obs={t1=101, t2=102, t3=103}}\n", "{arr=[0, 22], id=2, obs={t1=201, t2=202}}\n", "{arr=[31, 32, 33], id=3, obs={t1=301, t2=302}}\n" ] } ], "source": [ "// examine document bins - ensure that:\n", "// id=1: obs object has a new element t3=103\n", "// the second element in arr array is removed\n", "// id=2: arr array has 0 in the first position\n", "// id=3: arr array has 33 as the last element\n", "PrintDocuments();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Batch operations with Complex JSON Paths, Unique Records\n", "In the following example we illustrate JSON paths with wildcard and function.\n", "- key1: make all elements of arr array 0, or arr[\\*] = 0\n", "- key2: remove t3 from obs, or obs.t3" ] }, { "cell_type": "code", "execution_count": 116, "metadata": {}, "outputs": [], "source": [ "// batch operations\n", "// id=1: make all arr elments 0\n", "BatchOperation op1 = new PutBatchOperation(\n", " key1, \n", " Collections.singletonList(documentBinName),\n", " \"$.arr[*]\",\n", " 0\n", ");\n", "\n", "// id=2: remove t3 from obs\n", "BatchOperation op2 = new DeleteBatchOperation(\n", " key2,\n", " Collections.singletonList(documentBinName),\n", " \"$.obs.t3\"\n", ");\n", "\n", "\n", "// collect operations\n", "List batchOpsList = new ArrayList<>();\n", "batchOpsList.add(op1);\n", "batchOpsList.add(op2);\n", "\n", "// execute\n", "List results = documentClient.batchPerform(batchOpsList, true);\n" ] }, { "cell_type": "code", "execution_count": 117, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{arr=[0], id=1, obs={t1=101, t2=102, t3=103}}\n", "{arr=[0, 22], id=2, obs={t1=201, t2=202}}\n", "{arr=[31, 32, 33], id=3, obs={t1=301, t2=302}}\n" ] } ], "source": [ "// examine document bins - ensure that:\n", "// id=1: arr elements should be all 0\n", "// id=2: t3 is removed from obs \n", "PrintDocuments();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Error case: Batch Operations with Complex JSON Paths, Non-Unique Records\n", "Below we submit two operations on the same record with one or more 2-step operations involving a complex JSON path. Note it disallows the request by generating an exception." ] }, { "cell_type": "code", "execution_count": 118, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Error: java.lang.IllegalArgumentException: Multiple two-step operations with the same key are not allowed\n" ] } ], "source": [ "// batch operations\n", "// id=1: a 2-step operation with a complex json path\n", "BatchOperation op1 = new PutBatchOperation(\n", " key1, \n", " Collections.singletonList(documentBinName),\n", " \"$.arr[*]\",\n", " 0\n", ");\n", "\n", "// id=1: another operation \n", "BatchOperation op2 = new AppendBatchOperation(\n", " key1, \n", " Collections.singletonList(documentBinName),\n", " \"$.arr\",\n", " 100\n", ");\n", "\n", "// collect operations\n", "List batchOpsList = new ArrayList<>();\n", "batchOpsList.add(op1);\n", "batchOpsList.add(op2);\n", "\n", "// excute - should produce an exception\n", "try {\n", " List results = documentClient.batchPerform(batchOpsList, true);\n", "}\n", "catch (java.lang.Exception e) {\n", " System.out.format(\"Error: %s\\n\", e);\n", "}\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# JSONPath Queries\n", "Please refer to this introduction to JSONPath: [JSONPath, XPATH for JSON](https://goessner.net/articles/JsonPath/). \n", "\n", "## Syntax\n", "The following table summarizes its syntax.\n", "\n", "JSONPath | Description\n", ":------- |:-----------\n", "\\$ | the root object/element\n", "@ | the current object/element\n", ". or [] | child operator\n", ".. | recursive descent\n", "* | wildcard. All objects/elements regardless their names\n", "[] | subscript operator in array\n", "[,] | alternate names or array indices as a set\n", "[start:end:step] | array slice operator \n", "[?()] | boolean filter expression\n", "() | script expression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use Examples\n", "The following table summarizes its key uses.\n", "\n", "JSONPath | Result\n", ":-------|:-------\n", "\\$.store.book[*].author | the authors of all books in the store\n", "\\$..author | all authors\n", "\\$.store.* | all things in store, which are some books and a red bicycle\n", "\\$.store..price | the price of everything in the store\n", "\\$..book[2] | the third book\n", "\\$..book[(@.length-1)]\n", "\\$..book[-1:] | the last book in order\n", "\\$..book[0,1]\n", "\\$..book[:2] | the first two books\n", "\\$..book[?(@.isbn)] | filter all books with isbn number\n", "\\$..book[?(@.price<10)] | filter all books cheaper than 10\n", "\\$..* | all members of JSON structure\n", "\\$..book.length() | the number of books" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Code Examples\n", "Below are the code examples from the blog post [Aerospike Document API: JSONPath Queries](https://medium.com/aerospike-developer-blog/aerospike-document-api-jsonpath-queries-bd6260b2d076)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example JSON Document\n", "The example JSON document it uses is stored in file `doc_api_example_store.json`, and has this json:\n", "\n", "
\n",
    "{\n",
    " \"store\": {\n",
    "    \"book\": [\n",
    "      {\n",
    "        \"category\": \"reference\",\n",
    "        \"author\": \"Nigel Rees\",\n",
    "        \"title\": \"Sayings of the Century\",\n",
    "        \"price\": 8.95,\n",
    "        \"ref\": [1,2]\n",
    "      },\n",
    "      {\n",
    "        \"category\": \"fiction\",\n",
    "        \"author\": \"Evelyn Waugh\",\n",
    "        \"title\": \"Sword of Honour\",\n",
    "        \"price\": 12.99,\n",
    "        \"ref\": [2,4,16]\n",
    "      },\n",
    "      {\n",
    "        \"category\": \"fiction\",\n",
    "        \"author\": \"Herman Melville\",\n",
    "        \"title\": \"Moby Dick\",\n",
    "        \"isbn\": \"0-553-21311-3\",\n",
    "        \"price\": 8.99,\n",
    "        \"ref\": [1,3,5]\n",
    "      },\n",
    "      {\n",
    "        \"category\": \"fiction\",\n",
    "        \"author\": \"J. R. R. Tolkien\",\n",
    "        \"title\": \"The Lord of the Rings\",\n",
    "        \"isbn\": \"0-395-19395-8\",\n",
    "        \"price\": 22.99,\n",
    "        \"ref\": [1,2,7]\n",
    "      }\n",
    "    ],\n",
    "    \"bicycle\": {\n",
    "      \"color\": \"red\",\n",
    "      \"price\": 19.95\n",
    "    }\n",
    "  },\n",
    "  \"expensive\": 10\n",
    "}\n",
    "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Store JSON Document\n", "#### Read JSON Document from File" ] }, { "cell_type": "code", "execution_count": 119, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Read JSON doc from file.\n" ] } ], "source": [ "String jsonString = readJSONFromAFile(\"doc_api_example_store.json\");\n", "System.out.println(\"Read JSON doc from file.\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Add JSON Document to Database" ] }, { "cell_type": "code", "execution_count": 120, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"store\":{\"book\":[{\"category\":\"reference\",\"author\":\"Nigel Rees\",\"title\":\"Sayings of the Century\",\"price\":8.95,\"ref\":[1,2]},{\"category\":\"fiction\",\"author\":\"Evelyn Waugh\",\"title\":\"Sword of Honour\",\"price\":12.99,\"ref\":[2,4,16]},{\"category\":\"fiction\",\"author\":\"Herman Melville\",\"title\":\"Moby Dick\",\"isbn\":\"0-553-21311-3\",\"price\":8.99,\"ref\":[1,3,5]},{\"category\":\"fiction\",\"author\":\"J. R. R. Tolkien\",\"title\":\"The Lord of the Rings\",\"isbn\":\"0-395-19395-8\",\"price\":22.99,\"ref\":[1,2,7]}],\"bicycle\":{\"color\":\"red\",\"price\":19.95}},\"expensive\":10}\n", "Stored JSON doc to database.\n" ] } ], "source": [ "// Store the JSON doc in the database and read it back.\n", "\n", "// Create a document client via an existing aerospikeClient\n", "AerospikeDocumentClient documentClient = new AerospikeDocumentClient(aerospikeClient);\n", "\n", "// Convert JSON string to a JsonNode\n", "JsonNode jsonNode = JsonConverters.convertStringToJsonNode(jsonString);\n", "System.out.println(jsonNode);\n", "\n", "// Construct an appropriate key\n", "Key documentKey = new Key(NAMESPACE, SET, \"jsonExampleKey\");\n", "\n", "String documentBinName = \"documentBin\";\n", "// Add to database\n", "documentClient.put(documentKey, documentBinName, jsonNode);\n", "System.out.println(\"Stored JSON doc to database.\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Query All Subnodes" ] }, { "cell_type": "code", "execution_count": 121, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{bicycle={color=red, price=19.95}, book=[{ref=[1, 2], category=reference, title=Sayings of the Century, author=Nigel Rees, price=8.95}, {ref=[2, 4, 16], category=fiction, title=Sword of Honour, author=Evelyn Waugh, price=12.99}, {ref=[1, 3, 5], category=fiction, title=Moby Dick, author=Herman Melville, price=8.99, isbn=0-553-21311-3}, {ref=[1, 2, 7], category=fiction, title=The Lord of the Rings, author=J. R. R. Tolkien, price=22.99, isbn=0-395-19395-8}]}\n" ] } ], "source": [ "// Get all products, both books and bicycles\n", "String jsonPath = \"$.store.*\";\n", "Object objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.println(objectFromDB);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Query Specific Field" ] }, { "cell_type": "code", "execution_count": 122, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\"Nigel Rees\",\"Evelyn Waugh\",\"Herman Melville\",\"J. R. R. Tolkien\"]\n" ] } ], "source": [ "// Get the authors of all books\n", "String jsonPath = \"$.store.book[*].author\";\n", "Object objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.println(objectFromDB);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Modify Field" ] }, { "cell_type": "code", "execution_count": 123, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\"J.K. Rowling\",\"J.K. Rowling\",\"J.K. Rowling\",\"J.K. Rowling\"]\n" ] } ], "source": [ "// 3. Modify the authors of all books to “J.K. Rowling”\n", "// Get the authors of all books\n", "String jsonPath = \"$.store.book[*].author\";\n", "String jsonObject = \"J.K. Rowling\";\n", "// Modify the authors of all books to \"J.K. Rowling\"\n", "documentClient.put(documentKey, documentBinName, jsonPath, jsonObject);\n", "Object objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.println(objectFromDB);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Query with Exists Predicate" ] }, { "cell_type": "code", "execution_count": 124, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[{\"ref\":[1,3,5],\"category\":\"fiction\",\"title\":\"Moby Dick\",\"author\":\"J.K. Rowling\",\"price\":8.99,\"isbn\":\"0-553-21311-3\"},{\"ref\":[1,2,7],\"category\":\"fiction\",\"title\":\"The Lord of the Rings\",\"author\":\"J.K. Rowling\",\"price\":22.99,\"isbn\":\"0-395-19395-8\"}]\n" ] } ], "source": [ "// 4. Get all the books that have an ISBN number\n", "jsonPath = \"$..book[?(@.isbn)]\";\n", "objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.println(objectFromDB);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Query with Boolean Predicate" ] }, { "cell_type": "code", "execution_count": 125, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[{\"ref\":[1,2],\"category\":\"reference\",\"title\":\"Sayings of the Century\",\"author\":\"J.K. Rowling\",\"price\":8.95},{\"ref\":[1,3,5],\"category\":\"fiction\",\"title\":\"Moby Dick\",\"author\":\"J.K. Rowling\",\"price\":8.99,\"isbn\":\"0-553-21311-3\"}]\n" ] } ], "source": [ "// 5. Get all the books in store cheaper than 10\n", "jsonPath = \"$.store.book[?(@.price < 10)]\";\n", "objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.println(objectFromDB);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Query with RegEx" ] }, { "cell_type": "code", "execution_count": 126, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[{\"ref\":[1,2],\"category\":\"reference\",\"title\":\"Sayings of the Century\",\"author\":\"J.K. Rowling\",\"price\":8.95},{\"ref\":[2,4,16],\"category\":\"fiction\",\"title\":\"Sword of Honour\",\"author\":\"J.K. Rowling\",\"price\":12.99},{\"ref\":[1,3,5],\"category\":\"fiction\",\"title\":\"Moby Dick\",\"author\":\"J.K. Rowling\",\"price\":8.99,\"isbn\":\"0-553-21311-3\"},{\"ref\":[1,2,7],\"category\":\"fiction\",\"title\":\"The Lord of the Rings\",\"author\":\"J.K. Rowling\",\"price\":22.99,\"isbn\":\"0-395-19395-8\"}]\n" ] } ], "source": [ "// 6. Get all the books matching regex (ignore case)\n", "jsonPath = \"$..book[?(@.author =~ /.*ROWLING/i)]\";\n", "objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.println(objectFromDB);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Delete Field" ] }, { "cell_type": "code", "execution_count": 127, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before delete: [19.95,8.95,12.99,8.99,22.99]\n", "After delete: []\n" ] } ], "source": [ "// 7. Delete the price field of every object exists in store\n", "// Get the price of everything\n", "String jsonPath = \"$.store..price\";\n", "Object objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.format(\"Before delete: %s\\n\", objectFromDB);\n", "// Delete the price field of every object exists in the store\n", "documentClient.delete(documentKey, documentBinName, jsonPath);\n", "Object objectFromDB = documentClient.get(documentKey, documentBinName, jsonPath);\n", "System.out.format(\"After delete: %s\\n\", objectFromDB);;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### More Query and Transformation Examples\n", "Consider the document: \n", "```\n", " {arr=[{a=1, b=10}, \n", " {a=2, b=20}], \n", " id=4, \n", " obs={\n", " c=[100, 200], \n", " id=x}\n", " }\n", "```\n", "\n", "JSON queries:\n", "```\n", "\"$.arr[*].a\" -> \n", " [1, 2]\n", " \n", "\"$.obs.c[\\*]\" -> \n", " [100, 200]\n", " \n", "\"$..id\" -> \n", " [4, x]\n", " \n", "\"$..c.length()\" -> \n", " 2\n", "```\n", "\n", "Transformations using JSON queries:\n", "```\n", "put(“$.arr[*].a”, 0) -> \n", " {arr=[{a=0, b=10}, {a=0, b=20}], id=4, obs={c=[100, 200], id=x}}\n", " \n", "put(“$.obj.c[\\*]”, 0) -> \n", " {arr=[{a=0, b=10}, {a=0, b=20}], id=4, obs={c=[0, 0], id=x}}\n", " \n", "put(“$..id”, 0) -> \n", " {arr=[{a=0, b=10}, {a=0, b=20}], id=0, obs={c=[0, 0], id=0}}\n", " \n", "put)”$..a”, 1) -> \n", " {arr=[{a=1, b=10}, {a=1, b=20}], id=0, obs={c=[0, 0], id=0}}\n", " \n", "put(\"$..c\", List.of(11, 111)) -> \n", " {arr=[{a=1, b=10}, {a=1, b=20}], id=0, obs={c=[11, 111], id=0}}\n", " \n", "append(\"$.obs.c\", 0) -> \n", " arr=[{a=1, b=10}, {a=1, b=20}], id=0, obs={c=[11, 111, 0], id=0}}\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# More JSONPath Capabilities and Examples\n", "Restore the original `store` document for the following examples." ] }, { "cell_type": "code", "execution_count": 128, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{expensive=10, store={bicycle={color=red, price=19.95}, book=[{ref=[1, 2], category=reference, title=Sayings of the Century, author=Nigel Rees, price=8.95}, {ref=[2, 4, 16], category=fiction, title=Sword of Honour, author=Evelyn Waugh, price=12.99}, {ref=[1, 3, 5], category=fiction, title=Moby Dick, author=Herman Melville, price=8.99, isbn=0-553-21311-3}, {ref=[1, 2, 7], category=fiction, title=The Lord of the Rings, author=J. R. R. Tolkien, price=22.99, isbn=0-395-19395-8}]}}" ] }, "execution_count": 128, "metadata": {}, "output_type": "execute_result" } ], "source": [ "documentClient.put(documentKey, documentBinName, jsonNode);\n", "documentClient.get(documentKey, documentBinName, \"$\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Functions\n", "JSONPath has functions `min`, `sum`, `size`, `length`, etc. See the [JSONPath repo](https://github.com/json-path/JsonPath_) for a full list." ] }, { "cell_type": "code", "execution_count": 129, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 129, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// length(): find number of references to Moby Dick\n", "jsonPath = \"$..store.book[?(@.title == 'Moby Dick')].ref.length()\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "code", "execution_count": 130, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "13.48" ] }, "execution_count": 130, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// avg(): find average price of books\n", "jsonPath = \"$..book..price.avg()\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Filter Operators\n", "Operators like `in` and `anyof` are supported. See the [JSONPath repo](https://github.com/json-path/JsonPath) for a full list." ] }, { "cell_type": "code", "execution_count": 131, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{\"ref\":[1,2],\"category\":\"reference\",\"title\":\"Sayings of the Century\",\"author\":\"Nigel Rees\",\"price\":8.95}]" ] }, "execution_count": 131, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// in: books with category in ['reference', 'biography']\n", "jsonPath = \"$..book[?(@.category in ['reference', 'biography'])]\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "code", "execution_count": 132, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{\"ref\":[1,2],\"category\":\"reference\",\"title\":\"Sayings of the Century\",\"author\":\"Nigel Rees\",\"price\":8.95},{\"ref\":[2,4,16],\"category\":\"fiction\",\"title\":\"Sword of Honour\",\"author\":\"Evelyn Waugh\",\"price\":12.99},{\"ref\":[1,2,7],\"category\":\"fiction\",\"title\":\"The Lord of the Rings\",\"author\":\"J. R. R. Tolkien\",\"price\":22.99,\"isbn\":\"0-395-19395-8\"}]" ] }, "execution_count": 132, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// in: books that have 2 in their reference list\n", "jsonPath = \"$..book[?(2 in @.ref)]\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "code", "execution_count": 133, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{\"ref\":[1,2],\"category\":\"reference\",\"title\":\"Sayings of the Century\",\"author\":\"Nigel Rees\",\"price\":8.95},{\"ref\":[1,3,5],\"category\":\"fiction\",\"title\":\"Moby Dick\",\"author\":\"Herman Melville\",\"price\":8.99,\"isbn\":\"0-553-21311-3\"},{\"ref\":[1,2,7],\"category\":\"fiction\",\"title\":\"The Lord of the Rings\",\"author\":\"J. R. R. Tolkien\",\"price\":22.99,\"isbn\":\"0-395-19395-8\"}]" ] }, "execution_count": 133, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// anyof: books whose ref list has any of 1, 3, 7, or 100.\n", "jsonPath = \"$..book[?(@.ref anyof [1, 3, 7, 100])]\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Predicates" ] }, { "cell_type": "code", "execution_count": 134, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{\"ref\":[2,4,16],\"category\":\"fiction\",\"title\":\"Sword of Honour\",\"author\":\"Evelyn Waugh\",\"price\":12.99}]" ] }, "execution_count": 134, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// books with price between 10 and 20\n", "jsonPath = \"$..book[?(@.price > 10 && @.price < 20)]\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "code", "execution_count": 135, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "[{\"ref\":[1,2],\"category\":\"reference\",\"title\":\"Sayings of the Century\",\"author\":\"Nigel Rees\",\"price\":8.95},{\"ref\":[1,3,5],\"category\":\"fiction\",\"title\":\"Moby Dick\",\"author\":\"Herman Melville\",\"price\":8.99,\"isbn\":\"0-553-21311-3\"}]" ] }, "execution_count": 135, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// comparing with another element\n", "jsonPath = \"$..book[?(@.price < $.expensive)]\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "code", "execution_count": 136, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{\"ref\":[1,3,5],\"category\":\"fiction\",\"title\":\"Moby Dick\",\"author\":\"Herman Melville\",\"price\":8.99,\"isbn\":\"0-553-21311-3\"}]" ] }, "execution_count": 136, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// composable conditions: books in fiction category with isbn and priced less than 10\n", "jsonPath = \"$..book[?(@.category == 'fiction' && @.isbn && @.price < 10 )]\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Try Your Own Path" ] }, { "cell_type": "code", "execution_count": 137, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{expensive=10, store={bicycle={color=red, price=19.95}, book=[{ref=[1, 2], category=reference, title=Sayings of the Century, author=Nigel Rees, price=8.95}, {ref=[2, 4, 16], category=fiction, title=Sword of Honour, author=Evelyn Waugh, price=12.99}, {ref=[1, 3, 5], category=fiction, title=Moby Dick, author=Herman Melville, price=8.99, isbn=0-553-21311-3}, {ref=[1, 2, 7], category=fiction, title=The Lord of the Rings, author=J. R. R. Tolkien, price=22.99, isbn=0-395-19395-8}]}}" ] }, "execution_count": 137, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// substitute your json path\n", "jsonPath = \"$\";\n", "documentClient.get(documentKey, documentBinName, jsonPath );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Cleaning Up\n", "Remove tutorial data and close connection." ] }, { "cell_type": "code", "execution_count": 138, "metadata": {}, "outputs": [], "source": [ "//truncateTestData();\n", "//aerospikeClient.close();\n", "//System.out.println(\"Removed tutorial data and closed server connection.\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Further Exploration and Resources\n", "Here are some links for further exploration.\n", "\n", "## Resources\n", "- Blog posts\n", " - [Aerospike Document API](https://medium.com/aerospike-developer-blog/aerospike-document-api-fd8870b4106c)\n", " - [Aerospike Document API: JSONPath Queries](https://medium.com/aerospike-developer-blog/aerospike-document-api-jsonpath-queries-bd6260b2d076)\n", "- Related notebooks\n", " - [Working with Lists](java-working_with_lists.ipynb)\n", " - [Working with Maps](java-working_with_maps.ipynb)\n", " - [Implementing SQL Operations: SELECT](sql_select.ipynb), \n", " - [Implementing SQL Operations: CREATE, UPDATE, DELETE](sql_updates.ipynb)\n", "- Aerospike Developer Hub\n", " - [Java Developers Resources](https://developer.aerospike.com/java-developers)\n", "- Github repos\n", " - [Aerospike Document API Library](https://github.com/aerospike/aerospike-document-lib)\n", " - [Jayway JSONPath](https://github.com/json-path/JsonPath)\n", " - [Java code examples](https://github.com/aerospike/aerospike-client-java/tree/master/examples/src/com/aerospike/examples)\n", " - [Java Client](https://www.aerospike.com/docs/client/java/index.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exploring Other Notebooks\n", "\n", "Visit [Aerospike notebooks repo](https://github.com/aerospike/aerospike-dev-notebooks.docker) to run additional Aerospike notebooks. To run a different notebook, download the notebook from the repo to your local machine, and then click on File->Open in the notebook menu, and select Upload." ] } ], "metadata": { "kernelspec": { "display_name": "Java", "language": "java", "name": "java" }, "language_info": { "codemirror_mode": "java", "file_extension": ".jshell", "mimetype": "text/x-java-source", "name": "Java", "pygments_lexer": "java", "version": "11.0.8+10-LTS" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }