{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Aerospike Java Client – Reading and Updating Maps\n", "*Last updated: June 22, 2021*\n", "\n", "This notebook demonstrates Java Aerospike CRUD operations (Create, Read, Update, Delete) for maps of data, focusing on server-side **read** and **update** operations. \n", "\n", "Aerospike stores records by association with a **key**. Maps contain key:value pairs. This notebook makes use of the word **mapkey** to distinguish from a record **key**. \n", "\n", "This [Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html) requires the Aerospike Database running locally with Java kernel and Aerospike Java Client. To create a Docker container that satisfies the requirements and holds a copy of these notebooks, visit the [Aerospike Notebooks Repo](https://github.com/aerospike-examples/interactive-notebooks)." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# Notebook Setup \n", "\n", "Run these first to initialize Jupyter, download the Java Client, and make sure the Aerospike Database is running." ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Import Jupyter Java Integration \n", "\n", "Make it easier to work with Java in Jupyter." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "hidden": true }, "outputs": [], "source": [ "import io.github.spencerpark.ijava.IJava;\n", "import io.github.spencerpark.jupyter.kernel.magic.common.Shell;\n", "\n", "IJava.getKernelInstance().getMagics().registerMagics(Shell.class);" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Start Aerospike\n", "\n", "Ensure Aerospike Database is running locally." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "hidden": true }, "outputs": [], "source": [ "%sh asd" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Download the Aerospike Java Client\n", "\n", "Ask Maven to download and install the project object model (POM) of the Aerospike Java Client." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "hidden": true }, "outputs": [], "source": [ "%%loadFromPOM\n", "\n", " \n", " com.aerospike\n", " aerospike-client\n", " 5.0.0\n", " \n", "" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Start the Aerospike Java Client and Connect\n", "\n", "Create an instance of the Aerospike Java Client, and connect to the demo cluster.\n", "\n", "The default cluster location for the Docker container is *localhost* port *3000*. If your cluster is not running on your local machine, modify *localhost* and *3000* to the values for your Aerospike cluster." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Initialized the client and connected to the cluster.\n" ] } ], "source": [ "import com.aerospike.client.AerospikeClient;\n", "\n", "AerospikeClient client = new AerospikeClient(\"localhost\", 3000);\n", "System.out.println(\"Initialized the client and connected to the cluster.\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# CREATING Maps in Aerospike" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create and Print Map Data\n", "\n", "Create a string map representing fish metadata. Create an integer map containing timestamped fish observation locations." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fish Map: {color=Aquamarine, fruit=Pineapple, name=Annette, tree=Redwood}\n", "Observations Map:{13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 12345={lat=-85, long=-130}}\n" ] } ], "source": [ "import java.util.ArrayList;\n", "import java.util.Arrays;\n", "import java.util.HashMap;\n", "import java.util.List;\n", "import java.util.Map;\n", "\n", "HashMap mapFish = new HashMap ();\n", "mapFish.put(\"name\", \"Annette\");\n", "mapFish.put(\"fruit\", \"Pineapple\");\n", "mapFish.put(\"color\", \"Aquamarine\");\n", "mapFish.put(\"tree\", \"Redwood\");\n", "System.out.println(\"Fish Map: \" + mapFish);\n", "\n", "\n", "HashMap mapObs = new HashMap ();\n", "HashMap mapCoords0 = new HashMap ();\n", "mapCoords0.put(\"lat\", -85);\n", "mapCoords0.put(\"long\", -130);\n", "HashMap mapCoords1 = new HashMap ();\n", "mapCoords1.put(\"lat\", -25);\n", "mapCoords1.put(\"long\", -50);\n", "HashMap mapCoords2 = new HashMap ();\n", "mapCoords2.put(\"lat\", 35);\n", "mapCoords2.put(\"long\", 30);\n", "\n", "mapObs.put(13456, mapCoords1);\n", "mapObs.put(14567, mapCoords2);\n", "mapObs.put(12345, mapCoords0);\n", "System.out.println(\"Observations Map:\" + mapObs);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Insert the Maps into Aerospike\n", "\n", "Insert one record in Aerospike with **Key** \"koi\", and **Bin Names** *mapfishbin* and *mapobsbin*. \n", "\n", "By default, Aerospike data is unsorted, however Aerospike preserves order by index when inserting data. Java HashMaps are sorted by mapkey by default. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a Key Object\n", "\n", "A **Key** uniquely identifies a specific record in your Aerospike server or cluster. Each key must have a **Namespace** and optionally a **Set** name. \n", "* In Aerospike, a **Namespace** is like a relational database's tablespace. \n", "* A **Set** is like a relational database table.\n", "* A **Record** is like a row in a relational database table.\n", "\n", "The namespace *test* is configured on your Aerospike server or cluster. \n", "\n", "For additional information on the Aerospike Data Model, go [here](https://www.aerospike.com/docs/architecture/data-model.html). " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Key created.\n" ] } ], "source": [ "import com.aerospike.client.Key;\n", "\n", "String mapSet = \"mapset1\";\n", "String mapNamespace = \"test\";\n", "\n", "String theKey = \"koi\";\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "System.out.println(\"Key created.\" );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a Bin Object for Each Map\n", "\n", "A **Bin** is a data field in an Aerospike record." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created mapfishbin:{color=Aquamarine, fruit=Pineapple, name=Annette, tree=Redwood} and mapobsbin:{13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 12345={lat=-85, long=-130}}.\n" ] } ], "source": [ "import com.aerospike.client.Bin;\n", "\n", "String mapFishBinName = \"mapfishbin\";\n", "String mapObsBinName = \"mapobsbin\";\n", "\n", "Bin bin1 = new Bin(mapFishBinName, mapFish);\n", "Bin bin2 = new Bin(mapObsBinName, mapObs);\n", "\n", "System.out.println( \"Created \" + bin1 + \" and \" + bin2 + \".\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a Policy Object for Record Insertion \n", "\n", "A **Policy** tells Aerospike the intent of a database operation. \n", "\n", "For more information on policies, go [here](https://www.aerospike.com/docs/guide/policies.html)." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created a client policy.\n" ] } ], "source": [ "import com.aerospike.client.policy.ClientPolicy;\n", "\n", "ClientPolicy clientPolicy = new ClientPolicy();\n", "System.out.println(\"Created a client policy.\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Put the Map Data into Aerospike" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Key: koi\n", "mapfishbin: {color=Aquamarine, fruit=Pineapple, name=Annette, tree=Redwood}\n", "mapobsbin: {13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 12345={lat=-85, long=-130}}\n" ] } ], "source": [ "client.put(clientPolicy.writePolicyDefault, key, bin1, bin2);\n", "System.out.println(\"Key: \" + theKey + \"\\n\" + mapFishBinName + \": \" + mapFish + \"\\n\" + \n", " mapObsBinName + \": \" + mapObs );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# READING Maps and Map Elements From the Server\n", "\n", "Now that the maps are in Aerospike, the client can return full or partial maps from **bin** contents. No data is modified by these ops." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get the Record\n", "\n", "A record can be retrieved using the **key**, **namespace**, and **set** name.\n", "\n", "In the output: \n", "* **gen** is the generation number, the number of record writes. \n", "* **exp** is the expiration counter for the record.\n", "\n", "For more information on [both generation number and expiration](https://www.aerospike.com/docs/guide/FAQ.html), see the [Aerospike FAQ](https://www.aerospike.com/docs/guide/FAQ.html)." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(gen:1),(exp:359404034),(bins:(mapfishbin:{name=Annette, tree=Redwood, color=Aquamarine, fruit=Pineapple}),(mapobsbin:{13456={lat=-25, long=-50}, 12345={lat=-85, long=-130}, 14567={lat=35, long=30}}))\n" ] } ], "source": [ "import com.aerospike.client.Record;\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "System.out.println(record);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get String Elements by Mapkey, Rank, and Value\n", "\n", "Aerospike provides **MapOperations** to read string mapkeys and values from the database. \n", "\n", "The mapFishBin is a map containing string mapkey/value pairs associated with the fish, \"Koi\".\n", "\n", "For more information on map operations, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapOperation.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get String by Mapkey\n", "\n", "Aerospike API can be used to look up a value by mapkey. The client returns the specified value as the contents of the bin. \n", "\n", "For the list of return type options, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapReturnType.html). " ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The string map: {name=Annette, tree=Redwood, color=Aquamarine, fruit=Pineapple}\n", "The color in the string map is: Aquamarine\n" ] } ], "source": [ "import com.aerospike.client.Operation;\n", "import com.aerospike.client.Value;\n", "import com.aerospike.client.cdt.MapOperation;\n", "import com.aerospike.client.cdt.MapReturnType;\n", "\n", "String mapKeyToFind = \"color\";\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record colorString = client.operate(null, key, \n", " MapOperation.getByKey(mapFishBinName, Value.get(mapKeyToFind), MapReturnType.VALUE)\n", " );\n", "\n", "System.out.println(\"The string map: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"The \" + mapKeyToFind + \" in the string map is: \" + colorString.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get Highest Rank String\n", "\n", "Aerospike's API contains operations to look up a map element by rank. \n", "\n", "For information on list ranking, go [here](https://en.wikipedia.org/wiki/List_ranking)." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The string map: {name=Annette, tree=Redwood, color=Aquamarine, fruit=Pineapple}\n", "The highest rank string is: Redwood\n" ] } ], "source": [ "Integer highestRank = -1;\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record highestRankString = client.operate(null, key, \n", " MapOperation.getByRank(mapFishBinName, highestRank, MapReturnType.VALUE)\n", " );\n", "\n", "System.out.println(\"The string map: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"The highest rank string is: \" + highestRankString.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get Mapkey By String Value\n", "\n", "Aerospike provides operations to look up an element by value and return the mapkey. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The string map: {name=Annette, tree=Redwood, color=Aquamarine, fruit=Pineapple}\n", "The mapkey associated with Pineapple is: [fruit]\n" ] } ], "source": [ "String valueToFind = \"Pineapple\";\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record foundMapKey = client.operate(null, key, \n", " MapOperation.getByValue(mapFishBinName, Value.get(valueToFind), MapReturnType.KEY)\n", " );\n", "\n", "System.out.println(\"The string map: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"The mapkey associated with \" + valueToFind + \" is: \" + foundMapKey.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get Map Size and Integer Elements by Index and Key Range\n", "\n", "Aerospike operations can read integers associated with fish observations.\n", "\n", "The mapobsbin is a list of Latitude/Longitude pairs stored by the time of fish observation in seconds from the start of the experiment. The number of seconds, latitude, and longitude are all integers." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get the Number of Observations in the Map\n", "\n", "Aerospike API's size operation returns a count of the mapkeys in a map." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The Observation Map: {13456={lat=-25, long=-50}, 12345={lat=-85, long=-130}, 14567={lat=35, long=30}}\n", "The number of Observations in the Map: 3\n" ] } ], "source": [ "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record sizeString = client.operate(null, key, \n", " MapOperation.size(mapObsBinName)\n", " );\n", "\n", "System.out.println(\"The Observation Map: \" + record.getValue(mapObsBinName));\n", "System.out.println(\"The number of Observations in the Map: \" \n", " + sizeString.getValue(mapObsBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get The First Observation from the Map \n", "\n", "Aerospike API operations can look up a value by index. In Aerospike, the index operation can get one or more map elements by key order. Aerospike allows indexing forward from the beginning of the map using zero-based numbering. Negative numbers index backwards from the end of a map. \n", "\n", "\n", "In this example, the first element by index represents the first time the fish was observed. Because the key 12345 is before 13456 and 14567, the first element by index is 12345.\n", "\n", "\n", "For examples of indexes, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapOperation.html)." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The Observation Map: {13456={lat=-25, long=-50}, 12345={lat=-85, long=-130}, 14567={lat=35, long=30}}\n", "The First Observation: [12345={lat=-85, long=-130}]\n" ] } ], "source": [ "Integer firstIdx = 0;\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record firstObservation = client.operate(null, key, \n", " MapOperation.getByIndex(mapObsBinName, firstIdx, MapReturnType.KEY_VALUE)\n", " );\n", "\n", "System.out.println(\"The Observation Map: \" + record.getValue(mapObsBinName));\n", "System.out.println(\"The First Observation: \" + firstObservation.getValue(mapObsBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get All Locations Observed Between 13,000 and 15,000 seconds.\n", "\n", "Aerospike delivers values by mapkey range. Get the latitude and longitude pairs for all observations between 13,000 and 15,000 seconds. " ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The Observation Map: {13456={lat=-25, long=-50}, 12345={lat=-85, long=-130}, 14567={lat=35, long=30}}\n", "The Observations between 13000 and 15000 seconds: [13456={lat=-25, long=-50}, 14567={lat=35, long=30}]\n" ] } ], "source": [ "Integer lowerBound = 13000;\n", "Integer upperBound = 15000;\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record rangeObservations = client.operate(null, key, \n", " MapOperation.getByKeyRange(mapObsBinName, Value.get(lowerBound), Value.get(upperBound), \n", " MapReturnType.KEY_VALUE)\n", " );\n", "\n", "System.out.println(\"The Observation Map: \" + record.getValue(mapObsBinName));\n", "System.out.println(\"The Observations between 13000 and 15000 seconds: \" \n", " + rangeObservations.getValue(mapObsBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# UPDATING Maps on the Aerospike Server\n", "\n", "Aerospike's **MapOperations** can also modify data in the Aerospike Database." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Update the Fish Bin in Aerospike\n", "\n", "The Fish Bin contains metadata about the fish." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a MapPolicy Java Object for the Fish Bin\n", "\n", "When modifying maps, Aerospike requires a **MapPolicy** that governs write protection and order. The default MapPolicy works for Fish Bin.\n", "\n", "\n", "For more information on mappolicy, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapPolicy.html)." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created default MapPolicy for mapfishbin.\n" ] } ], "source": [ "import com.aerospike.client.cdt.MapPolicy;\n", "\n", "MapPolicy mapFishBinPolicy = new MapPolicy();\n", "\n", "System.out.println(\"Created default MapPolicy for \" + mapFishBinName + \".\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Change the Tree to Larch\n", "When new data is put into a map, Aerospike returns the size of the resulting map. " ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before: {name=Annette, tree=Redwood, color=Aquamarine, fruit=Pineapple}\n", "The size after the operation: 4\n", " After: {name=Annette, tree=Larch, color=Aquamarine, fruit=Pineapple}\n" ] } ], "source": [ "String treeMapkeyName = \"tree\";\n", "String newTree = \"Larch\";\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record sizeOfMapWithNewTree = client.operate(null, key, \n", " MapOperation.put(mapFishBinPolicy, mapFishBinName, Value.get(treeMapkeyName), \n", " Value.get(newTree))\n", " );\n", "Record mapWithNewTree = client.get(null, key);\n", "\n", "System.out.println(\"Before: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"The size after the operation: \" \n", " + sizeOfMapWithNewTree.getValue(mapFishBinName));\n", "System.out.println(\" After: \" + mapWithNewTree.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Remove the Fruit\n", "\n", "When removing a mapkey:value pair, Aerospike client returns the removed data." ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before: {name=Annette, tree=Larch, color=Aquamarine, fruit=Pineapple}\n", "The removed mapkey/value pair: [fruit=Pineapple]\n", "After removing the fruit: {color=Aquamarine, name=Annette, tree=Larch}\n" ] } ], "source": [ "String fruitMapkeyName = \"fruit\";\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record valOfRemovedFruit = client.operate(null, key, \n", " MapOperation.removeByKey(mapFishBinName, Value.get(fruitMapkeyName), \n", " MapReturnType.KEY_VALUE)\n", " );\n", "Record mapWithoutFruit = client.get(null, key);\n", "\n", "System.out.println(\"Before: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"The removed mapkey/value pair: \" \n", " + valOfRemovedFruit.getValue(mapFishBinName));\n", "System.out.println(\"After removing the \" + fruitMapkeyName + \": \" \n", " + mapWithoutFruit.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Add Bait\n", "\n", "To be sure that other scientists can catch the fish, add the fish's preferrred bait to the record." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before: {color=Aquamarine, name=Annette, tree=Larch}\n", "After adding Bait: {name=Annette, tree=Larch, color=Aquamarine, bait=Mosquito Larva}\n" ] } ], "source": [ "String mapkeyForBait = \"bait\";\n", "String valueForBait = \"Mosquito Larva\";\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record sizeOfRecordWithBait = client.operate(null, key, \n", " MapOperation.put(mapFishBinPolicy, mapFishBinName, Value.get(mapkeyForBait), \n", " Value.get(valueForBait))\n", " );\n", "Record recordWithBait = client.get(null, key);\n", "\n", "System.out.println(\"Before: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"After adding Bait: \" + recordWithBait.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Put an Observation Counter in the Map \n", "\n", "The experiment continued past the original end date. The new work requires keeping track of the total number of observations. " ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before: {name=Annette, tree=Larch, color=Aquamarine, bait=Mosquito Larva}\n", "After Adding the Counter: {name=Annette, tree=Larch, Count=3, color=Aquamarine, bait=Mosquito Larva}\n" ] } ], "source": [ "String mapkeyObsCount = \"Count\";\n", "Integer numObservations = 3;\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record sizeOfRecordWithObsCounter = \n", " client.operate(null, key, MapOperation.put(mapFishBinPolicy, mapFishBinName, \n", " Value.get(mapkeyObsCount), \n", " Value.get(numObservations))\n", " );\n", "Record recordWithObsCount = client.get(null, key);\n", "\n", "System.out.println(\"Before: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"After Adding the Counter: \" + recordWithObsCount.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Update the Observation Map\n", "Aerospike client can update map elements, such as integers and sub-maps.\n", "\n", "The experiment continued past the original end date. The new work requires the regular addition of new observations and keeping track of the total number of observations." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a MapPolicy Object for the Observations Bin\n", "\n", "In this example, the Observations Map should be maintained as mapkey-sorted in Aerospike, but are put unordered into the database by default. When storing any map on SSD hardware, Key Ordered Maps hold a significant performance advantage over Unordered Maps, at a cost of 4 bytes of storage for metadata.\n", "\n", "The MapPolicy contains two types of configurations, **MapOrder** and **MapWriteFlags**. The maporder determines the sort order of the map. The mapwriteflags determine write behaviors, such as if the operation should fail when a mapkey/value already exists. \n", "\n", "For more information on maporder, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapOrder.html).\n", "\n", "For more information on mapwriteflags, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapWriteFlags.html)." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before Sorting: {13456={lat=-25, long=-50}, 12345={lat=-85, long=-130}, 14567={lat=35, long=30}}\n", "Applied mapkey-ordered MapPolicy for mapobsbin.\n", "After Sorting: {12345={lat=-85, long=-130}, 13456={lat=-25, long=-50}, 14567={lat=35, long=30}}\n" ] } ], "source": [ "import com.aerospike.client.cdt.MapOrder;\n", "import com.aerospike.client.cdt.MapWriteFlags;\n", "\n", "Record recordObsUnordered = client.get(null, key);\n", "MapPolicy mapObsBinPolicy = new MapPolicy(MapOrder.KEY_ORDERED, MapWriteFlags.DEFAULT);\n", "Record changeOrder = \n", " client.operate(null, key, MapOperation.setMapPolicy(mapObsBinPolicy, mapObsBinName));\n", "Record recordObsOrdered = client.get(null, key);\n", "\n", "System.out.println(\"Before Sorting: \" + recordObsUnordered.getValue(mapObsBinName));\n", "System.out.println(\"Applied mapkey-ordered MapPolicy for \" + mapObsBinName + \".\");\n", "System.out.println(\"After Sorting: \" + recordObsOrdered.getValue(mapObsBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Add a new Observation" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before: {12345={lat=-85, long=-130}, 13456={lat=-25, long=-50}, 14567={lat=35, long=30}}\n", "The Size After Adding the Observation: 4\n", "After Adding the Observation: {12345={lat=-85, long=-130}, 13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 15678={lat=80, long=110}}\n" ] } ], "source": [ "int newObsTimestamp = 15678;\n", "int newObsLat = 80;\n", "int newObsLong = 110;\n", "\n", "HashMap mapNewObs = new HashMap ();\n", "HashMap mapNewCoords = new HashMap ();\n", "mapNewCoords.put(\"lat\", newObsLat);\n", "mapNewCoords.put(\"long\", newObsLong);\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record sizeOfNewObs = client.operate(null, key, \n", " MapOperation.put(mapObsBinPolicy, mapObsBinName, Value.get(newObsTimestamp), Value.get(mapNewCoords))\n", " );\n", "Record recordWithNewObs = client.get(null, key);\n", "\n", "System.out.println(\"Before: \" + record.getValue(mapObsBinName));\n", "System.out.println(\"The Size After Adding the Observation: \" + sizeOfNewObs.getValue(mapObsBinName));\n", "System.out.println(\"After Adding the Observation: \" + recordWithNewObs.getValue(mapObsBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Remove the Oldest Observation by Index\n", "\n", "This study only maintains the three most recent observations." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before: {12345={lat=-85, long=-130}, 13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 15678={lat=80, long=110}}\n", "The Removed Observation: [12345={lat=-85, long=-130}]\n", "After Observation Removal: {13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 15678={lat=80, long=110}}\n" ] } ], "source": [ "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record oldObs = client.operate(null, key, \n", " MapOperation.removeByIndex(mapObsBinName, firstIdx, MapReturnType.KEY_VALUE)\n", " );\n", "Record updatedRecord = client.get(null, key);\n", "\n", "System.out.println(\"Before: \" + record.getValue(mapObsBinName));\n", "System.out.println(\"The Removed Observation: \" + oldObs.getValue(mapObsBinName));\n", "System.out.println(\"After Observation Removal: \" + updatedRecord.getValue(mapObsBinName));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Increment the Observation Counter\n", "\n", "When incrementing a map value, Aerospike returns the new value." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before: {name=Annette, tree=Larch, Count=3, color=Aquamarine, bait=Mosquito Larva}\n", "The New Count: 4\n", "After Increment: {name=Annette, tree=Larch, Count=4, color=Aquamarine, bait=Mosquito Larva}\n" ] } ], "source": [ "int incNum = 1;\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record obsCount = client.operate(null, key, \n", " MapOperation.increment(mapFishBinPolicy, mapFishBinName, Value.get(mapkeyObsCount), Value.get(incNum))\n", " );\n", "Record updatedRecord = client.get(null, key);\n", "\n", "System.out.println(\"Before: \" + record.getValue(mapFishBinName));\n", "System.out.println(\"The New Count: \" + obsCount.getValue(mapFishBinName));\n", "System.out.println(\"After Increment: \" + updatedRecord.getValue(mapFishBinName));" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# Notebook Cleanup" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Truncate the Set\n", "Truncate the set from the Aerospike Database." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Set Truncated.\n" ] } ], "source": [ "import com.aerospike.client.policy.InfoPolicy;\n", "InfoPolicy infoPolicy = new InfoPolicy();\n", "\n", "client.truncate(infoPolicy, mapNamespace, mapSet, null);\n", "System.out.println(\"Set Truncated.\");" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Close the Client connections to Aerospike" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Server connection(s) closed.\n" ] } ], "source": [ "client.close();\n", "System.out.println(\"Server connection(s) closed.\");" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# Code Summary" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Overview\n", "Here is a collection of all of the non-Jupyter code from this tutorial.\n", "1. Import Java Libraries.\n", "2. Import Aerospike Client Libraries.\n", "3. Start the Aerospike Client.\n", "4. Create Test Data.\n", "5. Put Record into Aerospike.\n", "6. Get Data from Aerospike.\n", " 1. Get the Record.\n", " 2. Get String by MapKey and Highest Rank.\n", " 3. Get MapKey by String.\n", " 3. Get the Number of Observations and 1st Observation By Index.\n", " 4. Get Observations by MapKey Range. \n", "7. Update the Record in Aerospike\n", " 1. Change the Tree to a Larch\n", " 2. Remove the Fruit and add Bait.\n", " 3. Sort the Observation Map.\n", " 4. Add an Observation Counter.\n", " 5. Add a New Observation.\n", " 6. Remove the Oldest Operation.\n", " 7. Increment the Observation Counter.\n", "8. Truncate the Set.\n", "9. Close the Client Connections." ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Initialized the client and connected to the cluster.\n", "Created Fish Map: {color=Aquamarine, fruit=Pineapple, name=Annette, tree=Redwood}\n", "Created Observations Map: {13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 12345={lat=-85, long=-130}}\n", "Inserted Key: koi\n", " mapfishbin: {color=Aquamarine, fruit=Pineapple, name=Annette, tree=Redwood}\n", " mapobsbin: {13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 12345={lat=-85, long=-130}}\n", "\n", "Read the Full Record From Aerospike:(gen:1),(exp:359404037),(bins:(mapfishbin:{name=Annette, tree=Redwood, color=Aquamarine, fruit=Pineapple}),(mapobsbin:{13456={lat=-25, long=-50}, 12345={lat=-85, long=-130}, 14567={lat=35, long=30}}))\n", "The color in the string map is: Aquamarine\n", "The highest rank string is: Redwood\n", "The mapkey associated with Pineapple is: [fruit]\n", "The number of Observations in the Map: 3\n", "The First Observation: [12345={lat=-85, long=-130}]\n", "The Observations between 13000 and 15000 seconds: [13456={lat=-25, long=-50}, 14567={lat=35, long=30}]\n", "\n", "Changed tree to Larch; there are now 4 map items in mapfishbin\n", "Removed item [fruit=Pineapple]\n", "Added item [bait=Mosquito Larva]; there are now 4 map items in mapfishbin\n", "Added Observation Counter; there are now 5 map items in mapfishbin\n", "Sorted mapobsbin\n", "Added New Observation {15678={lat=80, long=110}}, there are now 4 map items in mapobsbin\n", "Removed Oldest Observation: [12345={lat=-85, long=-130}]\n", "Incremented Observation Counter to reflect 4th observation\n", "\n", "After Record Edits: (gen:2),(exp:359404038),(bins:(mapfishbin:{name=Annette, tree=Larch, Count=4, color=Aquamarine, bait=Mosquito Larva}),(mapobsbin:{13456={lat=-25, long=-50}, 14567={lat=35, long=30}, 15678={lat=80, long=110}}))\n", "Set Truncated.\n" ] } ], "source": [ "// Import Java Libraries.\n", "\n", "import java.util.ArrayList;\n", "import java.util.Arrays;\n", "import java.util.HashMap;\n", "import java.util.List;\n", "import java.util.Map;\n", "\n", "\n", "// Import Aerospike Client Libraries.\n", "\n", "import com.aerospike.client.AerospikeClient;\n", "import com.aerospike.client.Key;\n", "import com.aerospike.client.Bin;\n", "import com.aerospike.client.policy.ClientPolicy;\n", "import com.aerospike.client.Record;\n", "import com.aerospike.client.Operation;\n", "import com.aerospike.client.Value;\n", "import com.aerospike.client.cdt.MapOperation;\n", "import com.aerospike.client.cdt.MapReturnType;\n", "import com.aerospike.client.cdt.MapPolicy;\n", "import com.aerospike.client.cdt.MapOrder;\n", "import com.aerospike.client.cdt.MapWriteFlags;\n", "import com.aerospike.client.policy.InfoPolicy;\n", "\n", "\n", "// Start the Aerospike Client.\n", "\n", "AerospikeClient client = new AerospikeClient(\"localhost\", 3000);\n", "System.out.println(\"Initialized the client and connected to the cluster.\");\n", "\n", "\n", "// Create Test Data.\n", "\n", "HashMap mapFish = new HashMap ();\n", "mapFish.put(\"name\", \"Annette\");\n", "mapFish.put(\"fruit\", \"Pineapple\");\n", "mapFish.put(\"color\", \"Aquamarine\");\n", "mapFish.put(\"tree\", \"Redwood\");\n", "System.out.println(\"Created Fish Map: \" + mapFish);\n", "\n", "HashMap mapObs = new HashMap ();\n", "HashMap mapCoords0 = new HashMap ();\n", "mapCoords0.put(\"lat\", -85);\n", "mapCoords0.put(\"long\", -130);\n", "HashMap mapCoords1 = new HashMap ();\n", "mapCoords1.put(\"lat\", -25);\n", "mapCoords1.put(\"long\", -50);\n", "HashMap mapCoords2 = new HashMap ();\n", "mapCoords2.put(\"lat\", 35);\n", "mapCoords2.put(\"long\", 30);\n", "\n", "mapObs.put(13456, mapCoords1);\n", "mapObs.put(14567, mapCoords2);\n", "mapObs.put(12345, mapCoords0);\n", "System.out.println(\"Created Observations Map: \" + mapObs);\n", "\n", "\n", "// Put Record into Aerospike.\n", "\n", "String mapSet = \"mapset1\";\n", "String mapNamespace = \"test\";\n", "String theKey = \"koi\";\n", "String mapFishBin = \"mapfishbin\";\n", "String mapObsBin = \"mapobsbin\";\n", "ClientPolicy clientPolicy = new ClientPolicy();\n", "InfoPolicy infoPolicy = new InfoPolicy();\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Bin bin1 = new Bin(mapFishBin, mapFish);\n", "Bin bin2 = new Bin(mapObsBin, mapObs);\n", "client.put(clientPolicy.writePolicyDefault, key, bin1, bin2);\n", "\n", "System.out.println(\"Inserted Key: \" + theKey + \"\\n \" + mapFishBin + \": \" + mapFish + \"\\n \" + \n", " mapObsBin + \": \" + mapObs );\n", "System.out.println();\n", "\n", "\n", "// Get Data from Aerospike.\n", "// 1. Get the Record.\n", "// 2. Get String by MapKey and Highest Rank.\n", "// 3. Get MapKey by String.\n", "// 3. Get the Number of Observations and 1st Observation By Index.\n", "// 4. Get Observations by MapKey Range.\n", "\n", "String mapKeyToFind = \"color\";\n", "Integer highestRank = -1;\n", "String valueToFind = \"Pineapple\";\n", "Integer firstIdx = 0;\n", "Integer lowerBound = 13000;\n", "Integer upperBound = 15000;\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record results = client.operate(null, key, \n", " MapOperation.getByKey(mapFishBin, Value.get(mapKeyToFind), MapReturnType.VALUE),\n", " MapOperation.getByRank(mapFishBin, highestRank, MapReturnType.VALUE),\n", " MapOperation.getByValue(mapFishBin, Value.get(valueToFind), MapReturnType.KEY),\n", " MapOperation.size(mapObsBin),\n", " MapOperation.getByIndex(mapObsBin, firstIdx, MapReturnType.KEY_VALUE),\n", " MapOperation.getByKeyRange(mapObsBin, Value.get(lowerBound), Value.get(upperBound), MapReturnType.KEY_VALUE)\n", " );\n", "\n", "List resultsFish = results.getList(mapFishBin);\n", "List resultsObs = results.getList(mapObsBin);\n", "\n", "System.out.println(\"Read the Full Record From Aerospike:\" + record);\n", "System.out.println(\"The \" + mapKeyToFind + \" in the string map is: \" + resultsFish.get(0));\n", "System.out.println(\"The highest rank string is: \" + resultsFish.get(1));\n", "System.out.println(\"The mapkey associated with \" + valueToFind + \" is: \" + resultsFish.get(2));\n", "System.out.println(\"The number of Observations in the Map: \" + resultsObs.get(0));\n", "System.out.println(\"The First Observation: \" + resultsObs.get(1));\n", "System.out.println(\"The Observations between 13000 and 15000 seconds: \" + resultsObs.get(2));\n", "System.out.println();\n", "\n", "\n", "// 7. Update the Record in Aerospike\n", "// 1. Change the Tree to a Larch\n", "// 2. Remove the Fruit and add Bait.\n", "// 3. Add an Observation Counter.\n", "// 4. Sort the Observation Map.\n", "// 5. Add a New Observation.\n", "// 6. Remove the Oldest Operation.\n", "// 7. Increment the Observation Counter.\n", "\n", "MapPolicy mapFishBinPolicy = new MapPolicy();\n", "MapPolicy mapObsBinPolicy = new MapPolicy(MapOrder.KEY_ORDERED, MapWriteFlags.DEFAULT);\n", "\n", "String treeMapkeyName = \"tree\";\n", "String newTree = \"Larch\";\n", "String fruitMapkeyName = \"fruit\";\n", "String mapkeyForBait = \"bait\";\n", "String valueForBait = \"Mosquito Larva\";\n", "String mapkeyObsCount = \"Count\";\n", "Integer numObservations = 3;\n", "int newObsTimestamp = 15678;\n", "int newObsLat = 80;\n", "int newObsLong = 110;\n", "int incNum = 1;\n", "\n", "HashMap mapNewObs = new HashMap ();\n", "HashMap mapNewCoords = new HashMap ();\n", "mapNewCoords.put(\"lat\", newObsLat);\n", "mapNewCoords.put(\"long\", newObsLong);\n", "\n", "Key key = new Key(mapNamespace, mapSet, theKey);\n", "Record record = client.get(null, key);\n", "Record updatingRecord = client.operate(null, key, \n", " MapOperation.put(mapFishBinPolicy, mapFishBin, Value.get(treeMapkeyName), \n", " Value.get(newTree)),\n", " MapOperation.removeByKey(mapFishBin, Value.get(fruitMapkeyName), \n", " MapReturnType.KEY_VALUE),\n", " MapOperation.put(mapFishBinPolicy, mapFishBin, Value.get(mapkeyForBait), \n", " Value.get(valueForBait)),\n", " MapOperation.put(mapFishBinPolicy, mapFishBin, Value.get(mapkeyObsCount), \n", " Value.get(numObservations)),\n", " MapOperation.setMapPolicy(mapObsBinPolicy, mapObsBin),\n", " MapOperation.put(mapObsBinPolicy, mapObsBin, Value.get(newObsTimestamp), \n", " Value.get(mapNewCoords)),\n", " MapOperation.removeByIndex(mapObsBin, firstIdx, MapReturnType.KEY_VALUE),\n", " MapOperation.increment(mapFishBinPolicy, mapFishBin, Value.get(mapkeyObsCount), \n", " Value.get(incNum))\n", " );\n", "Record finalRecord = client.get(null, key);\n", "\n", "List updateFish = updatingRecord.getList(mapFishBin);\n", "List updateObs = updatingRecord.getList(mapObsBin);\n", "\n", "System.out.println(\"Changed \" + treeMapkeyName + \" to \" + newTree + \"; there are now \" + updateFish.get(0) + \" map items in \" + mapFishBin);\n", "System.out.println(\"Removed item \" + updateFish.get(1));\n", "System.out.println(\"Added item [\" + mapkeyForBait + \"=\" + valueForBait + \"]; there are now \" + updateFish.get(2) + \" map items in \" + mapFishBin);\n", "System.out.println(\"Added Observation Counter; there are now \" + updateFish.get(3) + \" map items in \" + mapFishBin);\n", "System.out.println(\"Sorted \" + mapObsBin);\n", "System.out.println(\"Added New Observation {\" + newObsTimestamp + \"=\" + mapNewCoords + \"}, there are now \" + updateObs.get(1) + \" map items in \" + mapObsBin);\n", "System.out.println(\"Removed Oldest Observation: \" + updateObs.get(2));\n", "System.out.println(\"Incremented Observation Counter to reflect \" + updateFish.get(4) + \"th observation\");\n", "System.out.println();\n", "System.out.println(\"After Record Edits: \" + finalRecord);\n", "\n", "\n", "// Truncate the Set.\n", "\n", "client.truncate(infoPolicy, mapNamespace, mapSet, null);\n", "System.out.println(\"Set Truncated.\");\n", "\n", "\n", "// Close the Client Connections.\n", "\n", "client.close();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Takeaway – Aerospike Does Maps\n", "\n", "Aerospike and its Java Client are up to the task of working with your map data. Its API provides rich operations to read and update list data using index, mapkey, value, and rank. Not modeled in this tutorial, Aerospike map operation also supports nested lists and maps, by assigning **CTX** or contexts to operations.\n", "\n", "For more information on contexts, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/CTX.html). For examples of contexts, go [here](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapOperation.html). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# What's Next?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Next Steps\n", "\n", "Have questions? Don't hesitate to reach out if you have additional questions about working with lists at https://discuss.aerospike.com/.\n", "\n", "Want to check out other Java notebooks?\n", "1. [Hello, World](hello_world.ipynb)\n", "2. [Reading and Updating Lists](java-working_with_lists.ipynb)\n", "3. [Modeling Using Lists](java-modeling__using_lists.ipynb) \n", "4. [Aerospike Query and UDF](query_udf.ipynb)\n", "\n", "Are you running this from Binder? [Download the Aerospike Notebook Repo](https://github.com/aerospike-examples/interactive-notebooks) and work with Aerospike Database and Jupyter locally using a Docker container." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Additional Resources\n", "\n", "* Want to get started with Java? [Download](https://www.aerospike.com/download/client/) or [install](https://github.com/aerospike/aerospike-client-java) the Aerospike Java Client. \n", "* What other ways can we work with Maps? Take a look at [Aerospike's Map Operations](https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapOperation.html).\n", "* What are Namespaces, Sets, and Bins? Check out the [Aerospike Data Model](https://www.aerospike.com/docs/architecture/data-model.html). \n", "* How robust is the Aerospike Database? Browses the [Aerospike Database Architecture](https://www.aerospike.com/docs/architecture/index.html)." ] } ], "metadata": { "kernelspec": { "display_name": "Java", "language": "java", "name": "java" }, "language_info": { "codemirror_mode": "java", "file_extension": ".jshell", "mimetype": "text/x-java-source", "name": "Java", "pygments_lexer": "java", "version": "11.0.8+10-LTS" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "240px" }, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 4 }