{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"toc": true
},
"source": [
"Table of Contents
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Aerospike Basic Operations\n",
"\n",
"Basic CRUD (Create, Read, Update, and Delete) operations in Aerospike, and how multiple operations on a record are performed in a single request.\n",
"
\n",
"This notebook requires Aerospike datbase running on localhost and that python and the Aerospike python client have been installed (`pip install aerospike`). Visit [Aerospike notebooks repo](https://github.com/aerospike-examples/interactive-notebooks) for additional details and the docker container."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Ensure the database is running\n",
"This notebook requires that Aerospike datbase is running."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Aerospike database is running!\r\n"
]
}
],
"source": [
"!asd >& /dev/null\n",
"!pgrep -x asd >/dev/null && echo \"Aerospike database is running!\" || echo \"**Aerospike database is not running!**\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Initialize the client\n",
"Initialize the client and connect it to the database."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Client initialized and connected to database\n"
]
}
],
"source": [
"import aerospike\n",
"import sys\n",
"config = {\n",
" 'hosts': [ ('127.0.0.1', 3000) ]\n",
"}\n",
"try:\n",
" client = aerospike.client(config).connect()\n",
"except:\n",
" import sys\n",
" print(\"failed to connect to the cluster with\", config['hosts'])\n",
" sys.exit(1)\n",
"print('Client initialized and connected to database')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Understanding Records in Aerospike\n",
"Data in Aerospike consists of records. A record belongs to a namespace (equivalent to a database) and optionally to a set (equivalent to a table). A record has multiple bins (or fields), which are named, strongly-typed containers that hold both atomic (string, integer, bytes) and complex (map, list) data types.\n",
"\n",
"A record has two metadata values:\n",
"- generation: the number of times it has been modified\n",
"- ttl: seconds remaining until record expiration (default = 0; never expire)\n",
"\n",
"Expired records are garbage-collected by the database. On a write or touch operations, a record's ttl is updated based on the specified policy.\n",
"\n",
"### Record Structure\n",
"The following code cell illustrates the record structure.\n",
"\n",
"Note:\n",
"- key: (namespace, set, user_key, digest), digest is computed from the first three elements.\n",
"- metadata: {'gen': generation, 'ttl': ttl}\n",
"- bins: key-value pairs of data bins dictionary\n",
"\n",
"And also:\n",
"- digest (the bytearray in output) is the actual record unique identifier.\n",
"- user_key input to produce digest may or may not be stored with the record, and is governed by the key policy.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Successfully wrote the record.\n",
"Successfully read the record.\n",
"Key: ('test', 'demo', 'foo', bytearray(b'\\xf5~\\xc1\\x835\\xf7\\x10\\x0c\\x04X\\xf8\\xa6D\\xbc\\xbcvm\\x93G\\x1e'))\n",
"Metadata: {'ttl': 2592000, 'gen': 1}\n",
"Bins: {'name': 'John Doe', 'age': 15, 'gpa': 4.3}\n"
]
}
],
"source": [
"# Record structure\n",
"# key: (namespace, set, user_key, digest), digest is computed from first three values \n",
"# metadata: {'gen': generation, 'ttl': ttl}\n",
"# bins: key-value pairs of data bins dictionary\n",
"\n",
"namespace = 'test'\n",
"demoset = 'demo'\n",
"user_key = 'foo'\n",
"meta = {'ttl': 0}\n",
"bins = {'name': 'John Doe', 'age': 15, 'gpa': 4.3 }\n",
"policy = {'key': aerospike.POLICY_KEY_SEND} # policy to store the user_key along with the record\n",
"\n",
"# insert/update the record\n",
"try:\n",
" client.put((namespace, demoset, user_key), bins, meta, policy)\n",
"except:\n",
" print('failed to put record')\n",
" sys.exit(1)\n",
"print('Successfully wrote the record.')\n",
"\n",
"# read back the record\n",
"try:\n",
" (key, metadata, bins)= client.get((namespace, demoset, user_key), policy)\n",
"except:\n",
" print('failed to get record')\n",
" sys.exit(1)\n",
"print('Successfully read the record.')\n",
"print ('Key: ', key)\n",
"print ('Metadata: ', metadata)\n",
"print ('Bins: ', bins)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Writing Records\n",
"The Python client's \"put\" operations is used to write data to the Aerospike cluster.\n",
"\n",
"### Defining the Key\n",
"As described above, the key tuple serves as the record's unique identifier.\n",
"\n",
"Below we define a key tuple in the set \"characters\" with the user key \"bender\" in namespace \"test\"."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"# create the key tuple identifying the record\n",
"key = ('test', 'characters', 'bender')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Specifying Record Data\n",
"Specify record data in a dict, where the top-level object fields represent the bin names, and field values correspond to bin values.\n",
"\n",
"This example writes six bins: name, serialnum, lastsentence, composition, apartment, and quote_cnt."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# The record data to write to the cluster\n",
"bins = {\n",
" 'name': 'Bender',\n",
" 'serialnum': 2716057,\n",
" 'lastsentence': {\n",
" 'BBS': \"Well, we're boned\",\n",
" 'TBwaBB': 'I love you, meatbags!',\n",
" 'BG': 'Whip harder, Professor!',\n",
" 'ltWGY': 'Into the breach, meatbags. Or not, whatever'},\n",
" 'composition': [ \"40% zinc\", \"40% titanium\", \"30% iron\", \"40% dolomite\" ],\n",
" 'apartment': bytearray(b'\\x24'),\n",
" 'quote_cnt': 47\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Storing the Record\n",
"Store the record in the database with put."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Successfully stored the record.\n",
"Bins: {'name': 'Bender', 'serialnum': 2716057, 'lastsentence': {'ltWGY': 'Into the breach, meatbags. Or not, whatever', 'BG': 'Whip harder, Professor!', 'BBS': \"Well, we're boned\", 'TBwaBB': 'I love you, meatbags!'}, 'composition': ['40% zinc', '40% titanium', '30% iron', '40% dolomite'], 'apartment': bytearray(b'$'), 'quote_cnt': 47}\n"
]
}
],
"source": [
"# Put the record to the database.\n",
"try:\n",
" client.put(key, bins)\n",
"except:\n",
" print('failed to put record')\n",
" sys.exit(1)\n",
"print('Successfully stored the record.')\n",
"\n",
"try:\n",
" (key, metadata, bins) = client.get(key, policy)\n",
"except:\n",
" print('failed to get record')\n",
" sys.exit(1)\n",
"print ('Bins: ', bins)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Appending, Prepending and Incrementing a Bin\n",
"The following APIs are used to prepend and append string bins, and increment integer bins:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Bins: {'name': 'Dr. Bender Bending Rodriguez', 'serialnum': 2716057, 'lastsentence': {'ltWGY': 'Into the breach, meatbags. Or not, whatever', 'BG': 'Whip harder, Professor!', 'BBS': \"Well, we're boned\", 'TBwaBB': 'I love you, meatbags!'}, 'composition': ['40% zinc', '40% titanium', '30% iron', '40% dolomite'], 'apartment': bytearray(b'$'), 'quote_cnt': 50}\n"
]
}
],
"source": [
"try:\n",
" client.prepend(key, 'name', 'Dr. ')\n",
" client.append(key, 'name', ' Bending Rodriguez')\n",
" client.increment(key, 'quote_cnt', 3)\n",
"except:\n",
" print('failed to get record')\n",
" sys.exit(1)\n",
"\n",
"try:\n",
" (key, metadata, bins)= client.get(key, policy)\n",
"except:\n",
" print('failed to get record')\n",
" sys.exit(1)\n",
"print ('Bins: ', bins)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Reading Records\n",
"There are multiple ways to read records from Aerospike database, \"get\" being the simplest one.\n",
"You will need the key as the record unique identifier as discussed above. Note the record was written above.\n",
"\n",
"It returns:\n",
"- key — The key tuple of the record that was read.\n",
"- meta — The dict containing the record metadata gen and ttl fields.\n",
"- bins — The dict containing the bins of the record.\n",
"\n",
"Meta and bins are None if the record is not found."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Successfully read the record.\n",
"Key: ('test', 'demo', 'foo', bytearray(b'\\xf5~\\xc1\\x835\\xf7\\x10\\x0c\\x04X\\xf8\\xa6D\\xbc\\xbcvm\\x93G\\x1e'))\n",
"Metadata: {'ttl': 2592000, 'gen': 4}\n",
"Bins: {'name': 'John Doe', 'age': 15, 'gpa': 4.3}\n"
]
}
],
"source": [
"# key of the record\n",
"key = ('test', 'demo', 'foo')\n",
"# Retrieve the record using the key.\n",
"try:\n",
" (key, meta, bins) = client.get(key, policy)\n",
"except:\n",
" print('failed to get record')\n",
" sys.exit(1)\n",
"print('Successfully read the record.')\n",
"print ('Key: ', key)\n",
"print ('Metadata: ', metadata)\n",
"print ('Bins: ', bins)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Projecting the Bins of a Record\n",
"It is possible to project or read specific bins of a record. The following example illustrates this. Note that the second argument is a tuple of bin names to project."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Bins: {'age': 15, 'gpa': 4.3}\n"
]
}
],
"source": [
"# retrieve only specified bins in the record\n",
"try:\n",
" (key, meta, bins) = client.select(('test','demo','foo'), ('age', 'gpa'))\n",
"except:\n",
" print('failed to get record')\n",
" sys.exit(1)\n",
"print ('Bins: ', bins)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Checking if a Record Exists\n",
"Use \"exists\" to check existence of a record in the database. Note, meta is None if the record is not found."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"User-key, Metadata: foo {'ttl': 2592000, 'gen': 1}\n",
"User-key, Metadata: nonexistent None\n"
]
}
],
"source": [
"# Retrieve the record using a non-existent key.\n",
"(key, metadata) = client.exists(('test','demo','foo'), policy)\n",
"print('User-key, Metadata: ', key[2], metadata)\n",
"(key, metadata) = client.exists(('test','demo','nonexistent'), policy)\n",
"print('User-key, Metadata: ', key[2], metadata)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Batch Read Operations\n",
"The get_many and exists_many batch operations allow the application to access multiple records."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[(('test',\n",
" 'demo',\n",
" 'key1',\n",
" bytearray(b'\\xec\\x91\\x19-K\\x7f\\x8c\\xe3]]x\\xd3K\\xcae\\xcb\\xaa\\xaa\\xc9`')),\n",
" {'gen': 1, 'ttl': 2592000},\n",
" {'batch': 1}),\n",
" (('test',\n",
" 'demo',\n",
" 'key2',\n",
" bytearray(b'&o\\xc5\\xc9\\x94\\xab\\x9d\\n\\xa1\\xc3g\\xd9\\x9a\\xbc\\xe6\\xd8'\n",
" b'\\x9e\\x97p\\x83')),\n",
" {'gen': 1, 'ttl': 2592000},\n",
" {'batch': 2}),\n",
" (('test',\n",
" 'demo',\n",
" 'key3',\n",
" bytearray(b'cN\\x96\\xf7\\xff\\\\\\xd9hS8\\x18\\xdc\"v\\x18\\xe2\\xe4D\\'\\xa5')),\n",
" None,\n",
" None)]\n"
]
}
],
"source": [
"import pprint\n",
"pp = pprint.PrettyPrinter(depth=4)\n",
"keys = []\n",
"for i in range(1,3):\n",
" key = ('test', 'demo', 'key' + str(i))\n",
" client.put(key, {'batch': i}, policy)\n",
"for i in range(1,4):\n",
" key = ('test', 'demo', 'key' + str(i))\n",
" keys.append(key)\n",
"records = client.get_many(keys)\n",
"pp.pprint (records)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deleting Records\n",
"To delete records from the database, there is \"remove\" operation.\n",
"\n",
"An exception is thrown if the record does not exist."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Key, metadata: ('test', 'demo', 'key1') None\n",
"Successfully deleted ('test', 'demo', 'key1')\n",
"failed to delete record: ('test', 'demo', 'non-existent')\n"
]
}
],
"source": [
"# Key of the record to be deleted\n",
"key1 = ('test', 'demo', 'key1')\n",
"\n",
"# Delete the record\n",
"try:\n",
" client.remove(key1)\n",
"except:\n",
" print('failed to delete record: ', key1)\n",
" sys.exit(1)\n",
"(key, metadata) = client.exists(key1, policy)\n",
"print('Key, metadata: ', key1, metadata)\n",
"print('Successfully deleted ', key1)\n",
" \n",
"# will throw an exception\n",
"nokey = ('test', 'demo', 'non-existent')\n",
"try:\n",
" client.remove(key)\n",
"except:\n",
" print('failed to delete record: ', nokey)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Multiple Operations on a Single Record\n",
"Multiple operations on a single a record can be conveniently performed in a single transaction. Multiple updates as well as reads of a record may be performed atomically in a single transaction. Operations are performed in the order they are specified. Below is a full example of a multi-op request."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Key: ('test', 'demo', 1, bytearray(b'\\xb7\\xf4\\xb88\\x89\\xe2\\xdag\\xdeh>\\x1d\\xf6\\x91\\x9a\\x1e\\xac\\xc4F\\xc8'))\n",
"--------------------------\n",
"Metadata: {'ttl': 2592000, 'gen': 2}\n",
"--------------------------\n",
"Bins: {'name': 'Phillip J. Fry', 'career': 'delivery boy', 'age': 1025}\n"
]
}
],
"source": [
"from __future__ import print_function\n",
"import aerospike\n",
"from aerospike_helpers.operations import operations as op_helpers\n",
"from aerospike import exception as ex\n",
"import sys\n",
"\n",
"config = { 'hosts': [('127.0.0.1', 3000)] }\n",
"client = aerospike.client(config).connect()\n",
"\n",
"try:\n",
" key = ('test', 'demo', 1)\n",
" client.put(key, {'age': 25, 'career': 'delivery boy'})\n",
" ops = [\n",
" op_helpers.increment(\"age\", 1000),\n",
" op_helpers.write(\"name\", \"J.\"),\n",
" op_helpers.prepend(\"name\", \"Phillip \"),\n",
" op_helpers.append(\"name\", \" Fry\"),\n",
" op_helpers.read(\"name\"),\n",
" op_helpers.read(\"career\"),\n",
" op_helpers.read(\"age\")\n",
" ]\n",
" (key, metadata, bins) = client.operate(key, ops, {'ttl':0}, {'total_timeout':500})\n",
"except ex.AerospikeError as e:\n",
" print(\"Error: {0} [{1}]\".format(e.msg, e.code))\n",
" sys.exit(1)\n",
"\n",
"print('Key: ', key)\n",
"print('--------------------------')\n",
"print('Metadata: ', metadata)\n",
"print('--------------------------')\n",
"print('Bins: ', bins) # shows only bins specified in read operations\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Multi-Op Example: Read and Delete in One Request\n",
"The example shows reading and deleting a record in a single operation. The record was create above."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Bins: {'name': 'Phillip J. Fry'}\n",
"Key, metadata: ('test', 'demo', 1) None\n",
"Successfully deleted ('test', 'demo', 1)\n"
]
}
],
"source": [
"from aerospike_helpers.operations import operations\n",
"key = ('test', 'demo', 1)\n",
"ops = [ operations.read('name'), \n",
" operations.delete() ]\n",
"_, _, bins = client.operate(key, ops)\n",
"print('Bins: ', bins)\n",
"# will throw an exception\n",
"(key_, metadata) = client.exists(key, policy)\n",
"print('Key, metadata: ', key, metadata)\n",
"print('Successfully deleted ', key)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"Visit [Aerospike notebooks repo](https://github.com/aerospike-examples/interactive-notebooks) to run additional Aerospike notebooks. To run a different notebook, download the notebook from the repo to your local machine, and then click on File->Open, and select Upload.\n"
]
}
],
"metadata": {
"file_extension": ".py",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.6"
},
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
"pygments_lexer": "ipython3",
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": true,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
},
"version": 3
},
"nbformat": 4,
"nbformat_minor": 2
}