{"openapi":"3.0.2","info":{"title":"Lnx Docs","description":"# 👋 Welcome to the docs\n\nFeel free to have a look around and explore the API of lnx.\n\nIf you're new you can get started by scrolling down a bit onto the `installing` section.\n\n\n# 🏗️ Installing\n\nSetting up lnx is pretty simple overall, you will want either a copy of the docker image\nor build from source to start using it.\n\n## Building from source\nYou will need a newish version of rust for this, this project was built on rustc `rustc 1.52.1`\nto be specific so any version beyond this should be alright.\n\n- Download the file via `git clone https://github.com/lnx-search/lnx.git` and enter the downloaded \nfolder.\n- Run `cargo build --release`\n- Extract the exported binary from the `target/release` folder.\n\n*alternatively you can use `cargo run --release -- <flags>` if you want to avoid the \njob of extracting the built binary*\n\n## Running from docker\nDocker images are pre-built following the `master` branch, this becomes the `latest`\ndocker image tag.\n\n#### Running via docker CLI\n```bash\ndocker run chillfish8/lnx:latest -p \"8000:8000\" -e \"AUTHORIZATION_KEY=hello\" -e \"LOG_LEVEL=info\" \n```\n\n*Note: Running without a persistent volume will mean no data will be kept\nbetween a restart, if you intend on deploying this it is **HIGHLY** advised\nyou mount a volume.*\n\n#### Running via docker-compose\n```yaml\nversion: '3'\n\nservices:\n    lnx:\n      image: chillfish8/lnx:latest\n      ports:\n        - \"8000:8000\"\n      volumes:\n        - \"/my/dir:/etc/lnx\" \n      environment:\n        - AUTHORIZATION_KEY=hello\n        - LOG_LEVEL=info\n```\n\n# 🚀 Running\nRunning a default version of lnx can be done simply using the following commands:\n\n#### Running\n```\nlnx <flags>\n```\nThis will start the sever in a default configuration.\n\n*NOTE: It is generally recommend to customise the running on lnx for production and optimum systems,\nsee [**optimising lnx**](/#section/Optimising) for information on why it matters for performance sake.* \n\n#### Help\n```\nlnx --help\n``` \nThis will bring up a detailed help command for \nall CLI options (including environment options.)\n\n## Optional Flags\nlnx provides a wide set of command line arguments to customise the running\nof the server.\n\n### Authentication Key\nIf specified this will require an authentication key on each request. \n \nGenerally, it's recommended to have this in a production environment.\n\nAs CLI:\n```\n-a, --authentication-key <authentication-key>\n```\n\nAs environment key:\n```\nAUTHENTICATION_KEY=<key>\n```\n\n\n### Server Host\nThe host to bind to (normally: '127.0.0.1' or '0.0.0.0'.) \n\nThe default is `127.0.0.1`\n\nAs CLI:\n```\n-h, --host <host>\n```\n\nAs environment key:\n```\nHOST=<host>\n```\n\n\n### Server Port\nThe port to bind the server to.\n\nThe default is `8000`\n\nAs CLI:\n```\n-p, --port <port>\n```\n\nAs environment key:\n```\nPORT=<port>\n```\n\n\n### Log File\nA optional file to send persistent logs.\n\nThis should be given as a file path.\n\nAs CLI:\n```\n--log-file <log-file>\n```\n\nAs environment key:\n```\nLOG_FILE=<log-file>\n```\n\n\n### Log Level\nThe log level filter, any logs that are above this level wont be displayed.\n\nDefaults to `info`\n\nAs CLI:\n```\n--log-level <log-level>\n```\n\nAs environment key:\n```\nLOG_LEVEL=<log-level>\n```\n\n\n### Pretty Logs\nAn optional bool to use ASNI colours for log levels. \nYou probably want to disable this if using file based logging.\n\nDefaults to `true`\n\nAs CLI:\n```\n--pretty-logs <pretty-logs>\n```\n\nAs environment key:\n```\nPRETTY_LOGS=<pretty-logs>\n```\n\n\n### Silent Search\nAn optional bool to disable info! level logs on every search request,\noften this can boost performance quite significantly.\nDefaults to `false`\n\nAs CLI:\n```\n--silent-search <silent-search>\n```\n\nAs environment key:\n```\nSILENT_SEARCH=<silent-search>\n```\n\n\n### Runtime Threads\nThe number of threads to use for the [tokio](https://tokio.rs) runtime.\n\n\nIf this is not set, the number of logical cores on the machine is used.\n\nAs CLI:\n```\n-t, --runtime-threads <runtime-threads>\n```\n\nAs environment key:\n```\nRUNTIME_THREADS=<runtime-threads>\n```\n","version":"0.6.2"},"paths":{"/indexes":{"post":{"tags":["📚 Managing indexes"],"summary":"Create Index","operationId":"Create_Index_indexes_post","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/IndexCreationPayload"}}},"required":true},"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index already exists (and override has not been set) or has been rejected by the engine due to a bad payload.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"The server was unable to deserialize the payload given.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/indexes/{index}":{"delete":{"tags":["📚 Managing indexes"],"summary":"Delete Index","operationId":"Delete_Index_indexes__index__delete","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"}],"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"This can never happen. (This is a docs issue)"},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/indexes/{index}/commit":{"post":{"tags":["💾 Managing transactions"],"summary":"Commit","description":"Finalises any changes to the index documents\nsince the last commit and saves them.","operationId":"Commit_indexes__index__commit_post","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"}],"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"This can never happen. (This is a docs issue)"},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/indexes/{index}/rollback":{"post":{"tags":["💾 Managing transactions"],"summary":"Rollback","description":"Reverts any changes to the index documents since the\nlast commit.","operationId":"Rollback_indexes__index__rollback_post","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"}],"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"This can never happen. (This is a docs issue)"},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/indexes/{index}/documents":{"post":{"tags":["📃 Managing documents"],"summary":"Add Documents","description":"Adding a document is relatively simple, you can either add a single document\nrepresented as a JSON object or you can submit a array of object.\n\nEvery document is checked for the required fields,\nif any docs are missing fields the *entire* request is rejected.","operationId":"Add_Documents_indexes__index__documents_post","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"}],"requestBody":{"content":{"application/json":{"schema":{"title":"Payload","anyOf":[{"type":"object","additionalProperties":{"anyOf":[{"type":"array","items":{"type":"string"}},{"type":"string"}]}},{"type":"array","items":{"type":"object","additionalProperties":{"anyOf":[{"type":"array","items":{"type":"string"}},{"type":"string"}]}}}]}}},"required":true},"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"The server was unable to deserialize the payload given.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}},"delete":{"tags":["📃 Managing documents"],"summary":"Delete Specific Documents","description":"Docs can only be deleted via terms, it's up to you to make sure a given term is\nunique otherwise multiple docs can be deleted via this method.\n\nNOTE: This only works with fast fields, so it's a good idea to make a unique\nid or use the document_id via the `_id` field.","operationId":"Delete_Specific_Documents_indexes__index__documents_delete","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"}],"requestBody":{"content":{"application/json":{"schema":{"title":"Payload","anyOf":[{"type":"object","additionalProperties":{"anyOf":[{"type":"array","items":{"type":"string"}},{"type":"string"}]}},{"type":"array","items":{"type":"object","additionalProperties":{"anyOf":[{"type":"array","items":{"type":"string"}},{"type":"string"}]}}}]}}},"required":true},"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"The server was unable to deserialize the payload given.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/indexes/{index}/documents/clear":{"delete":{"tags":["📃 Managing documents"],"summary":"Clear All Documents","description":"All docs can be cleared from the index via this endpoint.","operationId":"Clear_All_Documents_indexes__index__documents_clear_delete","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"}],"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"This can never happen. (This is a docs issue)"},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/indexes/{index}/documents/{document_id}":{"get":{"tags":["📃 Managing documents"],"summary":"Get Document By Id","description":"Get a single document from the index with it's given document_id.","operationId":"Get_Document_By_Id_indexes__index__documents__document_id__get","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"},{"required":true,"schema":{"title":"Document Id","type":"integer"},"name":"document_id","in":"path"}],"responses":{"200":{"description":"A standard response from Lnx, with a simple conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The index does not exist or the document does not exit.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"This can never happen. (This is a docs issue)"},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/indexes/{index}/search":{"post":{"tags":["🔍 Run searches"],"summary":"Search Index","description":"Search the index for the given query.","operationId":"Search_Index_indexes__index__search_post","parameters":[{"required":true,"schema":{"title":"Index","type":"string"},"name":"index","in":"path"}],"requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/QueryPayload"}}},"required":true},"responses":{"200":{"description":"A list of matching results ordered by and sorted according to the passed query.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/QueryResponse"}}}},"400":{"description":"The index does not exist or the query is malformed","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"The server was unable to deserialize the payload given.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/auth":{"post":{"tags":["🔑 Securing lnx"],"summary":"Create Token","description":"Creates a new access token with a given set of metadata.","operationId":"Create_Token_auth_post","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateTokenPayload"}}},"required":true},"responses":{"200":{"description":"A payload containing the response token and other metadata.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/TokenResponse"}}}},"422":{"description":"The server was unable to deserialize the payload given.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}},"delete":{"tags":["🔑 Securing lnx"],"summary":"Revoke All Token","description":"Revoke all access tokens.\n\n### WARNING:\nThis is absolutely only designed for use in an emergency.\nRunning this will revoke all tokens including the super user key,\nrun this at your own risk.","operationId":"Revoke_All_Token_auth_delete","responses":{"200":{"description":"A standard conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"The server was unable to deserialize the payload given.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/auth/{token}/revoke":{"post":{"tags":["🔑 Securing lnx"],"summary":"Revoke Token","description":"Revokes a given token, any requests after this with the given token\nwill be rejected.","operationId":"Revoke_Token_auth__token__revoke_post","responses":{"200":{"description":"A standard conformation message.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The token you provided in the url does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"This can never happen. (This is a docs issue)"},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}},"/auth/{token}/edit":{"post":{"tags":["🔑 Securing lnx"],"summary":"Edit Token","description":"Edits a given token's permissions and metadata.\nThe payload will replace **ALL** fields which will either set or unset the\nfields.","operationId":"Edit_Token_auth__token__edit_post","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateTokenPayload"}}},"required":true},"responses":{"200":{"description":"A payload containing the response token and other metadata.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"400":{"description":"The token you provided in the url does not exist.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"422":{"description":"The server was unable to deserialize the payload given.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}},"401":{"description":"You lack the permissions to run this operation.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BasicResponse"}}}}}}}},"components":{"schemas":{"BasicResponse":{"title":"BasicResponse","required":["status","data"],"type":"object","properties":{"status":{"title":"Status","type":"integer"},"data":{"title":"Data","anyOf":[{"type":"string"},{"type":"object"}]}}},"CreateTokenPayload":{"title":"CreateTokenPayload","required":["permissions"],"type":"object","properties":{"permissions":{"title":"Permissions","type":"integer"},"user":{"title":"User","type":"string"},"description":{"title":"Description","type":"string"},"allowed_indexes":{"title":"Allowed Indexes","type":"array","items":{"type":"string"}}}},"DocumentHit":{"title":"DocumentHit","required":["data","document_id"],"type":"object","properties":{"data":{"title":"Data","type":"object","additionalProperties":{"type":"array","items":{"type":"string"}}},"ratio":{"title":"Ratio","type":"number"},"document_id":{"title":"Document Id","type":"string"}}},"FastFieldType":{"title":"FastFieldType","enum":["single","multi"],"description":"An enumeration."},"FieldDeclaration":{"title":"FieldDeclaration","required":["type"],"type":"object","properties":{"type":{"$ref":"#/components/schemas/FieldType"},"stored":{"title":"Stored","type":"boolean"},"indexed":{"title":"Indexed","type":"boolean"},"fast":{"$ref":"#/components/schemas/FastFieldType"}}},"FieldType":{"title":"FieldType","enum":["f64","u64","i64","date","text","string","facet"],"description":"An enumeration."},"IndexCreationPayload":{"title":"IndexCreationPayload","required":["index"],"type":"object","properties":{"override_if_exists":{"title":"Override If Exists","type":"boolean","default":false},"index":{"$ref":"#/components/schemas/IndexDeclaration"}}},"IndexDeclaration":{"title":"IndexDeclaration","required":["name","storage_type","fields","search_fields","boost_fields","max_concurrency","writer_buffer","writer_threads"],"type":"object","properties":{"name":{"title":"Name","type":"string"},"storage_type":{"$ref":"#/components/schemas/StorageType"},"fields":{"title":"Fields","type":"object","additionalProperties":{"$ref":"#/components/schemas/FieldDeclaration"}},"search_fields":{"title":"Search Fields","type":"array","items":{"type":"string"}},"boost_fields":{"title":"Boost Fields","type":"object","additionalProperties":{"type":"string"}},"reader_threads":{"title":"Reader Threads","exclusiveMinimum":0.0,"type":"integer","default":1},"max_concurrency":{"title":"Max Concurrency","exclusiveMinimum":1.0,"type":"integer"},"writer_buffer":{"title":"Writer Buffer","minimum":300000.0,"type":"integer"},"writer_threads":{"title":"Writer Threads","exclusiveMinimum":0.0,"type":"integer"},"set_conjunction_by_default":{"title":"Set Conjunction By Default","type":"boolean","default":false},"use_fast_fuzzy":{"title":"Use Fast Fuzzy","type":"boolean","default":false},"strip_stop_words":{"title":"Strip Stop Words","type":"boolean","default":false}}},"Occur":{"title":"Occur","enum":["should","must","mustnot"],"description":"An enumeration."},"QueryData":{"title":"QueryData","required":["value"],"type":"object","properties":{"value":{"title":"Value","anyOf":[{"type":"string"},{"type":"integer"},{"type":"number"}]},"kind":{"allOf":[{"$ref":"#/components/schemas/QueryKind"}],"default":"fuzzy"},"occur":{"allOf":[{"$ref":"#/components/schemas/Occur"}],"default":"should"}}},"QueryKind":{"title":"QueryKind","enum":["normal","fuzzy","more-like-this",{"term":"field-name"}],"description":"An enumeration."},"QueryPayload":{"title":"QueryPayload","required":["query"],"type":"object","properties":{"query":{"title":"Query","anyOf":[{"type":"string"},{"$ref":"#/components/schemas/QueryData"},{"type":"array","items":{"$ref":"#/components/schemas/QueryData"}}]},"limit":{"title":"Limit","exclusiveMinimum":0.0,"type":"integer","default":20},"offset":{"title":"Offset","minimum":0.0,"type":"integer","default":0},"order_by":{"title":"Order By","type":"string"},"sort":{"allOf":[{"$ref":"#/components/schemas/Sort"}],"default":"desc"}}},"QueryResponse":{"title":"QueryResponse","required":["status","data"],"type":"object","properties":{"status":{"title":"Status","type":"integer"},"data":{"$ref":"#/components/schemas/QueryResults"}}},"QueryResults":{"title":"QueryResults","required":["hits","count","time_taken"],"type":"object","properties":{"hits":{"title":"Hits","type":"array","items":{"$ref":"#/components/schemas/DocumentHit"}},"count":{"title":"Count","type":"integer"},"time_taken":{"title":"Time Taken","type":"number"}}},"Sort":{"title":"Sort","enum":["acs","desc"],"description":"An enumeration."},"StorageType":{"title":"StorageType","enum":["memory","tempdir","filesystem"],"description":"An enumeration."},"TokenData":{"title":"TokenData","required":["permissions","created","token"],"type":"object","properties":{"permissions":{"title":"Permissions","type":"integer"},"user":{"title":"User","type":"string"},"description":{"title":"Description","type":"string"},"allowed_indexes":{"title":"Allowed Indexes","type":"array","items":{"type":"string"}},"created":{"title":"Created","type":"string","format":"date-time"},"token":{"title":"Token","type":"string"}}},"TokenResponse":{"title":"TokenResponse","required":["status","data"],"type":"object","properties":{"status":{"title":"Status","type":"integer"},"data":{"$ref":"#/components/schemas/TokenData"}}}}},"tags":[{"name":"📚 Managing indexes","description":"Creating and managing an index is fairly simple, once made lnx will it as a \npersistent config so that you don't need to continuously keep redefining it.\n\n### Gotcha Moments\nCreating an index can create quite a few gotcha moments so its important to understand the behaviour\nof the system before ramming your head into a wall wondering why things arent updating or being\ndestroyed.\n\n- This system is append / delete only, to update an index it must be deleted and re-made, all \ndata is lost in the process. This is because the index is schema-full and requires fields exist\nif define (This may change in future releases.)\n- Creating indexes are **VERY HEAVY** operations, not only are you acquiring a global lock across\nall indexes stopping anything else completing operations but you are also spawning and creating\nseveral threads and thread pools (more on this later.) They should not be made randomly during \nusage and should not be made via some user input *THIS WILL CAUSE SERIOUS ISSUES OTHERWISE*.\n- Deleting indexes clear all data stored, this means any previously uploaded documents will be\nremoved.\n- The `max_concurrency` determines the amount of readers proportionally, and \nthe `reader_threads` determines the amount of threads each reader gets, \ntherefore you spawn at least `max_concurrency` x `reader_threads` + `writer_threads` + `1` (for writer thread)\n\n### Support Field Data Types\n- `string` This is similar to `text` but wont be indexed. (Supports `stored` bool)\n- `text` This is similar to `string` but will be indexed. (Supports `stored` bool)\n- `f64` A 64 bit floating point integer field. (Supports [IntOptions](/getting_started/creating_a_index.html#int-options))\n- `i64` A 64 bit signed integer field. (Supports [IntOptions](/getting_started/creating_a_index.html#int-options))\n- `u64` A 64 bit unsigned point integer field. (Supports [IntOptions](/getting_started/creating_a_index.html#int-options))\n- `date` A UTC datetime field, stored as a u64 integer. (Supports [IntOptions](/getting_started/creating_a_index.html#int-options))\n\n\n### Int Options\nIf a field is *in italic* it's optional and not required.\n\n- `indexed`: bool - Whether or not to index this field. \nSetting an integer as indexed will generate a posting list for each value taken by the integer.\n\n- *`fast`: 'single' or 'multi'* - Set the field as a fast field with a\ngiven cardinality which makes it either a single-value or multi-value field. \nFast fields are designed for random access.\nAccess time are similar to a random lookup in an array.\nIf more than one value is associated to a fast field, only the last one is kept.\n\n- `stored`: bool - Set the field as stored.\nOnly the fields that are set as *stored* are persisted into the store.\n"},{"name":"💾 Managing transactions","description":"Transactions are one of the most important things in lnx, they control what data is seen by\nthe reader and what data is actually fully processed and saved.\n\nThe core concept of the transactions work on the same basis of tradition database transactions\nwith a one major difference:\n\n- There *should* only ever be **ONE** system interacting with a writer for a specific index at a time.\nThe system processes writes in single a queue, so if one system adds a document and another commits\nthen the document will be finalized and flushed to disk.</br> \n**It is highly recommended to wrap the writing endpoints in a lock**\n\n- Transactions are implicit vs explicit meaning you dont declare a transaction\nyou simple call commit and rollback respectively which apply their relevant affects\non writes made **since the last commit** once something is committed it cannot be rolled back.\n"},{"name":"📃 Managing documents","description":"Documents can only be added or removed, not updated.\nThis is preferred to be done explicitly hence no update endpoint.\n\nEach document is given a unique 64 bit integer which is returned as a string so\nthat it can be accessed later."},{"name":"🔍 Run searches","description":"Once you've added documents to you're ready to start searching!\n\nlnx provides you with 4 major ways to query the index:\n- `normal` The tantivy query parse system, this is not typo tolerant but is very powerful for custom user queries, think log searches.\n- `fuzzy`* A fuzzy query, this ignores the custom query system that the standard query parser would otherwise handle, but intern is typo tolerant.\n    *  if you have `use_fast_fuzzy` set to `true` for your given index this will\nuse the fast fuzzy system. \n- `more-like-this` Unlike the previous two options this takes a document reference address and produces documents similar to the given one. This is super useful for things like books etc... Wanting related items.\n- `term` expects the exact value in the query without any fuzzy matching or parsing like `normal`\n"},{"name":"🔑 Securing lnx","description":"Generally, it is highly recommended that you enable token authorization\nfor lnx although you should still put lnx behind a api / reverse proxy, \nmistakes do happen.\n\nLuckily for you lnx provides you with a simple a customisable authorization permission system.\n\nWhen you first setup permissions you will need to pass a `--super-user-key <key>` as a cli option\nwhen running lnx; this is not only how you access the endpoints to create new tokens at first\nbut also how lnx knows whether or not to enable auth.\n\nWhen passing authentication tokens to endpoints it's expected to be in the following\nformat in the headers:\n\n```\nAuthorization: <token>\n```\n\n## Warning\nlnx is still not designed to be fully public facing, although the authorization will\nproject from users having access to things they shouldn't it's still ***HIGHLY*** advised\nthat you do not rely on this system entirely to protect your system. We recommend putting\nlnx behind an existing backend api for most cases.\n\n## Permissions\n\n- **MODIFY_ENGINE** - `1 << 0` = 1\n    * Users with this permission can create and delete indexes.\n    This is arguably the most dangerous permission aside from super user.\n    \n- **SEARCH_INDEX** - `1 << 1` = 2\n    * Users with this permission can search their allowed indexes.\n    \n- **MODIFY_DOCUMENTS** - `1 << 2` = 4\n    * Users with this permission can add and remove documents from their\n    allowed indexes.\n    \n- **MODIFY_STOP_WORDS** - `1 << 3` = 8\n    * Users with this permission can add and remove stop words from their\n    allowed indexes.\n    \n- **MODIFY_AUTH** - `1 << 4` = 16\n    * Users with this permission can create, revoke and modify access tokens.\n    \n- **SUPER_USER** - `1 | 2 | 4 | 8 | 16` = 31\n    * This is just all of the above permissions combined, and can control everything.\n    \n    \n### Calculating Permissions\nPermissions are calculated bit fields meaning lnx expects an integer of the combined \npermissions you want. You can calculate the permission integer you need by taking\nthe bitwise `OR` / `|` of two permissions acting as a permission1 + permission2 flag.\nfor example if we want our token to be able to modify documents and search the index\nwe can do:\n\n```\npermissions = 1 << 2 | 1 << 1\n```\n\n"},{"name":"⚡ Optimising your index","description":"# Understanding Your Workload\nNot every set of data is going to have the same optimum configuration, there\nare a few points you probably want to be aware of when choosing your settings.\n\n### Large Datasets (millions+ of documents)\nSearching large datasets can benefit from having a larger number of reader\nthreads and a lower max concurrency set, having a low number of reader threads\nand a high concurrency has the negative affect of trying to run many searches\nat once very slowly vs a few searches very quickly. \n\nThere is also a hard capped limit on how much your system can actually handle.\nIf you set the max concurrency beyond what your system can handle you will\nseg fault at high loads which is a un-recoverable error. \n\n### Small Datasets (thousands+ of documents)\nSmaller datasets often cannot make good use of a larger number of reader threads,\nthis is because the data is not split into enough segments to make full use of\nthe parallel computation.\n\nOften with smaller datasets you're better having a higher concurrency and a lower\nnumber of reader threads.\n\nFor example, the movies.json dataset as part of the demo app is roughly most\noptimised at 1 or 2 reader threads and `n * 3` max concurrency where `n` is the\nnumber of cpu cores on the machine. \nThis can still change though depending on the clock speed of your CPU so still \nbenchmark and test before sticking with a configuration.\n"}]}