{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Fuzzy File Finding\n",
"\n",
"`importnb` contains a questionable design choice that allows for fuzzy file finding. `importnb` imports notebooks and those notebook file names contains can characters like ` ` or `-` or perhaps they start with numbers like a blog post. These files names would confuse the start import mechanisms.\n",
"\n",
"The biggest trouble with the Fuzzy Finder is that imports are non deterministic.\n",
"\n",
"`importnb` allows fuzzy search patterns using `_` and `__` to replace the names with `?` and `??` respectively."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## A Fuzzy Completer\n",
"\n",
"Our new completer will consider our prior blog posts on creating IPython completers and Markdown. Our completer should predict modules even when the code is indented.\n",
"\n",
"We have a hypothesis that fuzzy completion will assist the user in using Fuzzy patterns.\n",
"\n",
"[2018-07-03-Custom-IPython-Completer-for-Indented-Code.ipynb](2018-07-03-Custom-IPython-Completer-for-Indented-Code.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Fuzzy Completer pattern"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Below is a pattern to discover this blog post. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
" pattern = '__Fuzzy_import__weird__'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The underscores in the pattern are replaced in the following manner"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'*Fuzzy?import*weird*'"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
" pattern.replace('__', '*').replace('_', '?')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the `importnb` fuzzy find logic we will discover this post."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[WindowsPath('2018-07-10-Fuzzy-importing-files-with-weird-characters.ipynb')]"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
" from fnmatch import fnmatch\n",
" from pathlib import Path\n",
" import string\n",
" from importnb.finder import fuzzy_file_search\n",
" files = fuzzy_file_search('', pattern)\n",
" files"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
" def fuzzify_string(str, *, fuzzified = ''):\n",
" return str[0] in string.ascii_letters + '_' and str[0] or '_' \\\n",
" + ''.join('_' if letter in ' -' else letter for letter in str[1:])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* For the completer to work, the returned value must match the beginning of the `event.symbol`. `align_match` will replace the beginning portion with the required prefix as applied to our fuzzy search criterion. \n",
"\n",
"https://github.com/ipython/ipython/blob/49de6e5aa8e6840d78aafd042bfc6e361018b988/IPython/core/completer.py#L1739"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
" def align_match(match, prefix, *, i=0):\n",
" pattern = prefix.replace('__',' *').replace('_', '?').strip()\n",
" for i in range(len(match)): \n",
" if fnmatch(match[:i], pattern): break\n",
" return prefix + match[i:]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* `predict_fuzzy` will take a fully qualified fuzzy name completions."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
" def predict_fuzzy(fullname):\n",
" package, paths, specs = '', [], [] \n",
" if '.' in fullname:\n",
" package, fullname = fullname.rsplit('.', 1)\n",
" fullname = fullname.strip()\n",
" try:\n",
" module = __import__('importlib').import_module(package)\n",
" paths.append(Path(module.__file__).parent)\n",
" except: ...\n",
" else: paths = map(Path, __import__('sys').path)\n",
" query_name = fullname\n",
" while not query_name.endswith('__'): query_name += '_'\n",
" for path in paths: specs.extend(\n",
" str(object.relative_to(path).with_suffix('')) \n",
" for object in fuzzy_file_search(path, query_name))\n",
" return set((package and package + '.' or '') + align_match(fuzzify_string(spec), fullname) for spec in specs)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
" Ø = __name__ == '__main__';\n",
" if Ø: ip = get_ipython()\n",
" from pathlib import Path"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* [Every completer recieves self and event.](https://github.com/ipython/ipython/blob/master/IPython/core/completerlib.py)\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
" def event(self, event):\n",
" event.line = event.line.lstrip()\n",
" symbol = event.symbol\n",
" if event.line.startswith('from'):\n",
" if ' import' in event.line:\n",
" package = event.line.split(' import', 1)[0].lstrip().lstrip('from').lstrip()\n",
" return [object.lstrip(package).lstrip('.') for object in predict_fuzzy('.'.join((package, symbol)))]\n",
" return predict_fuzzy(symbol)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* The extension adds the new fuzzy completer. Our completer has a higher priority than the default completers. Since we stripped the leading whitespace from the completion line event; the extension will permit completion on tabbed lines."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
" def load_ipython_extension(ip): \n",
" ip.set_hook('complete_command', event, str_key=\"aimport\", priority=25) \n",
" ip.set_hook('complete_command', event, str_key=\"import\", priority=25) \n",
" ip.set_hook('complete_command', event, str_key=\"%reload_ext\", priority=25) \n",
" ip.set_hook('complete_command', event, str_key=\"%load_ext\", priority=25)\n",
" ip.set_hook('complete_command', event, str_key=\"from\", priority=25)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Results\n",
"\n",
"### Completion before adding the completer"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"('deathbeds.__complete', []) ('__complete', []) ('req', [])\n"
]
}
],
"source": [
" Ø and print(\n",
" ip.complete('deathbeds.__complete', 'import deathbeds.__complete'), ip.complete('__complete', 'import __complete'), ip.complete('req', '\\timport req'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Install the completer"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
" if Ø:\n",
" # load_ipython_extension(get_ipython())\n",
" %reload_ext deathbeds._018_07_03_Custom_IPython_Completer_for_Indented_Code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Valid Completion after installing the extension"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"('deathbeds.__complete', ['deathbeds.__completer_for_Indented_Code']) ('__complete', []) ('req', ['requests', 'requests_cache']) ('__complete', ['__completer_for_Indented_Code'])\n"
]
}
],
"source": [
" Ø and print(\n",
" ip.complete('deathbeds.__complete', 'import deathbeds.__complete'), \n",
" ip.complete('__complete', 'import __complete'), \n",
" ip.complete('req', '\\timport req'),\n",
" ip.complete('__complete', 'from deathbeds import __complete'))"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
" if Ø: import disqus"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "p6",
"language": "python",
"name": "other-env"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}