{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# A subtle bug in A.pretty()\n", "\n", "2019-04-12\n", "\n", "Christiaan Erwich, Gyusang Jin and Cody Kingham spotted weird behaviour in displaying certain verses.\n", "\n", "Look here in Psalm 18:49 :\n", "\n", "![ps18:49](images/Ps18-49.png)\n", "\n", "The dashed circles enclose the words that are in this verse, the other words belong to the previous part.\n", "\n", "When displaying the sentence atom, which started in the previous part, Text-Fabric *forgot* to remember to\n", "cutoff this sentence atom at the verse boundary.\n", "\n", "And why was that?\n", "\n", "The root cause is in a few lines in the generic TF display library, not in the BHSA TF-app.\n", "\n", "```\n", " getBoundary(api, n) if d.condenseType is None else\n", "```\n", "\n", "Here TF computes the boundary slots of the node in question.\n", "The thing is, if there is a condense type active, we should not take the boundary of the node itself, but its container of the\n", "condense type.\n", "\n", "However, in our case we have the situation that `d.condenseType` appeared to be `verse`.\n", "Yet, there is no condensing, because `d.condensed` is `False`.\n", "\n", "So this part of the condition was wrong and should be:\n", "\n", "```\n", " getBoundary(api, n) if not d.condensed or not d.condenseType else\n", "```\n", "\n", "[See the bug on GitHub](https://github.com/annotation/text-fabric/blob/421e7b577f00cbc782ca6ac9f3a132688071f59f/tf/applib/display.py#L435)\n", "\n", "This indeed fixed the bug, as you see below." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from tf.applib.helpers import dm\n", "from tf.app import use" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that we are developing/debugging code. We do not want to download code/data from github,\n", "and we want to look in our own github clone of the TF app.\n", "\n", "Hence `'bhsa:clone'` and `checkout='local'`." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using TF-app in /Users/dirk/github/annotation/app-bhsa/code:\n", "\trepo clone offline under ~/github (local github)\n", "Using data in /Users/dirk/text-fabric-data/etcbc/bhsa/tf/c:\n", "\trv1.6 offline under ~/text-fabric-data (local release)\n", "Using data in /Users/dirk/text-fabric-data/etcbc/phono/tf/c:\n", "\tr1.2 offline under ~/text-fabric-data (local release)\n", "Using data in /Users/dirk/text-fabric-data/etcbc/parallels/tf/c:\n", "\tr1.2 offline under ~/text-fabric-data (local release)\n" ] }, { "data": { "text/html": [ "Documentation: BHSA Character table Feature docs bhsa API Text-Fabric API 7.6.3 Search Reference
Loaded features:\n", "

BHSA = Biblia Hebraica Stuttgartensia Amstelodamensis: book book@ll chapter code det freq_lex function g_word g_word_utf8 gloss gn label language lex lex_utf8 ls nametype nu number otype pdp prs_gn prs_nu prs_ps ps qere qere_trailer qere_trailer_utf8 qere_utf8 rank_lex rela sp st trailer trailer_utf8 txt typ verse voc_lex voc_lex_utf8 vs vt mother oslots

Parallel Passages: crossref

Phonetic Transcriptions: phono phono_trailer

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
API members:\n", "C Computed, Call AllComputeds, Cs ComputedString
\n", "E Edge, Eall AllEdges, Es EdgeString
\n", "ensureLoaded, TF, ignored, loadLog
\n", "L Locality
\n", "cache, error, indent, info, reset
\n", "N Nodes, sortKey, sortKeyTuple, otypeRank, sortNodes
\n", "F Feature, Fall AllFeatures, Fs FeatureString
\n", "S Search
\n", "T Text
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "A = use('bhsa:clone', checkout='local', hoist=globals())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Inspection\n", "\n", "Before we knew what was the matter we made a lens to have a detailed view of our data:\n", "\n", "We gathered all word nodes of Ps 18:47 - 49 and made a table of the objects they are contained in." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "v0 = T.nodeFromSection(('Psalms', 18, 47))\n", "v1 = T.nodeFromSection(('Psalms', 18, 49))\n", "w0 = L.d(v0, otype='word')[0]\n", "w1 = L.d(v1, otype='word')[-1]\n", "allWords = range(w0, w1 + 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We are only interested in certain kind of objects." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['verse',\n", " 'sentence',\n", " 'sentence_atom',\n", " 'clause',\n", " 'clause_atom',\n", " 'phrase',\n", " 'phrase_atom']" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "skipTypes = {'book', 'chapter', 'half_verse', 'lex', 'subphrase', 'word'}\n", "nodeTypes = [x[0] for x in C.levels.data if x[0] not in skipTypes]\n", "nodeTypes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We represent the nodes of different types in slightly different ways." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "repNode = dict(\n", " verse=lambda n: '{} {}:{}'.format(*T.sectionFromNode(n)),\n", " sentence=lambda n: F.number.v(n),\n", " sentence_atom=lambda n: F.number.v(n),\n", " clause=lambda n: F.number.v(n),\n", " clause_atom=lambda n: F.number.v(n),\n", " phrase=lambda n: F.number.v(n),\n", " phrase_atom=lambda n: F.number.v(n),\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And we make a markdown table." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "verse | sentence | sentence_atom | clause | clause_atom | phrase | phrase_atom\n", "--- | --- | --- | --- | --- | --- | ---\n", "Psalms 18:47 | 93 | 511 | 1 | 761 | 1 | 1818\n", "Psalms 18:47 | 93 | 511 | 1 | 761 | 2 | 1819\n", "Psalms 18:47 | 94 | 512 | 1 | 762 | 1 | 1820\n", "Psalms 18:47 | 94 | 512 | 1 | 762 | 2 | 1821\n", "Psalms 18:47 | 94 | 512 | 1 | 762 | 3 | 1822\n", "Psalms 18:47 | 95 | 513 | 1 | 763 | 1 | 1823\n", "Psalms 18:47 | 95 | 513 | 1 | 763 | 2 | 1824\n", "Psalms 18:47 | 95 | 513 | 1 | 763 | 3 | 1825\n", "Psalms 18:47 | 95 | 513 | 1 | 763 | 3 | 1825\n", "Psalms 18:48 | 93 | 514 | 2 | 764 | 1 | 1826\n", "Psalms 18:48 | 93 | 514 | 2 | 764 | 1 | 1826\n", "Psalms 18:48 | 93 | 514 | 3 | 765 | 1 | 1827\n", "Psalms 18:48 | 93 | 514 | 3 | 765 | 2 | 1828\n", "Psalms 18:48 | 93 | 514 | 3 | 765 | 3 | 1829\n", "Psalms 18:48 | 93 | 514 | 3 | 765 | 4 | 1830\n", "Psalms 18:48 | 93 | 514 | 4 | 766 | 1 | 1831\n", "Psalms 18:48 | 93 | 514 | 4 | 766 | 2 | 1832\n", "Psalms 18:48 | 93 | 514 | 4 | 766 | 3 | 1833\n", "Psalms 18:48 | 93 | 514 | 4 | 766 | 4 | 1834\n", "Psalms 18:49 | 93 | 514 | 5 | 767 | 1 | 1835\n", "Psalms 18:49 | 93 | 514 | 5 | 767 | 2 | 1836\n", "Psalms 18:49 | 93 | 514 | 5 | 767 | 2 | 1836\n", "Psalms 18:49 | 96 | 515 | 1 | 768 | 1 | 1837\n", "Psalms 18:49 | 96 | 515 | 1 | 768 | 1 | 1837\n", "Psalms 18:49 | 96 | 515 | 1 | 768 | 1 | 1837\n", "Psalms 18:49 | 96 | 515 | 1 | 768 | 2 | 1838\n", "Psalms 18:49 | 97 | 516 | 1 | 769 | 1 | 1839\n", "Psalms 18:49 | 97 | 516 | 1 | 769 | 1 | 1839\n", "Psalms 18:49 | 97 | 516 | 1 | 769 | 1 | 1839\n", "Psalms 18:49 | 97 | 516 | 1 | 769 | 2 | 1840" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "rows = []\n", "rows.append(' | '.join(nodeTypes))\n", "rows.append(' | '.join(['---'] * len(nodeTypes)))\n", "\n", "\n", "for w in allWords:\n", " parents = {F.otype.v(p): repNode[F.otype.v(p)](p) for p in L.u(w) if F.otype.v(p) in nodeTypes}\n", " rows.append(' | '.join(str(parents.get(nType, 'x')) for nType in nodeTypes))\n", " \n", "dm('\\n'.join(rows))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This gave us the clue that we had a bunch of objects that crossed the verse boundary." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# After the fix\n", "\n", "We show the verse after the fix." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "n = T.nodeFromSection(('Psalms', 18, 49))" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Psalms 18:49מְפַלְּטִ֗י מֵאֹ֫יְבָ֥י אַ֣ף מִן־קָ֭מַי תְּרֹומְמֵ֑נִי מֵאִ֥ישׁ חָ֝מָ֗ס תַּצִּילֵֽנִי׃ " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "A.plain(n)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "
\n", "
Psalms 18:49
\n", " \n", "
\n", "\n", "
\n", "\n", "
\n", " sentence 93|514\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " clause Coor Ptcp\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase PtcO VP\n", "
\n", "
\n", "\n", "
\n", "
מְפַלְּטִ֗י
\n", "
verb escape piel ptca
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Cmpl PP\n", "
\n", "
\n", "\n", "
\n", "
מֵ
\n", "
prep from
\n", "\n", "\n", "
\n", "\n", "
\n", "
אֹ֫יְבָ֥י
\n", "
subs be hostile qal ptca
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " sentence 96|515\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " clause xYq0\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Cmpl PP\n", "
\n", "
\n", "\n", "
\n", "
אַ֣ף
\n", "
advb even
\n", "\n", "\n", "
\n", "\n", "
\n", "
מִן־
\n", "
prep from
\n", "\n", "\n", "
\n", "\n", "
\n", "
קָ֭מַי
\n", "
subs arise qal ptca
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase PreO VP\n", "
\n", "
\n", "\n", "
\n", "
תְּרֹומְמֵ֑נִי
\n", "
verb be high piel impf
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " sentence 97|516\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " clause xYq0\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Cmpl PP\n", "
\n", "
\n", "\n", "
\n", "
מֵ
\n", "
prep from
\n", "\n", "\n", "
\n", "\n", "
\n", "
אִ֥ישׁ
\n", "
subs man
\n", "\n", "\n", "
\n", "\n", "
\n", "
חָ֝מָ֗ס
\n", "
subs violence
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase PreO VP\n", "
\n", "
\n", "\n", "
\n", "
תַּצִּילֵֽנִי׃
\n", "
verb deliver hif impf
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "A.pretty(n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Other example:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "

result 1

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "
\n", "
Psalms 129:6
\n", " \n", "
\n", "\n", "
\n", "\n", "
\n", " sentence 11|4672\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " clause ZYq0\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Pred VP\n", "
\n", "
\n", "\n", "
\n", "
יִ֭הְיוּ
\n", "
verb be qal impf
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase PreC PP\n", "
\n", "
\n", "\n", "
\n", "
כַּ
\n", "
prep as
\n", "\n", "\n", "
\n", "\n", "
\n", "
חֲצִ֣יר
\n", "
subs grass
\n", "\n", "\n", "
\n", "\n", "
\n", "
גַּגֹּ֑ות
\n", "
subs roof
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " clause Attr xQt0\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Rela CP\n", "
\n", "
\n", "\n", "
\n", "
שֶׁ
\n", "
conj <relative>
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Adju NP\n", "
\n", "
\n", "\n", "
\n", "
קַּדְמַ֖ת
\n", "
subs beginning
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Pred VP\n", "
\n", "
\n", "\n", "
\n", "
שָׁלַ֣ף
\n", "
verb draw qal perf
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " clause Coor ZQt0\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Pred VP\n", "
\n", "
\n", "\n", "
\n", "
יָבֵֽשׁ׃
\n", "
verb be dry qal perf
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "
\n", "
Psalms 129:7
\n", " \n", "
\n", "\n", "
\n", "\n", "
\n", " sentence 11|4672\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " clause Attr xQtX\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Rela CP\n", "
\n", "
\n", "\n", "
\n", "
שֶׁ
\n", "
conj <relative>
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Nega NegP\n", "
\n", "
\n", "\n", "
\n", "
לֹּ֤א
\n", "
nega not
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Pred VP\n", "
\n", "
\n", "\n", "
\n", "
מִלֵּ֖א
\n", "
verb be full piel perf
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Objc NP\n", "
\n", "
\n", "\n", "
\n", "
כַפֹּ֥ו
\n", "
subs palm
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Subj NP\n", "
\n", "
\n", "\n", "
\n", "
קֹוצֵ֗ר
\n", "
subs harvest qal ptca
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " clause Ellp\n", "
\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Conj CP\n", "
\n", "
\n", "\n", "
\n", "
וְ
\n", "
conj and
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Objc NP\n", "
\n", "
\n", "\n", "
\n", "
חִצְנֹ֥ו
\n", "
subs bosom
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", " phrase Subj NP\n", "
\n", "
\n", "\n", "
\n", "
מְעַמֵּֽר׃
\n", "
subs gather ears piel ptca
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n", "\n", "\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "s1 = 1222671\n", "A.show(((s1,),))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 4 }