{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# pyCAT" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Picat language model implementation in metaL/Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "github: https://github.com/ponyatov/pycat\n", "\n", "Jupyter notebook render: https://nbviewer.jupyter.org/github/ponyatov/pycat/blob/master/pycat.ipynb\n", "\n", "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/ponyatov/pycat/master?filepath=pycat.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This manual is about learning Picat programming language by reimplementing it yourself from scratch. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This method is not for a total newbie in programming as you must know Python a bit to start, and have some entry-level programming skills. The Python was selected as it is the most popular programming language now for programming learning, and because this language implementation is so dynamic, as you can implement your own language system atop of PVM runtime. The advantage of learning by reimplementing is you are able not only embed this scripting engine to any software system you are doing but also deeply understand the original Picat." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "But it is not the end as this Picat implementation uses not pure Python but the `metaL`: homoiconic hypergraph model written itself in Python -- *executable data structure (EDS)* built from some sort of Marvin Minsky frames [minsky]. It was mostly selected to make this language to be not visual but **visualizable**: it is important for the newbies that a learning programming system to be Smalltalk-like interactive and deeply researchable." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### MIT License\n", "\n", "Copyright (c) 2019 Dmitry Ponyatov <>\n", "\n", "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n", "\n", "The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.\n", "\n", "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```sh\n", " ~$ git clone -o gh https://github.com/ponyatov/pycat ; cd pycat\n", "~/pycat$ make install\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Run" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```sh\n", "~/pycat$ .bin/activate\n", " (venv)$ jupyter notebook pycat.ipynb &\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Links" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* [minsky] [A Framework for Representing Knowledge](https://courses.media.mit.edu/2004spring/mas966/Minsky%201974%20Framework%20for%20knowledge.pdf) (c) Marvin Minsky, 1974\n", " * [Фреймы для представления знаний](https://royallib.com/book/minskiy_marvin/freymi_dlya_predstavleniya_znaniy.html) (c) Марвин Минский, М.: Мир, 1979\n", "\n", "* **Programming Python, 4th Edition** (c) Mark Lutz, O'Reilly 2019\n", " * [Изучаем Python. Том 1](https://www.ozon.ru/context/detail/id/156082566/) (c) Марк Лутц, М.: Вильямс, 2019\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Picat language" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* [Picat Google group](https://groups.google.com/forum/#!forum/picat-lang)\n", " * http://picat-lang.org/\n", " * [Constraint Solving and Planning with Picat](http://picat-lang.org/picatbook2015.html) (c) Neng-Fa Zhou, Håkan Kjellerstrand, Jonathan Fruhman\n", " * [Hakan Kjellerstrand page](http://hakank.org/picat/)\n", "\n", "* [Picat code samples](https://github.com/claudiosa/CCS/tree/master/picat) (c) Claudio Cesar de Sá\n", " * [Udemy course in Portuguese](https://www.udemy.com/course/picat-uma-linguagem-de-programacao-multiparadigma/)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prolog implementation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* [Yield Prolog](http://yieldprolog.sourceforge.net/)\n", "* [Warren's Abstract Machine: A Tutorial Reconstruction](http://wambook.sourceforge.net/)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Graph visualization" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This Jupyter notebook is packed with `graphviz` binding to show data structures we'll make later. To make it work locally, or [via Binder from the `master` branch on the GitHub](https://mybinder.org/v2/gh/ponyatov/pycat/master?filepath=pycat.ipynb) your host system must be preinstalled with some APT packages:\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "graphviz\r\n" ] } ], "source": [ "! cat apt.txt" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "graphviz==0.13.2\r\n" ] } ], "source": [ "! grep graphviz requirements.txt" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "Hello\n", "\n", "Hello\n", "\n", "\n", "World\n", "\n", "World\n", "\n", "\n", "Hello->World\n", "\n", "\n", "pycat\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import graphviz\n", "viz = graphviz.Digraph(graph_attr={'rankdir':'LR'})\n", "viz.edge('Hello','World',label='pycat')\n", "viz" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Frame" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The hypothesis: **directed hypergraph is a universal knowledge representation** that can describe anything including software specifications, programs, data and documentation in the same universal form. As we use the *same structure for program and data* representation, and this structure can be executed by interpretation, we have a **homoiconic metaprogramming system**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `Frame` is a root object class that is implemented on ideas of a [minsky] book. The original frame model was *extended* to support **ordered storage** is definitely required for representing\n", "* *attribute grammar* widely known in programming languages design, and\n", "* ordered data containers itself.\n", "\n", "The presence of an ordered collection is definitively required for representing program source code in any programming language, as this is very close to classical attribute grammar, parsing and abstract syntax trees, and graphs used in compiler design. The frame (object) hypergraph representation of a program as an executable form is clear and native for any work involved with source code transformations: synthesis, modifications, analysis, cross-language translation, etc." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### EDS: Executable Data Structure method" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In software design in a case when our programming language or computing platform does not provide some features, we can add a layer of interpretation for some data structure to get more dynamics or extra features such as metaprogramming, program self-modification, compact low memory bytecode, etc. For example, in the Lisp language, all programs are represented in the form of executable lists. Java uses bytecode as the main program form, which does not require, but often uses JIT for runtime compilation as a way to make the program run faster." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `metaL` homoiconic layer uses attributed hypergraph as an *executable data structure*:\n", "* every hypergraph **frame defines N-ary relations between operands** in `nest[]` and named attributes in `slot{}`s the way close to Prolog predicates\n", "* **frames are typed** so every frame class can describe different behavior and properties\n", "* the graph has the advantage to be not visual but **visualizable language**, which can be used for multiparadigm programming" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Homoiconic programming languages" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> **Homoiconicity** (homoiconic) \\\n", "is a property of some programming languages in which *data and program representation is the same*\n", "and defined in core types of the same language. In other words, homoiconicity is when the *program source code* is written *as the reflectable (and mutable) data structure* and the programming language provides transparent access to the program as data in runtime.\n", "\n", "In homoiconic language, **metaprogramming** *is the main software development technique*, which is also used to expand the language to the capabilities that a particular programmer needs. As the metaprogramming language, the Lisp language is the first sample, which was created to process data presented in the form of nested lists. Lisp programs are also recorded, stored, and executed as lists; as a result, it makes the running program able to access its own source code as a native data structure, as well as automatically change itself on the fly. Homoiconic languages, as a rule, include full support for *syntactic macros*, allowing the programmer to define new syntactic structures and express transformations of programs in a compact form." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "Frame\n", "\n", "Frame\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "viz = graphviz.Digraph(graph_attr={'rankdir':'LR'}) ; viz.node('Frame') ; viz" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "## Marvin Minsky's extended frame model\n", "\n", "class Frame:\n", "\n", " def __init__(self,V):\n", "\n", " # type/class tag /required for lexer using PLY library/\n", " self.type = self.__class__.__name__.lower()\n", "\n", " # scalar value: frame name, string, number,..\n", " self.val = V\n", "\n", " # slots = attributes = vocabulary\n", " self.slot = {}\n", "\n", " # nested AST = universal ordered container = stack\n", " self.nest = []" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "<__main__.Frame at 0x7fa3a4070048>" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Frame('Hello World')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tree dump to plain text" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll use tree dump as an easy to use method to see any frame graph in a text form." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "class Frame(Frame):\n", "\n", " ## dump\n", "\n", " # print/str conversion\n", " def __repr__(self):\n", " return self.dump()\n", "\n", " # full tree dump\n", " def dump(self,depth=0,prefix=''):\n", "\n", " # subtree header: tab padding and header\n", " tree = self._pad(depth) + self.head(prefix)\n", "\n", " # block infinite recursion if we have cycles in a graph\n", " if not depth: Frame._dump = [] # recursion root -- zero depth\n", " if self in Frame._dump: return tree + ' _/'\n", " else: Frame._dump.append(self)\n", "\n", " # slot{}s: recursive traversal via hypergraph references\n", " for i in self.slot:\n", " tree += self.slot[i].dump(depth+1,'%s = '%i)\n", "\n", " # nest[]ed: recursive traversal via ordered subgraphs\n", " idx = 0\n", " for j in self.nest:\n", " tree += j.dump(depth+1,'%i: '%idx) ; idx +=1\n", "\n", " # resulting subtree dump\n", " return tree\n", "\n", " # short header-only dump\n", " def head(self,prefix=''):\n", " return '%s<%s:%s> id:%x' % (prefix,self.type,self._val(),id(self))\n", "\n", " # tree dump padding\n", " def _pad(self,depth):\n", " return '\\n' + '\\t' * depth\n", "\n", " # .val in dump must be overridable for strings, numbers,..\n", " def _val(self):\n", " return '%s' % self.val" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "prefix = id:7fa3a4070a58\n" ] } ], "source": [ "print(Frame('Hello World').dump(prefix='prefix = '))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**`` header** contains\n", "* `T` type/*class tag*\n", "* `V` scalar *value*\n", "* optional `prefix` used for *slot names printing in tree dump*\n", "* `idx` counter shows index of every element in the universal ordered container *mostly used as a stack*\n", "* `id:hex` a pointer is unique for every object in a system lets to *distinct objects with the same `T:V` pair*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Graph plot" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As we pretend to make not visual but **visualizable programming language**, we must have a graph plotting tool to see any part of an EDS program graph as a diagram." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "class Frame(Frame):\n", "\n", " ## plot\n", "\n", " # recursive traversal with graphviz calls\n", " def plot(self,dot=None,parent=None,link='',color='black'):\n", "\n", " if not dot: # plotter must be initialized in a recursion root\n", " dot = graphviz.Digraph(graph_attr={'rankdir':'LR'}) # horizontal layout\n", " Frame._plot = [] # skip visited branches\n", "\n", " # plot node and optional edge from parent / like header in .dump() /\n", " dot.node('%s'%id(self),label='%s:%s'%(self.type,self._val())) # ref via id()\n", " if parent: dot.edge('%s'%id(parent),'%s'%id(self),label='%s'%link,color=color)\n", "\n", " # rings/infinite recursion block\n", " if self in Frame._plot: return dot\n", " else: Frame._plot.append(self)\n", "\n", " # slot{}s with blue edges and slot names\n", " for i in self.slot:\n", " dot = self.slot[i].plot(dot,parent=self,link=i,color='blue')\n", "\n", " # nest[]ed with green edges and integer indexes\n", " idx = 0\n", " for j in self.nest:\n", " dot = j.plot(dot,parent=self,link=idx,color='green') ; idx += 1\n", "\n", " # return resulting subtree plot\n", " return dot" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "140340808320840\n", "\n", "frame:Hello World\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Frame('Hello World').plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Operators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To manipulate frames in a pythonic way we need to define some used operators to do frame combinations:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "class Frame(Frame):\n", "\n", " ## operators\n", "\n", " # automatic type cast\n", " def cast(self,that):\n", " if isinstance(that,Frame): return that\n", " return {\n", " str : lambda x:Str(x),\n", " float: lambda x:Num(x),\n", " int : lambda x:Int(x)\n", " }[that.__class__](that)\n", "\n", " # ` A[key] ` get frame by slot name (vocabulary lookup/fetch)\n", " def __getitem__(self,key):\n", " return self.slot[key]\n", "\n", " # ` A[key] = B ` set/create slot with name and frame (vocabulary write)\n", " def __setitem__(self,key,that):\n", " self.slot[key] = self.cast(that) ; return self\n", "\n", " # ` A << B ` set slot with operand type A[B.type] = B\n", " def __lshift__(self,that):\n", " that = self.cast(that) ; self[that.type] = that ; return self\n", "\n", " # ` A << B ` set slot with operand value A[B.val] = B\n", " def __rshift__(self,that):\n", " that = self.cast(that) ; self[that.val] = that ; return self\n", "\n", " # ` A // B ` push to nest[]ed\n", " def __floordiv__(self,that):\n", " self.nest.append(self.cast(that)) ; return self" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Primitives" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "Frame\n", "\n", "Frame\n", "\n", "\n", "Prim\n", "\n", "Prim\n", "\n", "\n", "Frame->Prim\n", "\n", "\n", "\n", "\n", "Sym\n", "\n", "Sym\n", "\n", "\n", "Prim->Sym\n", "\n", "\n", "\n", "\n", "Str\n", "\n", "Str\n", "\n", "\n", "Prim->Str\n", "\n", "\n", "\n", "\n", "Num\n", "\n", "Num\n", "\n", "\n", "Prim->Num\n", "\n", "\n", "\n", "\n", "Int\n", "\n", "Int\n", "\n", "\n", "Num->Int\n", "\n", "\n", "\n", "\n", "Hex\n", "\n", "Hex\n", "\n", "\n", "Int->Hex\n", "\n", "\n", "\n", "\n", "Bin\n", "\n", "Bin\n", "\n", "\n", "Int->Bin\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "viz.edge('Frame','Prim')\n", "viz.edge('Prim','Sym')\n", "viz.edge('Prim','Str')\n", "viz.edge('Prim','Num')\n", "viz.edge('Num','Int')\n", "viz.edge('Int','Hex')\n", "viz.edge('Int','Bin')\n", "viz" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "class Prim(Frame): pass\n", "class Sym(Prim): pass\n", "class Str(Prim): pass\n", "class Num(Prim): pass\n", "class Int(Num): pass\n", "class Hex(Int): pass\n", "class Bin(Int): pass" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " id:7fa3a407f048\n", "\tself = id:7fa3a407f048 _/\n", "\tstring = id:7fa3a407f208\n", "\tsome = id:7fa3a4087fd0\n", "\t\tframe = id:7fa3a407f198\n", "\t0: id:7fa3a407f0b8\n", "\t1: id:7fa3a407f048 _/\n", "\t2: id:7fa3a407f128\n" ] }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "140340808380488\n", "\n", "frame:Hello\n", "\n", "\n", "140340808380488->140340808380488\n", "\n", "\n", "self\n", "\n", "\n", "140340808380488->140340808380488\n", "\n", "\n", "1\n", "\n", "\n", "140340808380936\n", "\n", "str:typecasted\n", "\n", "\n", "140340808380488->140340808380936\n", "\n", "\n", "string\n", "\n", "\n", "140340808417232\n", "\n", "frame:attribute\n", "\n", "\n", "140340808380488->140340808417232\n", "\n", "\n", "some\n", "\n", "\n", "140340808380600\n", "\n", "frame:World\n", "\n", "\n", "140340808380488->140340808380600\n", "\n", "\n", "0\n", "\n", "\n", "140340808380712\n", "\n", "str:string\n", "\n", "\n", "140340808380488->140340808380712\n", "\n", "\n", "2\n", "\n", "\n", "140340808380824\n", "\n", "frame:slot\n", "\n", "\n", "140340808417232->140340808380824\n", "\n", "\n", "frame\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hello = Frame('Hello') // Frame('World')\n", "hello['some'] = Frame('attribute') << Frame('slot')\n", "hello['self'] = hello // hello # self ring\n", "hello['string'] = 'typecasted' ; hello // 'string'\n", "print(hello)\n", "hello.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Picat object classes" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "Frame\n", "\n", "Frame\n", "\n", "\n", "Term\n", "\n", "Term\n", "\n", "\n", "Frame->Term\n", "\n", "\n", "\n", "\n", "Var\n", "\n", "Var\n", "\n", "\n", "Term->Var\n", "\n", "\n", "\n", "\n", "Atomic\n", "\n", "Atomic\n", "\n", "\n", "Term->Atomic\n", "\n", "\n", "\n", "\n", "Compound\n", "\n", "Compound\n", "\n", "\n", "Term->Compound\n", "\n", "\n", "\n", "\n", "Atom\n", "\n", "Atom\n", "\n", "\n", "Atomic->Atom\n", "\n", "\n", "\n", "\n", "Number\n", "\n", "Number\n", "\n", "\n", "Atomic->Number\n", "\n", "\n", "\n", "\n", "Integer\n", "\n", "Integer\n", "\n", "\n", "Number->Integer\n", "\n", "\n", "\n", "\n", "Real\n", "\n", "Real\n", "\n", "\n", "Number->Real\n", "\n", "\n", "\n", "\n", "List\n", "\n", "List\n", "\n", "\n", "Compound->List\n", "\n", "\n", "\n", "\n", "Struct\n", "\n", "Struct\n", "\n", "\n", "Compound->Struct\n", "\n", "\n", "\n", "\n", "String\n", "\n", "String\n", "\n", "\n", "List->String\n", "\n", "\n", "\n", "\n", "Array\n", "\n", "Array\n", "\n", "\n", "Struct->Array\n", "\n", "\n", "\n", "\n", "Map\n", "\n", "Map\n", "\n", "\n", "Struct->Map\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "picat = graphviz.Digraph(graph_attr={'rankdir':'LR'})\n", "picat.edge('Frame','Term')\n", "picat.edge('Term','Var') ; picat.edge('Term','Atomic') ; picat.edge('Atomic','Atom')\n", "picat.edge('Atomic','Number') ; picat.edge('Number','Integer') ; picat.edge('Number','Real')\n", "picat.edge('Term','Compound') ; picat.edge('Compound','List') ; picat.edge('List','String')\n", "picat.edge('Compound','Struct') ; picat.edge('Struct','Array') ; picat.edge('Struct','Map')\n", "picat" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Picat class tree is isolated from `metaL` class tree and have some aliases in atoms. It is done to follow Pical model as close as possible, and then map resulting structures to `metaL` classes the way like cross-model translation." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "class Term(Frame): pass\n", "class Var(Term): pass" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "class Atomic(Term): pass\n", "class Atom(Atomic): pass\n", "class Number(Atomic): pass\n", "class Integer(Number): pass\n", "class Real(Number): pass" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "class Compound(Term): pass\n", "class List(Compound): pass\n", "class String(List): pass\n", "class Struct(Compound): pass\n", "class Array(Struct): pass\n", "class Map(Struct): pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Parser" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "## Pycat syntax parser\n", "\n", "__file__ = 'pycat.ipynb' # required to run PLY in Jupyter\n", "\n", "import ply.lex as lex\n", "import ply.yacc as yacc\n", "\n", "# list of token types can be detected by the lexer\n", "# the token can be any object (Frame-inherited in our case) but \n", "# it must have set of required fields including .type\n", "\n", "tokens = ['var']\n", "\n", "t_ignore = ' \\t\\r'\n", "t_ignore_comment = '%.*'\n", "\n", "# end of lines (LF, line feed) must be counted obviously\n", "def t_lf(t):\n", " r'\\n+'\n", " t.lexer.lineno += len(t.value)\n", "\n", "def t_var(t):\n", " r'[A-Z][A-Za-z0-9_]*'\n", " return Var(t.value)\n", "\n", "def t_error(t):\n", " raise SyntaxError(t)\n", "\n", "lexer = lex.lex()" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " id:7fa3a407e828\n" ] } ], "source": [ "lexer.input('''\n", "\n", "% line comment\n", "\n", "Variable\n", "\n", "''')\n", "\n", "while True:\n", " token = lexer.token()\n", " if not token: break\n", " print(token)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.3" } }, "nbformat": 4, "nbformat_minor": 2 }