{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Формулировка проблемы\n", "\n", "Low-level wrapper для Python должен являться **биндингом** имеющегося API в Питон, учитывающим паттерны программирования в Питоне. При этом он не должен являться самостоятельной библиотекой, требующей поддержки и диктующей свои правила, как это происходит с существующий Low-level Python API.\n", "\n", "В идеале:\n", " - человек по [документации C-API](http://docs.bigartm.org/en/latest/ref/c_interface.html) должен без дополнительных знаний брать и пользоваться оберткой в питоне\n", " - при изменении C-API необходимо мимнимум правок в Python-обертку\n", " - у пользователя не должно быть попоболи от жутких конструкций, не свойственных Python\n", " \n", "Предлагаемое решение:\n", "Обертка C-API для питона должна генерироваться автоматчиески по достаточно простому описанию. Обертка выполняется в виде одного небольшого модуля (не более 1к строк, лучше — меньше), содержащего класс `LibArtm`. \n", "\n", "На основе Low-level wrapper будет строиться удобная python-библиотека для тематического моделирования (реализация `ArtmModel`)." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ARTM_LIBRARY_PATH = '/home/romovpa/bigartm/build/src/artm/libartm.so'" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import ctypes\n", "from artm import messages_pb2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Сообщения об ошибках" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ARTM_SUCCESS = 0\n", "ARTM_STILL_WORKING = -1\n", "\n", "class ArtmException(BaseException): pass\n", "\n", "class InternalError(ArtmException): pass\n", "class ArgumentOutOfRangeException(ArtmException): pass\n", "class InvalidMasterIdException(ArtmException): pass\n", "class CorruptedMessageException(ArtmException): pass\n", "class InvalidOperationException(ArtmException): pass\n", "class DiskReadException(ArtmException): pass\n", "class DiskWriteException(ArtmException): pass\n", "\n", "ARTM_EXCEPTION_BY_CODE = {\n", " -2: InternalError,\n", " -3: ArgumentOutOfRangeException,\n", " -4: InvalidMasterIdException,\n", " -5: CorruptedMessageException,\n", " -6: InvalidOperationException,\n", " -7: DiskReadException,\n", " -8: DiskWriteException,\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Обертка для удобного вызова низкоуровнего API" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import ctypes\n", "from google import protobuf\n", "from artm import messages_pb2\n", "\n", "class LibArtm(object):\n", " \n", " def __init__(self, lib_name):\n", " self.cdll = ctypes.CDLL(lib_name)\n", " \n", " def __getattr__(self, name):\n", " func = getattr(self.cdll, name)\n", " if func is None:\n", " raise AttributeError('%s is not a function of libartm' % name)\n", " return self._wrap_call(func)\n", " \n", " def _check_error(self, error_code):\n", " if error_code < -1:\n", " lib.cdll.ArtmGetLastErrorMessage.restype = ctypes.c_char_p\n", " error_message = lib.cdll.ArtmGetLastErrorMessage()\n", " \n", " # remove exception name from error message\n", " error_message = error_message.split(':', 1)[-1].strip()\n", " \n", " exception_class = ARTM_EXCEPTION_BY_CODE.get(error_code)\n", " if exception_class is not None:\n", " raise exception_class(error_message)\n", " else:\n", " raise RuntimeError(error_message)\n", " \n", " def _wrap_call(self, func):\n", " def artm_api_call(*args):\n", " cargs = []\n", " for arg in args:\n", " if isinstance(arg, basestring):\n", " arg_cstr_p = ctypes.create_string_buffer(arg)\n", " cargs.append(arg_cstr_p)\n", " \n", " elif isinstance(arg, protobuf.message.Message):\n", " message_str = arg.SerializeToString()\n", " message_cstr_p = ctypes.create_string_buffer(message_str)\n", " cargs += [len(message_str), message_cstr_p]\n", " \n", " else:\n", " cargs.append(arg)\n", " \n", " result = func(*cargs)\n", " self._check_error(result)\n", " return result\n", " \n", " return artm_api_call\n", " \n", " def _copy_request_result(self, length):\n", " message_blob = ctypes.create_string_buffer(length)\n", " error_code = self.lib_.ArtmCopyRequestResult(length, message_blob)\n", " self._check_error(error_code)\n", " return message_blob\n", " " ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [], "source": [ "lib = LibArtm(ARTM_LIBRARY_PATH)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [], "source": [ "config = messages_pb2.MasterComponentConfig()\n", "config.processors_count = -1\n", "\n", "master_id = lib.ArtmCreateMasterComponent(config)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lib.ArtmCreateModel(master_id, messages_pb2.ModelConfig())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Спецификация API и автоматическая генерация обертки\n", "\n", "Модификация обертки, приведенной выше, решает следующие задачи:\n", "\n", " - Автоматическа проверка аргументов\n", " - Осмысленное возвращаемое значение: \n", " - `ArtmCreateMasterComponent` возвращает `int`\n", " - функции вроде `ArtmRequestThetaMatrix` возвращают сообщение\n", " - функции без возвращаемого знвчения возвращают `None` (вместо `ARTM_SUCCESS`)\n", " - Автодокументирование: при вызове справки, отображается нормальная спецификация функции\n", " - Быстрое создание сообщений: вместо сообщения можно передать `dict`, который умным образом преобразуется в соответствующее сообщение (гораздо удобнее в питоне)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Вводим элементы языка, которыми будет описано API" ] }, { "cell_type": "code", "execution_count": 72, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class CallSpec(object):\n", " def __init__(self, name, arguments, result=None, request=None):\n", " self.name = name\n", " self.arguments = arguments\n", " self.result_type = result\n", " self.request_type = request" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Описываем API при помощи специального dsl\n", "\n", "Список `ARTM_API` нужно будет редактировать при изменении low-level API. Заметьте: необходимо будет минимальное число правок, в отличие от существующего `artm.library`.\n", "\n", "Некоторые функции из API объявляются _специальными_, для них по понятной причине не делается обертка:\n", " - `ArtmGetLastErrorMessage`\n", " - `ArtmCopyRequestResult`" ] }, { "cell_type": "code", "execution_count": 77, "metadata": { "collapsed": false }, "outputs": [], "source": [ "ARTM_API = [\n", " CallSpec(\n", " 'ArtmCreateMasterComponent', \n", " [('config', messages_pb2.MasterComponentConfig)],\n", " result=ctypes.c_int,\n", " ),\n", " CallSpec(\n", " 'ArtmReconfigureMasterComponent',\n", " [('master_id', int), ('config', messages_pb2.MasterComponentConfig)],\n", " ),\n", " CallSpec(\n", " 'ArtmDisposeMasterComponent',\n", " [('master_id', int)],\n", " ),\n", " CallSpec(\n", " 'ArtmCreateModel',\n", " [('master_id', int), ('config', messages_pb2.ModelConfig)],\n", " ),\n", " CallSpec(\n", " 'ArtmReconfigureModel',\n", " [('master_id', int), ('config', messages_pb2.ModelConfig)],\n", " ),\n", " CallSpec(\n", " 'ArtmDisposeModel',\n", " [('master_id', int), ('name', str)],\n", " ),\n", " CallSpec(\n", " 'ArtmCreateRegularizer',\n", " [('master_id', int), ('config', messages_pb2.RegularizerConfig)],\n", " ),\n", " CallSpec(\n", " 'ArtmReconfigureRegularizer',\n", " [('master_id', int), ('config', messages_pb2.RegularizerConfig)],\n", " ),\n", " CallSpec(\n", " 'ArtmDisposeRegularizer',\n", " [('master_id', int), ('name', str)],\n", " ),\n", " CallSpec(\n", " 'ArtmCreateDictionary',\n", " [('master_id', int), ('config', messages_pb2.DictionaryConfig)],\n", " ),\n", " CallSpec(\n", " 'ArtmReconfigureDictionary',\n", " [('master_id', int), ('config', messages_pb2.DictionaryConfig)],\n", " ),\n", " CallSpec(\n", " 'ArtmDisposeDictionary',\n", " [('master_id', int), ('name', str)],\n", " ),\n", " CallSpec(\n", " 'ArtmAddBatch',\n", " [('master_id', int), ('args', messages_pb2.AddBatchArgs)],\n", " ),\n", " CallSpec(\n", " 'ArtmInvokeIteration',\n", " [('master_id', int), ('args', messages_pb2.InvokeIterationArgs)],\n", " ),\n", " CallSpec(\n", " 'ArtmSynchronizeModel',\n", " [('master_id', int), ('args', messages_pb2.SynchronizeModelArgs)],\n", " ),\n", " CallSpec(\n", " 'ArtmInitializeModel',\n", " [('master_id', int), ('args', messages_pb2.InitializeModelArgs)],\n", " ),\n", " CallSpec(\n", " 'ArtmExportModel',\n", " [('master_id', int), ('args', messages_pb2.ExportModelArgs)],\n", " ),\n", " CallSpec(\n", " 'ArtmImportModel',\n", " [('master_id', int), ('args', messages_pb2.ImportModelArgs)],\n", " ),\n", " CallSpec(\n", " 'ArtmWaitIdle',\n", " [('master_id', int), ('args', messages_pb2.WaitIdleArgs)],\n", " ),\n", " CallSpec(\n", " 'ArtmOverwriteTopicModel',\n", " [('master_id', int), ('model', messages_pb2.TopicModel)],\n", " ),\n", " CallSpec(\n", " 'ArtmRequestThetaMatrix',\n", " [('master_id', int), ('args', messages_pb2.GetThetaMatrixArgs)],\n", " request=messages_pb2.ThetaMatrix,\n", " ),\n", " CallSpec(\n", " 'ArtmRequestTopicModel',\n", " [('master_id', int), ('args', messages_pb2.GetTopicModelArgs)],\n", " request=messages_pb2.TopicModel,\n", " ),\n", " CallSpec(\n", " 'ArtmRequestRegularizerState',\n", " [('master_id', int), ('name', str)],\n", " request=messages_pb2.RegularizerInternalState,\n", " ),\n", " CallSpec(\n", " 'ArtmRequestScore',\n", " [('master_id', int), ('args', messages_pb2.GetScoreValueArgs)],\n", " request=messages_pb2.ScoreData,\n", " ),\n", " CallSpec(\n", " 'ArtmRequestParseCollection',\n", " [('args', messages_pb2.CollectionParserConfig)],\n", " request=messages_pb2.DictionaryConfig,\n", " ),\n", " CallSpec(\n", " 'ArtmRequestLoadDictionary',\n", " [('filename', str)],\n", " request=messages_pb2.DictionaryConfig,\n", " ),\n", " CallSpec(\n", " 'ArtmRequestLoadBatch',\n", " [('filename', str)],\n", " request=messages_pb2.Batch,\n", " ),\n", " CallSpec(\n", " 'ArtmSaveBatch',\n", " [('filename', str), ('batch', messages_pb2.Batch)],\n", " ),\n", " CallSpec(\n", " 'ArtmSaveBatch',\n", " [('filename', str), ('batch', messages_pb2.Batch)],\n", " ),\n", "]" ] }, { "cell_type": "code", "execution_count": 78, "metadata": { "collapsed": false }, "outputs": [], "source": [ "spec = CallSpec(\n", " 'ArtmRequestScore',\n", " [('master_id', int), ('args', messages_pb2.GetScoreValueArgs)],\n", " messages_pb2.ScoreData,\n", " request=True,\n", " )" ] }, { "cell_type": "code", "execution_count": 79, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "[('master_id', int), ('args', artm.messages_pb2.GetScoreValueArgs)]" ] }, "execution_count": 79, "metadata": {}, "output_type": "execute_result" } ], "source": [ "spec.arguments" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Конвертация dict -> protobuf message" ] }, { "cell_type": "code", "execution_count": 80, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from types import StringTypes\n", "import logging\n", "\n", "def dict_to_message(record, message_type):\n", " \"\"\"Convert dict to protobuf message\"\"\"\n", " \n", " def parse_list(values, message):\n", " if isinstance(values[0], dict):\n", " for v in values:\n", " cmd = message.add()\n", " parse_dict(v,cmd)\n", " else:\n", " message.extend(values)\n", "\n", " def parse_dict(values, message):\n", " for k, v in values.iteritems():\n", " if isinstance(v, dict):\n", " parse_dict(v, getattr(message, k))\n", " elif isinstance(v, list):\n", " parse_list(v, getattr(message, k))\n", " else:\n", " try:\n", " setattr(message, k, v)\n", " except AttributeError:\n", " raise TypeError('Cannot convert dict to protobuf message {message_type}: bad field \"{field}\"'.format(\n", " message_type=str(message_type),\n", " field=k,\n", " ))\n", " \n", " message = message_type()\n", " parse_dict(record, message)\n", " \n", " return message" ] }, { "cell_type": "code", "execution_count": 81, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "'0\\n8{'" ] }, "execution_count": 81, "metadata": {}, "output_type": "execute_result" } ], "source": [ "config = messages_pb2.MasterComponentConfig()\n", "config.processor_queue_max_size = 123\n", "config.processors_count = 10\n", "config.SerializeToString()" ] }, { "cell_type": "code", "execution_count": 82, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "'0\\n8{'" ] }, "execution_count": 82, "metadata": {}, "output_type": "execute_result" } ], "source": [ "config = dict_to_message({'processor_queue_max_size': 123, 'processors_count': 10}, messages_pb2.MasterComponentConfig)\n", "config.SerializeToString()" ] }, { "cell_type": "code", "execution_count": 83, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 83, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dict_to_message({}, messages_pb2.MasterComponentConfig)" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "collapsed": false }, "outputs": [ { "ename": "TypeError", "evalue": "Cannot convert dict to protobuf message : bad field \"sdfsdfsdf\"", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mTypeError\u001b[0m Traceback (most recent call last)", "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mdict_to_message\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m{\u001b[0m\u001b[1;34m'sdfsdfsdf'\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;36m24134\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmessages_pb2\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMasterComponentConfig\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[1;32m\u001b[0m in \u001b[0;36mdict_to_message\u001b[1;34m(record, message_type)\u001b[0m\n\u001b[0;32m 29\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 30\u001b[0m \u001b[0mmessage\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmessage_type\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 31\u001b[1;33m \u001b[0mparse_dict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mrecord\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 32\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 33\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mmessage\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;32m\u001b[0m in \u001b[0;36mparse_dict\u001b[1;34m(values, message)\u001b[0m\n\u001b[0;32m 25\u001b[0m raise TypeError('Cannot convert dict to protobuf message {message_type}: bad field \"{field}\"'.format(\n\u001b[0;32m 26\u001b[0m \u001b[0mmessage_type\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mstr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmessage_type\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 27\u001b[1;33m \u001b[0mfield\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 28\u001b[0m ))\n\u001b[0;32m 29\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;31mTypeError\u001b[0m: Cannot convert dict to protobuf message : bad field \"sdfsdfsdf\"" ] } ], "source": [ "dict_to_message({'sdfsdfsdf': 24134}, messages_pb2.MasterComponentConfig)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Объект класса `LibArtm` при помощи `__getattr__` автоматически создает обертки над вызовами API" ] }, { "cell_type": "code", "execution_count": 85, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import ctypes\n", "from google import protobuf\n", "from artm import messages_pb2\n", "\n", "class LibArtm(object):\n", "\n", " def __init__(self, lib_name):\n", " self.cdll = ctypes.CDLL(lib_name)\n", " self._spec_by_name = {spec.name: spec for spec in ARTM_API}\n", " \n", " def __getattr__(self, name):\n", " spec = self._spec_by_name.get(name)\n", " if spec is None:\n", " raise AttributeError('%s is not a function of libartm' % name)\n", " func = getattr(self.cdll, name)\n", " return self._wrap_call(func, spec)\n", " \n", " def _check_error(self, error_code):\n", " if error_code < -1:\n", " lib.cdll.ArtmGetLastErrorMessage.restype = ctypes.c_char_p\n", " error_message = lib.cdll.ArtmGetLastErrorMessage()\n", " \n", " # remove exception name from error message\n", " error_message = error_message.split(':', 1)[-1].strip()\n", " \n", " exception_class = ARTM_EXCEPTION_BY_CODE.get(error_code)\n", " if exception_class is not None:\n", " raise exception_class(error_message)\n", " else:\n", " raise RuntimeError(error_message)\n", "\n", " def _copy_request_result(self, length):\n", " message_blob = ctypes.create_string_buffer(length)\n", " error_code = self.lib_.ArtmCopyRequestResult(length, message_blob)\n", " self._check_error(error_code)\n", " return message_blob\n", "\n", " def _wrap_call(self, func, spec):\n", " \n", " def artm_api_call(*args):\n", " # check the number of arguments\n", " n_args_given = len(args)\n", " n_args_takes = len(spec.arguments)\n", " if n_args_given != n_args_takes:\n", " raise TypeError('{func_name} takes {n_takes} argument ({n_given} given)'.format(\n", " func_name=spec.name,\n", " n_takes=n_args_takes,\n", " n_given=n_args_given,\n", " ))\n", " \n", " cargs = []\n", " for (arg_index, arg), (arg_name, arg_type) in zip(enumerate(args), spec.arguments):\n", " # try to cast argument to the required type\n", " arg_casted = arg\n", " if issubclass(arg_type, protobuf.message.Message) and isinstance(arg, dict):\n", " # dict -> protobuf message\n", " arg_casted = dict_to_message(arg, arg_type)\n", " \n", " # check argument type\n", " if not isinstance(arg_casted, arg_type):\n", " raise TypeError('Argument {arg_index} ({arg_name}) should have type {arg_type} but {given_type} given'.format(\n", " arg_index=arg_index,\n", " arg_name=arg_name,\n", " arg_type=str(arg_type),\n", " given_type=str(type(arg)),\n", " ))\n", " arg = arg_casted\n", " \n", " # construct c-style arguments \n", " if issubclass(arg_type, basestring):\n", " arg_cstr_p = ctypes.create_string_buffer(arg)\n", " cargs.append(arg_cstr_p)\n", " \n", " elif issubclass(arg_type, protobuf.message.Message):\n", " message_str = arg.SerializeToString()\n", " message_cstr_p = ctypes.create_string_buffer(message_str)\n", " cargs += [len(message_str), message_cstr_p]\n", " \n", " else:\n", " cargs.append(arg)\n", " \n", " # make api call\n", " if spec.result_type is not None:\n", " func.restype = spec.result_type\n", " result = func(*cargs)\n", " self._check_error(result)\n", " \n", " # return result value\n", " if spec.request_type is not None:\n", " return self._copy_request_result(length=result)\n", " if spec.result_type is not None:\n", " return result\n", " \n", " return artm_api_call\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Тестирование" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Создаем объект `LibArtm`" ] }, { "cell_type": "code", "execution_count": 86, "metadata": { "collapsed": true }, "outputs": [], "source": [ "lib = LibArtm(ARTM_LIBRARY_PATH)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Создаем мастер" ] }, { "cell_type": "code", "execution_count": 87, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "({'cache_theta': True, 'processors_count': 10},)\n", "[('config', )]\n", "[((0, {'cache_theta': True, 'processors_count': 10}), ('config', ))]\n" ] }, { "data": { "text/plain": [ "6" ] }, "execution_count": 87, "metadata": {}, "output_type": "execute_result" } ], "source": [ "master_id = lib.ArtmCreateMasterComponent({'cache_theta': True, 'processors_count': 10})\n", "master_id" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Забыл передать аргумент `master_id`" ] }, { "cell_type": "code", "execution_count": 88, "metadata": { "collapsed": false }, "outputs": [ { "ename": "TypeError", "evalue": "ArtmCreateModel takes 2 argument (1 given)", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mTypeError\u001b[0m Traceback (most recent call last)", "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mlib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mArtmCreateModel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m{\u001b[0m\u001b[1;34m'topics_count'\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;36m20\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'class_id'\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;34m'words'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'labels'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'class_weight'\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m0.2\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[1;32m\u001b[0m in \u001b[0;36martm_api_call\u001b[1;34m(*args)\u001b[0m\n\u001b[0;32m 46\u001b[0m \u001b[0mfunc_name\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mspec\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 47\u001b[0m \u001b[0mn_takes\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mn_args_takes\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 48\u001b[1;33m \u001b[0mn_given\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mn_args_given\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 49\u001b[0m ))\n\u001b[0;32m 50\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;31mTypeError\u001b[0m: ArtmCreateModel takes 2 argument (1 given)" ] } ], "source": [ "lib.ArtmCreateModel({'topics_count': 20, 'class_id': ['words', 'labels'], 'class_weight': [1, 0.2]})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Исправился, создал модель" ] }, { "cell_type": "code", "execution_count": 89, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(6, {'class_id': ['words', 'labels'], 'topics_count': 20, 'class_weight': [1, 0.2]})\n", "[('master_id', ), ('config', )]\n", "[((0, 6), ('master_id', )), ((1, {'class_id': ['words', 'labels'], 'topics_count': 20, 'class_weight': [1, 0.2]}), ('config', ))]\n" ] } ], "source": [ "lib.ArtmCreateModel(master_id, {'topics_count': 20, 'class_id': ['words', 'labels'], 'class_weight': [1, 0.2]})" ] }, { "cell_type": "code", "execution_count": 90, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(6, {})\n", "[('master_id', ), ('args', )]\n", "[((0, 6), ('master_id', )), ((1, {}), ('args', ))]\n" ] }, { "ename": "InvalidOperationException", "evalue": "Topic model does not exist", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mInvalidOperationException\u001b[0m Traceback (most recent call last)", "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mlib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mArtmRequestTopicModel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmaster_id\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m{\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[1;32m\u001b[0m in \u001b[0;36martm_api_call\u001b[1;34m(*args)\u001b[0m\n\u001b[0;32m 87\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrestype\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mspec\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mresult_type\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 88\u001b[0m \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0mcargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 89\u001b[1;33m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_check_error\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 90\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 91\u001b[0m \u001b[1;31m# return result value\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;32m\u001b[0m in \u001b[0;36m_check_error\u001b[1;34m(self, error_code)\u001b[0m\n\u001b[0;32m 26\u001b[0m \u001b[0mexception_class\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mARTM_EXCEPTION_BY_CODE\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0merror_code\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 27\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mexception_class\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 28\u001b[1;33m \u001b[1;32mraise\u001b[0m \u001b[0mexception_class\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0merror_message\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 29\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 30\u001b[0m \u001b[1;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0merror_message\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;31mInvalidOperationException\u001b[0m: Topic model does not exist" ] } ], "source": [ "lib.ArtmRequestTopicModel(master_id, {})" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.3" } }, "nbformat": 4, "nbformat_minor": 0 }