{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Python 扩展模块"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 简介"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"C Library | Interface | Python\n",
"---|---|---\n",
"`c header` <br> `c implementation` | Wrapper `C` $\\leftrightarrows$ `Python` <br> communication between `py + c` | `import fact` <br> `fact.fact(10)`\n",
"\n",
"**Python** 扩展模块将 `PyInt(10)` 转化为 `CInt(10)` 然后调用 `C` 程序中的 `fact()` 函数进行计算，再将返回的结果转换回 `PyInt`。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 产生一个扩展模块"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"假设我们有这样的一个头文件和程序："
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing fact.h\n"
]
}
],
"source": [
"%%file fact.h\n",
"#ifndef FACT_H\n",
"#define FACT_h\n",
"int fact(int n);\n",
"#endif"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing fact.c\n"
]
}
],
"source": [
"%%file fact.c\n",
"#include \"fact.h\"\n",
"int fact(int n)\n",
"{\n",
" if (n <= 1) return 1;\n",
" else return n * fact(n - 1);\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"定义包装函数："
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing fact_wrap.c\n"
]
}
],
"source": [
"%%file fact_wrap.c\n",
"\n",
"/* Must include Python.h before any standard headers*/\n",
"#include <Python.h>\n",
"#include \"fact.h\"\n",
"static PyObject* wrap_fact(PyObject *self, PyObject *args)\n",
"{\n",
" /* Python->C data conversion */\n",
" int n, result;\n",
" // the string i here means there is only one integer\n",
" if (!PyArg_ParseTuple(args, \"i\", &n))\n",
" return NULL;\n",
" \n",
" /* C Function Call */\n",
" result = fact(n);\n",
" \n",
" /* C->Python data conversion */\n",
" return Py_BuildValue(\"i\", result);\n",
"}\n",
"\n",
"/* Method table declaring the names of functions exposed to Python*/\n",
"static PyMethodDef ExampleMethods[] = {\n",
" {\"fact\", wrap_fact, METH_VARARGS, \"Calculate the factorial of n\"},\n",
" {NULL, NULL, 0, NULL} /* Sentinel */\n",
"};\n",
"\n",
"/* Module initialization function called at \"import example\"*/\n",
"PyMODINIT_FUNC \n",
"initexample(void)\n",
"{\n",
" (void) Py_InitModule(\"example\", ExampleMethods);\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 手动编译扩展模块"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"手动使用 `gcc` 编译，`Windows` 下如果没有 `gcc`，可以通过 `conda` 进行安装：\n",
"\n",
" conda install mingw4\n",
" \n",
"`Window 64-bit` 下编译需要加上 `-DMS_WIN64` 的选项，`include` 和 `lib` 文件夹的路径对应于本地 **Python** 安装的环境："
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"!gcc -DMS_WIN64 -c fact.c fact_wrap.c -IC:\\Miniconda\\include"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"!gcc -DMS_WIN64 -shared fact.o fact_wrap.o -LC:\\Miniconda\\libs -lpython27 -o example.pyd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`Windows` 下最终生成的文件后缀为 `.pyd` ， `Unix` 下生成的文件后缀名为 `.so`。\n",
"\n",
"用法为：\n",
"\n",
"- `Windows 32-bit`\n",
"```\n",
"gcc -c fact.c fact_wrap.c -I<your python path>\\include\n",
"gcc -shared fact.o fact_wrap.o -L<your python path>\\libs -lpython27 -o example.pyd\n",
"```\n",
"- `Unix`\n",
"```\n",
"gcc -c fact.c fact_wrap.c -I<your python path>\n",
"gcc -shared fact.o fact_wrap.o -L<your python path>\\config -lpython27 -o example.so\n",
"```\n",
"\n",
"编译完成后，我们就可以使用 `example` 这个模块了。\n",
"\n",
"导入生成的包："
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['__doc__', '__file__', '__name__', '__package__', 'fact']\n"
]
}
],
"source": [
"import example\n",
"print dir(example)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"使用 `example` 中的函数："
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"factorial of 10: 3628800\n"
]
}
],
"source": [
"print 'factorial of 10:', example.fact(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 使用 setup.py 进行编译"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"清理刚才生成的文件："
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"!rm -f example.pyd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"写入 `setup.py`："
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing setup.py\n"
]
}
],
"source": [
"%%file setup.py\n",
"from distutils.core import setup, Extension\n",
"\n",
"ext = Extension(name='example', sources=['fact_wrap.c', 'fact.c'])\n",
"\n",
"setup(name='example', ext_modules=[ext])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"使用 `distutils` 中的函数，我们进行 `build` 和 `install`：\n",
"\n",
" python setup.py build (--compiler=mingw64)\n",
" python setup.py install\n",
"\n",
"括号中的内容在 `windows` 中可能需要加上。\n",
"\n",
"这里我们使用 `build_ext --inplace` 选项将其安装在本地文件夹："
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"running build_ext\n",
"building 'example' extension\n",
"creating build\n",
"creating build\\temp.win-amd64-2.7\n",
"creating build\\temp.win-amd64-2.7\\Release\n",
"C:\\Miniconda\\Scripts\\gcc.bat -DMS_WIN64 -mdll -O -Wall -IC:\\Miniconda\\include -IC:\\Miniconda\\PC -c fact_wrap.c -o build\\temp.win-amd64-2.7\\Release\\fact_wrap.o\n",
"C:\\Miniconda\\Scripts\\gcc.bat -DMS_WIN64 -mdll -O -Wall -IC:\\Miniconda\\include -IC:\\Miniconda\\PC -c fact.c -o build\\temp.win-amd64-2.7\\Release\\fact.o\n",
"writing build\\temp.win-amd64-2.7\\Release\\example.def\n",
"C:\\Miniconda\\Scripts\\gcc.bat -DMS_WIN64 -shared -s build\\temp.win-amd64-2.7\\Release\\fact_wrap.o build\\temp.win-amd64-2.7\\Release\\fact.o build\\temp.win-amd64-2.7\\Release\\example.def -LC:\\Miniconda\\libs -LC:\\Miniconda\\PCbuild\\amd64 -lpython27 -lmsvcr90 -o \"C:\\Users\\Jin\\Documents\\Git\\python-tutorial\\07. interfacing with other languages\\example.pyd\"\n"
]
}
],
"source": [
"!python setup.py build_ext --inplace"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 使用编译的模块"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"进行测试："
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"factorial of 10: 3628800\n"
]
}
],
"source": [
"import example\n",
"\n",
"print 'factorial of 10:', example.fact(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"定义 `Python` 函数："
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3628800\n",
"3628800\n"
]
}
],
"source": [
"def pyfact(n):\n",
" if n <= 1: return 1\n",
" return n * pyfact(n-1)\n",
"\n",
"print pyfact(10)\n",
"print example.fact(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"时间测试："
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The slowest run took 13.17 times longer than the fastest. This could mean that an intermediate result is being cached \n",
"1000000 loops, best of 3: 213 ns per loop\n"
]
}
],
"source": [
"%timeit example.fact(10)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1000000 loops, best of 3: 1.43 µs per loop\n"
]
}
],
"source": [
"%timeit pyfact(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"如果使用 `fact` 计算比较大的值： "
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"example.fact(100)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"会出现溢出的结果，因为 `int` 表示的值有限，但是 `pyfact` 不会有这样的问题："
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000L"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pyfact(100)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"将生成的文件压缩到压缩文件中："
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import zipfile\n",
"\n",
"f = zipfile.ZipFile('07-02-example.zip','w',zipfile.ZIP_DEFLATED)\n",
"\n",
"names = 'fact.o fact_wrap.c fact_wrap.o example.pyd setup.py'.split()\n",
"for name in names:\n",
" f.write(name)\n",
"\n",
"f.close()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"清理生成的文件："
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"!rm -f fact*.*\n",
"!rm -f example.*\n",
"!rm -f setup*.*\n",
"!rm -rf build"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}