{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import io\n", "import panel as pn\n", "pn.extension()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The ``FileInput`` widget allows uploading one or more file from the frontend and makes the filename, file data and [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types) available in Python.\n", "\n", "For more information about listening to widget events and laying out widgets refer to the [widgets user guide](../../user_guide/Widgets.ipynb). Alternatively you can learn how to build GUIs by declaring parameters independently of any specific widgets in the [param user guide](../../user_guide/Param.ipynb). To express interactivity entirely using Javascript without the need for a Python server take a look at the [links user guide](../../user_guide/Param.ipynb).\n", "\n", "#### Parameters:\n", "\n", "For layout and styling related parameters see the [customization user guide](../../user_guide/Customization.ipynb).\n", "\n", "##### Core\n", "\n", "* **``accept``** (str): A list of file input filters that restrict what files the user can pick from\n", "* **``filename``** (str/list): The filename(s) of the uploaded file(s)\n", "* **``mime_type``** (str/list): The mime type(s) of the uploaded file(s)\n", "* **``multiple``** (boolean): Whether to allow uploading multiple files\n", "* **``value``** (bytes/list): A bytes object containing the file data or if `multiple` is set a list of bytes objects.\n", "\n", "___" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "file_input = pn.widgets.FileInput()\n", "\n", "file_input" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To read out the content of the file you can access the ``value`` parameter, which holds a [bytestring](https://docs.python.org/3/library/stdtypes.html#bytes-objects) containing the file's contents. Additionally information about the file type is made available on the ``filetype`` parameter expressed as a MIME type, e.g. ``image/png`` or ``text/csv``." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "file_input.value" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The widget also has a ``save`` method that allows saving the uploaded data to a file or [BytesIO](https://docs.python.org/3/library/io.html#binary-i-o) object." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# File\n", "if file_input.value is not None:\n", " file_input.save('test.png')\n", "\n", "# BytesIO object\n", "if file_input.value is not None:\n", " out = io.BytesIO()\n", " file_input.save(out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `accept` parameter restricts what files the user can pick from. This consists of comma-separated list of standard HTML\n", "file input filters. Values can be:\n", "\n", "* `` - Specific file extension(s) (e.g: .gif, .jpg, .png, .doc) are pickable\n", "* `audio/*` - all sound files are pickable\n", "* `video/*` - all video files are pickable\n", "* `image/*` - all image files are pickable\n", "* `` - A valid [IANA Media Type](https://www.iana.org/assignments/media-types/media-types.xhtml), with no parameters." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "file_input = pn.widgets.FileInput(accept='.csv,.json')\n", "\n", "file_input" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To allow uploading multiple files we can also set `multiple=True`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "file_input = pn.widgets.FileInput(accept='.png', multiple=True)\n", "\n", "file_input" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When uploading one or more files the `filename`, `mime_type` and `value` parameters will now be lists. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Upload size limits\n", "\n", "While the `FileInput` widget doesn't set any limit on the size of a file that can be selected by a user, the infrastructure onto which Panel relies (web browsers, Bokeh, Tornado, notebooks, etc.) limits significantly what is actually possible. By default the `FileInput` widget allows to upload data that is in the order of 10 MB. Even if it is possible to increase this limit by setting some parameters (described below), bear in mind that the `FileInput` widget is not meant to upload large files.\n", "\n", "#### How it works\n", "\n", "Before increasing the file size limit it is worth explaining the process that happens when a file is selected.\n", "\n", "The `FileInput` widget is a Bokeh widget whose data is communicated from the front-end to the back-end via a protocal called [Web Sockets](https://en.wikipedia.org/wiki/WebSocket). Bokeh didn't implement Web Sockets itself, instead it took advantage of an existing web framework that provided an implementation: [Tornado](https://www.tornadoweb.org/en/stable/).\n", "\n", "In even more concrete terms, here's what happens when a file is selected in a server context (it's the exact same process when multiple files are loaded at once!):\n", "\n", "1. The file is loaded into memory by the browser\n", "2. Its content is converted by BokehJS into a [base64 encoded string](https://en.wikipedia.org/wiki/Base64) (which turns a binary file, like a PNG image, into a very long ASCII string)\n", "3. BokehJS puts this long string into a JSON message along with some more information\n", "4. The message is sent through a `Tornado` web socket connection to the back-end\n", "5. The back-end uses it to update the Bokeh/Python model; that's when the properties of the widget in the Bokeh/Python world get updated\n", "6. Panel updates the attributes of the `FileInput` widget instance when the properties of the Python/Bokeh widget are updated. The long string is converted by Panel into a `bytes` object that is available with `.value`.\n", "\n", "#### Limits defined\n", "\n", "The steps described above almost all have potential or actual limits:\n", "\n", "1. Browsers can't upload an unlimited amount of data at once; they're usually limited to a few GB\n", "2. BokehJS uses the [FileReader.readAsDataURL](https://developer.mozilla.org/en-US/docs/Web/API/FileReader/readAsDataURL) method to encode the file as a data URL; browsers can [limit](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs#common_problems) the length of that URL\n", "3. BokehJS (or the libraries that it uses) may fail at creating the message from a very (very!) large string\n", "4. `Tornado` imposes two limits on the data being transferred: (1) on the maximum size of a buffer and (2) on the maximum size of the websocket message. (1) is 100 MB by default (set by Tornado) and (2) is 20 MB by default (set by Bokeh).\n", "\n", "#### Increase the limits\n", "\n", "While there's nothing much that can be done about 1., 2. and 3. (except informing your users), the limits defined by Tornado and Bokeh can be overwritten.\n", "\n", "##### Server context\n", "\n", "In a server context your application must be executed with `python your_app.py` (because `panel serve` doesn't allow to configure all the options provided by Bokeh and Tornado):\n", "\n", "```python\n", "# your_app.py\n", "import panel as pn\n", "\n", "app = ...\n", "\n", "MAX_SIZE_MB = 150\n", "\n", "pn.serve(\n", " app,\n", " # Increase the maximum websocket message size allowed by Bokeh\n", " websocket_max_message_size=MAX_SIZE_MB*1024*1014,\n", " # Increase the maximum buffer size allowed by Tornado\n", " http_server_kwargs={'max_buffer_size': MAX_SIZE_MB*1024*1014}\n", ")\n", "```\n", "\n", "##### Notebook context\n", "\n", "In a Jupyter notebook (classic or lab) the limits of Tornado (Tornado's Web Sockets are already used by the Jupyter notebook for communication purposes) can be set in a configuration file. The default maximum buffer size is 512 MB and the default maximum websocked message size is 10 MB.\n", "\n", "*Classic Notebook:*\n", "\n", "Generate a configuration file with `jupyter notebook --generate-config` and update it with:\n", "\n", "```python\n", "c.NotebookApp.tornado_settings = {\"websocket_max_message_size\": 150 * 1024 * 1024}\n", "c.NotebookApp.max_buffer_size = 150 * 1024 * 1024\n", "```\n", "\n", "*Lab:*\n", "\n", "Generate a configuration file with `jupyter lab --generate-config` and update it with:\n", "\n", "```python\n", "c.ServerApp.tornado_settings = {'websocket_max_message_size': 150 * 1024 * 1024}\n", "c.ServerApp.max_buffer_size = 150 * 1024 * 1024\n", "```\n", "\n", "#### Caveats\n", "\n", "* The maximum sizes set in either Bokeh or Tornado refer to the maximum size of the **message** that is transferred through the web socket connection, which is going to be larger than the actual size of the uploaded file since the file content is encoded in a `base64` string. So if you set a maximum size of 100 MB for your application, you should indicate your users that the upload limit is a value that is **less** than 100 MB.\n", "* When a file whose size is larger than the limits is selected by a user, their browser/tab may just crash. Alternatively the web socket connection can close (sometimes with an error message printed in the browser console such as `[bokeh] Lost websocket 0 connection, 1009 (message too big)`) which means the application will become unresponsive and needs to be refreshed. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Controls\n", "\n", "The `FileInput` widget exposes a number of options which can be changed from both Python and Javascript. Try out the effect of these parameters interactively:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pn.Row(file_input.controls(jslink=True), file_input)" ] } ], "metadata": { "language_info": { "name": "python", "pygments_lexer": "ipython3" } }, "nbformat": 4, "nbformat_minor": 4 }