{ "cells": [ { "cell_type": "markdown", "id": "a4cdce42", "metadata": {}, "source": [ "# NAME\n", "\n", "InferenceUsingTFHubMobileNetV2Model - Using TensorFlow to do image classification using a pre-trained model" ] }, { "cell_type": "markdown", "id": "d74d6520", "metadata": {}, "source": [ "# SYNOPSIS\n", "\n", "The following tutorial is based on the [Image Classification with TensorFlow Hub notebook](https://github.com/tensorflow/docs/blob/master/site/en/hub/tutorials/image_classification.ipynb). It uses a pre-trained model based on the *MobileNet V2* architecture trained on the *Imagenet* dataset. Running the code requires an Internet connection to download the model (from Google servers) and testing data (from Wikimedia servers).\n", "\n", "Please look at the [SECURITY note](https://github.com/tensorflow/tensorflow/blob/master/SECURITY.md) regarding running models as models are programs. You can also used `saved_model_cli scan` to check for [security-sensitive \"denylisted ops\"](https://github.com/tensorflow/tensorflow/pull/17529) which by default are operations that do I/O.\n", "\n", "If you would like to visualise a model, you can use [Netron](https://github.com/lutzroeder/netron) on the `.pb` file.\n", "\n", "# COLOPHON\n", "\n", "The following document is either a POD file which can additionally be run as a Perl script or a Jupyter Notebook which can be run in [IPerl](https://p3rl.org/Devel::IPerl) (viewable online at [nbviewer](https://nbviewer.org/github/EntropyOrg/perl-AI-TensorFlow-Libtensorflow/blob/master/notebook/InferenceUsingTFHubMobileNetV2Model.ipynb)). If you are reading this as POD, there should be a generated list of Perl dependencies in the [CPANFILE](#CPANFILE) section.\n", "\n", "If you are running the code, you may optionally install the [`tensorflow` Python package](https://www.tensorflow.org/install/pip) in order to access the `saved_model_cli` command, but this is only used for informational purposes." ] }, { "cell_type": "markdown", "id": "bb098c5b", "metadata": {}, "source": [ "# TUTORIAL\n", "\n", "## Load the library\n", "\n", "First, we need to load the `AI::TensorFlow::Libtensorflow` library and more helpers. We then create an `AI::TensorFlow::Libtensorflow::Status` object and helper function to make sure that the calls to the `libtensorflow` C library are working properly." ] }, { "cell_type": "code", "execution_count": 1, "id": "81eaec3a", "metadata": {}, "outputs": [], "source": [ "use strict;\n", "use warnings;\n", "use utf8;\n", "use constant IN_IPERL => !! $ENV{PERL_IPERL_RUNNING};\n", "no if IN_IPERL, warnings => 'redefine'; # fewer messages when re-running cells\n", "\n", "use feature qw(say state);\n", "use Syntax::Construct qw(each-array);\n", "\n", "use lib::projectroot qw(lib);\n", "\n", "BEGIN {\n", " if( IN_IPERL ) {\n", " $ENV{TF_CPP_MIN_LOG_LEVEL} = 3;\n", " }\n", " require AI::TensorFlow::Libtensorflow;\n", "}\n", "\n", "use URI ();\n", "use HTTP::Tiny ();\n", "use Path::Tiny qw(path);\n", "\n", "use File::Which ();\n", "\n", "use List::Util ();\n", "\n", "use Data::Printer ( output => 'stderr', return_value => 'void', filters => ['PDL'] );\n", "use Data::Printer::Filter::PDL ();\n", "use Text::Table::Tiny qw(generate_table);\n", "\n", "use Imager;\n", "\n", "my $s = AI::TensorFlow::Libtensorflow::Status->New;\n", "sub AssertOK {\n", " die \"Status $_[0]: \" . $_[0]->Message\n", " unless $_[0]->GetCode == AI::TensorFlow::Libtensorflow::Status::OK;\n", " return;\n", "}\n", "AssertOK($s);" ] }, { "cell_type": "markdown", "id": "b99b9097", "metadata": {}, "source": [ "In this notebook, we will use `PDL` to store and manipulate the ndarray data before converting it to a `TFTensor`. The following functions help with copying the data back and forth between the two object types.\n", "\n", "An important thing to note about the dimensions used by TensorFlow's TFTensors when compared with PDL is that the dimension lists are reversed. Consider a binary raster image with width W and height H stored in [row-major format](https://en.wikipedia.org/wiki/Row-_and_column-major_order) (meaning the pixels in the first row are stored next to each other followed by the second row and so on). With PDL, the dimension list for this will be `[ W H ]` and for TFTensor the dimension list will be `[ H W ]`. TensorFlow uses the same convention for the dimension list as NumPy with the faster changing dimensions at the end of the dimension list while PDL is the opposite (see [Dima Kogan's library and talk](https://github.com/dkogan/numpysane) for more on this).\n", "\n", "This difference will be explained more concretely further in the tutorial.\n", "\n", "Future work will provide an API for more convenient wrappers which will provide an option to either copy or share the same data (if possible)." ] }, { "cell_type": "code", "execution_count": 2, "id": "3d52ab35", "metadata": {}, "outputs": [], "source": [ "use PDL;\n", "use AI::TensorFlow::Libtensorflow::DataType qw(FLOAT);\n", "\n", "use FFI::Platypus::Memory qw(memcpy);\n", "use FFI::Platypus::Buffer qw(scalar_to_pointer);\n", "\n", "sub FloatPDLTOTFTensor {\n", " my ($p) = @_;\n", " return AI::TensorFlow::Libtensorflow::Tensor->New(\n", " FLOAT, [ reverse $p->dims ], $p->get_dataref, sub { undef $p }\n", " );\n", "}\n", "\n", "sub FloatTFTensorToPDL {\n", " my ($t) = @_;\n", "\n", " my $pdl = zeros(float,reverse( map $t->Dim($_), 0..$t->NumDims-1 ) );\n", "\n", " memcpy scalar_to_pointer( ${$pdl->get_dataref} ),\n", " scalar_to_pointer( ${$t->Data} ),\n", " $t->ByteSize;\n", " $pdl->upd_data;\n", "\n", " $pdl;\n", "}" ] }, { "cell_type": "markdown", "id": "8c61da2f", "metadata": {}, "source": [ "The following is just a small helper to generate an HTML `