.. TensorFlowModel: TensorFlowModel *************** See `API Documentation <../../../ads.model.framework.html#ads.model.framework.tensorflow_model.TensorFlowModel>`__ Overview ======== The ``ads.model.framework.tensorflow_model.TensorFlowModel`` class in ADS is designed to allow you to rapidly get a TensorFlow model into production. The ``.prepare()`` method creates the model artifacts that are needed to deploy a functioning model without you having to configure it or write code. However, you can customize the required ``score.py`` file. .. include:: ../_template/overview.rst The following steps take your trained ``TensorFlow`` model and deploy it into production with a few lines of code. **Create a TensorFlow Model** .. code-block:: python3 import tensorflow as tf mnist = tf.keras.datasets.mnist (trainx, trainy), (testx, testy) = mnist.load_data() trainx, testx = trainx / 255.0, testx / 255.0 model = tf.keras.models.Sequential( [ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(128, activation="relu"), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10), ]) loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"]) model.fit(trainx, trainy, epochs=1) Prepare Model Artifact ====================== .. code-block:: python3 from ads.common.model_metadata import UseCaseType from ads.model.framework.tensorflow_model import TensorFlowModel from uuid import uuid4 tensorflow_model = TensorFlowModel(estimator=model, artifact_dir=f"./model-artifact-{str(uuid4())}") tensorflow_model.prepare( inference_conda_env="tensorflow28_p38_cpu_v1", training_conda_env="tensorflow28_p38_cpu_v1", X_sample=trainx, y_sample=trainy, use_case_type=UseCaseType.MULTINOMIAL_CLASSIFICATION, ) Instantiate a ``ads.model.framework.tensorflow_model.TensorFlowModel()`` object with a TensorFlow model. Each instance accepts the following parameters: * ``artifact_dir: str``: Artifact directory to store the files needed for deployment. * ``auth: (Dict, optional)``: Defaults to ``None``. The default authentication is set using the ``ads.set_auth`` API. To override the default, use ``ads.common.auth.api_keys()`` or ``ads.common.auth.resource_principal()`` and create the appropriate authentication signer and the ``**kwargs`` required to instantiate the ``IdentityClient`` object. * ``estimator: Callable``: Any model object generated by the TensorFlow framework. * ``properties: (ModelProperties, optional)``: Defaults to ``None``. The ``ModelProperties`` object required to save and deploy a model. .. include:: ../_template/initialize.rst Summary Status ============== .. include:: ../_template/summary_status.rst .. figure:: ../figures/summary_status.png :align: center In ``TensorFlowModel``, data serialization is supported for JSON serializable objects. Plus, there is support for a dictionary, string, list, ``np.ndarray``, and ``tf.python.framework.ops.EagerTensor``. Not all these objects are JSON serializable, however, support to automatically serializes and deserialized is provided. Register Model ============== .. code-block:: python3 >>> # Register the model >>> model_id = tensorflow_model.save() Start loading model.h5 from model directory /tmp/tmpapjjzeol ... Model is successfully loaded. ['runtime.yaml', 'model.h5', 'score.py'] 'ocid1.datasciencemodel.oc1.xxx.xxxxx' Deploy and Generate Endpoint ============================ .. code-block:: python3 # Deploy and create an endpoint for the TensorFlow model tensorflow_model.deploy( display_name="TensorFlow Model For Classification", deployment_log_group_id="ocid1.loggroup.oc1.xxx.xxxxx", deployment_access_log_id="ocid1.log.oc1.xxx.xxxxx", deployment_predict_log_id="ocid1.log.oc1.xxx.xxxxx", # Shape config details mandatory for flexible shapes: # deployment_instance_shape="VM.Standard.E4.Flex", # deployment_ocpus=, # deployment_memory_in_gbs=, ) print(f"Endpoint: {tensorflow_model.model_deployment.url}") # Output: "Endpoint: https://modeldeployment.{region}.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.xxx.xxxxx" Run Prediction against Endpoint =============================== .. code-block:: python3 # Generate prediction by invoking the deployed endpoint tensorflow_model.predict(testx[:3])['prediction'] .. parsed-literal:: [[-2.9461750984191895, -5.293642997741699, 0.4030594229698181, 3.0270071029663086, -6.470805644989014, -2.07453989982605, -9.646402359008789, 9.256569862365723, -2.6433541774749756, -0.8167083263397217], [-3.4297854900360107, 2.4863781929016113, 8.968724250793457, 3.162344217300415, -11.153030395507812, 0.15335027873516083, -0.5451826453208923, -7.817524433135986, -1.0585914850234985, -10.736929893493652], [-4.420501232147217, 5.841022491455078, -0.17066864669322968, -1.0071465969085693, -2.261953592300415, -3.0983355045318604, -2.0874621868133545, 1.0745809078216553, -1.2511857748031616, -2.273810625076294]] Predict with Image ------------------ .. versionadded:: 2.6.7 Predict Image by passing a uri, which can be http(s), local path, or other URLs (e.g. starting with “oci://”, “s3://”, and “gcs://”), of the image or a PIL.Image.Image object using the ``image`` argument in ``predict()`` to predict a single image. The image will be converted to a tensor and then serialized so it can be passed to the endpoint. You can catch the tensor in ``score.py`` to perform further transformation. Common example for model deployment prediction with image passed: .. code-block:: python3 # Generate prediction by invoking the deployed endpoint prediction = tensorflow_model.predict(image=)['prediction'] See the "Predict with Image" example `here `_. Run Prediction with oci raw-request command =========================================== Model deployment endpoints can be invoked with the OCI-CLI. This example invokes a model deployment with the CLI with a ``numpy.ndarray`` payload: `numpy.ndarray` payload example ------------------------------- .. code-block:: python3 >>> # Prepare data sample for prediction and save it to file 'data-payload' >>> from io import BytesIO >>> import base64 >>> import numpy as np >>> data = testx[:3] >>> np_bytes = BytesIO() >>> np.save(np_bytes, data, allow_pickle=True) >>> data = base64.b64encode(np_bytes.getvalue()).decode("utf-8") >>> with open('data-payload', 'w') as f: >>> f.write('{"data": "' + data + '", "data_type": "numpy.ndarray"}') File ``data-payload`` will have this information: .. code-block:: bash {"data": "k05VTVBZAQB2AHsnZGVzY3InOiAnPGY4JywgJ2ZvcnRyYW5fb3JkZXInOi... ....................................................................... ...................AAAAAAAAAAAAAAAAAAA=", "data_type": "numpy.ndarray"} Use file ``data-payload`` with data and endpoint to invoke prediction with raw-request command in terminal: .. code-block:: bash export uri=https://modeldeployment.{region}.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.xxx.xxxxx/predict oci raw-request \ --http-method POST \ --target-uri $uri \ --request-body file://data-payload Expected output of raw-request command -------------------------------------- .. code-block:: bash { "data": { "prediction": [ [ -1.8818638324737549, -6.04175329208374, .................., -1.1257498264312744 ], [ 1.2170600891113281, 1.6379727125167847, .................., -9.877771377563477 ], [ -4.255424499511719, 5.320354461669922, .................., -2.858555555343628 ] ] }, "headers": { "Connection": "keep-alive", "Content-Length": "594", "Content-Type": "application/json", "Date": "Thu, 08 Dec 2022 23:25:47 GMT", "X-Content-Type-Options": "nosniff", "opc-request-id": "E70002DAA3024F46B074F9B53DB6BEBB/421B34FCB12CF33F23C85D5619A62926/CAABF4A269C63B112482B2E57463CA13", "server": "uvicorn" }, "status": "200 OK" } Example ======= .. code-block:: python3 from ads.common.model_metadata import UseCaseType from ads.model.framework.tensorflow_model import TensorFlowModel import tensorflow as tf from uuid import uuid4 # Load MNIST Data mnist = tf.keras.datasets.mnist (trainx, trainy), (testx, testy) = mnist.load_data() trainx, testx = trainx / 255.0, testx / 255.0 # Train TensorFlow model model = tf.keras.models.Sequential( [ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(128, activation="relu"), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10), ] ) loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"]) model.fit(trainx, trainy, epochs=1) # Prepare Model Artifact for TensorFlow model tensorflow_model = TensorFlowModel(estimator=model, artifact_dir=f"./model-artifact-{str(uuid4())}") tensorflow_model.prepare( inference_conda_env="tensorflow28_p38_cpu_v1", training_conda_env="tensorflow28_p38_cpu_v1", X_sample=trainx, y_sample=trainy, use_case_type=UseCaseType.MULTINOMIAL_CLASSIFICATION, ) # Check if the artifacts are generated correctly. # The verify method invokes the ``predict`` function defined inside ``score.py`` in the artifact_dir tensorflow_model.verify(testx[:10])["prediction"] # Register the model model_id = tensorflow_model.save(display_name="TensorFlow Model") # Deploy and create an endpoint for the TensorFlow model tensorflow_model.deploy( display_name="TensorFlow Model For Classification", deployment_log_group_id="ocid1.loggroup.oc1.xxx.xxxxx", deployment_access_log_id="ocid1.log.oc1.xxx.xxxxx", deployment_predict_log_id="ocid1.log.oc1.xxx.xxxxx", ) # Generate prediction by invoking the deployed endpoint tensorflow_model.predict(testx)["prediction"] # To delete the deployed endpoint uncomment the line below # tensorflow_model.delete_deployment(wait_for_completion=True)