Deploy Model with Container Runtime¶
The ADS GenericModel
and ModelDeployment
classes allow you to deploy model using container images.
To deploy model on container runtime, you need to first build a docker container image. See Bring Your Own Container for the end-to-end example. Once you have the image, push it to OCI container registry. See Creating a Repository and Pushing Images Using the Docker CLI for more details.
Deploy Using GenericModel Class¶
When the container runtime is ready, you can call deploy
function to deploy the model and generate the endpoint. You must specify the container deployment_image
. You can optionally specify the entrypoint and cmd for running the container (See Understand how CMD and ENTRYPOINT interact). For more details regarding the parameters allowed for container runtime, see BYOC Required Interfaces.
Below is an example of deploying Sklearn model on container runtime using SklearnModel
class:
import os
import pandas as pd
from joblib import load
from ads.model import SklearnModel
# Load data
data = pd.read_json(<path_to_data>)
data_test = data.transpose()
X = data_test.drop(data_test.loc[:, "Line":"# Letter"].columns, axis=1)
X_test = X.iloc[int("12"), :].values.reshape(1, -1)
# Load model
clf_lda = load(<path_to_model>)
# Instantiate ads.model.SklearnModel
sklearn_model = SklearnModel(estimator=clf_lda, artifact_dir=<path_to_artifact_directory>)
# Prepare related artifacts
sklearn_model.prepare(
model_file_name=<model_file_name>,
ignore_conda_error=True, # make sure to set ignore_conda_error=True for container runtime
)
# Verify model locally
sklearn_model.verify(X_test)
# Register Sklearn model
sklearn_model.save()
# Deploy Sklearn model on container runtime
sklearn_model.deploy(
display_name="Sklearn Model BYOC",
deployment_log_group_id="ocid1.loggroup.oc1.xxx.xxxxx",
deployment_access_log_id="ocid1.log.oc1.xxx.xxxxx",
deployment_predict_log_id="ocid1.log.oc1.xxx.xxxxx",
deployment_instance_subnet_id="ocid1.subnet.oc1.xxx.xxxxx",
deployment_image="iad.ocir.io/<namespace>/<image>:<tag>",
entrypoint=["python", "/opt/ds/model/deployed_model/api.py"],
server_port=5000,
health_check_port=5000,
environment_variables={"test_key": "test_value"},
)
# Get endpoint of deployed model
model_deployment_url = sklearn_model.model_deployment.url
# Generate prediction by invoking the deployed endpoint
sklearn_model.predict(data={"line": "12"})
Deploy Using ModelDeployment Class¶
To deploy a model deployment, you can define a ModelDeployment
object and call the .deploy()
of it. You could either use API or YAML to define the ModelDeployment
object.
When configuring the ModelDeploymentContainerRuntime
object, you must specify the container image. You can optionally specify the entrypoint and cmd for running the container (See Understand how CMD and ENTRYPOINT interact). For more details regarding the parameters allowed for container runtime, see BYOC Required Interfaces.
Below is an example of deploying model on container runtime using ModelDeployment
class:
from ads.model.deployment import ModelDeployment, ModelDeploymentInfrastructure, ModelDeploymentContainerRuntime
# configure model deployment infrastructure
infrastructure = (
ModelDeploymentInfrastructure()
.with_project_id("<PROJECT_OCID>")
.with_compartment_id("<COMPARTMENT_OCID>")
.with_shape_name("VM.Standard.E4.Flex")
.with_shape_config_details(
ocpus=1,
memory_in_gbs=16
)
.with_replica(1)
.with_bandwidth_mbps(10)
.with_web_concurrency(10)
.with_access_log(
log_group_id="<ACCESS_LOG_GROUP_OCID>",
log_id="<ACCESS_LOG_OCID>"
)
.with_predict_log(
log_group_id="<PREDICT_LOG_GROUP_OCID>",
log_id="<PREDICT_LOG_OCID>"
)
.with_subnet_id("<SUBNET_OCID>")
)
# configure model deployment runtime
container_runtime = (
ModelDeploymentContainerRuntime()
.with_image("iad.ocir.io/<namespace>/<image>:<tag>")
.with_image_digest("<IMAGE_DIGEST>")
.with_entrypoint(["python","/opt/ds/model/deployed_model/api.py"])
.with_server_port(5000)
.with_health_check_port(5000)
.with_env({"key":"value"})
.with_deployment_mode("HTTPS_ONLY")
.with_model_uri("<path_to_artifact>")
.with_auth({"auth_key":"auth_value"})
.with_region("us-ashburn-1")
.with_overwrite_existing_artifact(True)
.with_remove_existing_artifact(True)
.with_timeout(100)
.with_bucket_uri("oci://<bucket>@<namespace>/<prefix>")
)
# configure model deployment
deployment = (
ModelDeployment()
.with_display_name("Model Deployment Demo using ADS")
.with_description("The model deployment description")
.with_freeform_tags({"key1":"value1"})
.with_infrastructure(infrastructure)
.with_runtime(container_runtime)
)
# Deploy model on container runtime
deployment.deploy()
# Generate prediction by invoking the deployed endpoint
deployment.predict(data=<data>)
from ads.model.deployment import ModelDeployment
yaml_string = """
kind: deployment
spec:
displayName: Model Deployment Demo using ADS
description: The model deployment description
freeform_tags:
key1: value1
infrastructure:
kind: infrastructure
type: datascienceModelDeployment
spec:
compartmentId: <COMPARTMENT_OCID>
projectId: <PROJECT_OCID>
accessLog:
logGroupId: <ACCESS_LOG_GROUP_OCID>
logId: <ACCESS_LOG_OCID>
predictLog:
logGroupId: <PREDICT_LOG_GROUP_OCID>
logId: <PREDICT_LOG_OCID>
shapeName: VM.Standard.E4.Flex
shapeConfigDetails:
memoryInGBs: 16
ocpus: 1
replica: 1
bandWidthMbps: 10
webConcurrency: 10
subnetId: <SUBNET_OCID>
runtime:
kind: runtime
type: container
spec:
modelUri: <path_to_artifact>
image: iad.ocir.io/<namespace>/<image>:<tag>
imageDigest: <IMAGE_DIGEST>
entrypoint: ["python","/opt/ds/model/deployed_model/api.py"]
serverPort: 5000
healthCheckPort: 5000
env:
WEB_CONCURRENCY: "10"
auth:
auth_key: auth_value
region: us-ashburn-1
overwriteExistingArtifact: True
removeExistingArtifact: True
timeout: 100
bucketUri: oci://<bucket>@<namespace>/<prefix>
deploymentMode: HTTPS_ONLY
"""
# Initialize ads.ModelDeployment
deployment = ModelDeployment.from_yaml(yaml_string)
# Deploy model on container runtime
deployment.deploy()
# Generate prediction by invoking the deployed endpoint
deployment.predict(data=<data>)