Deploy Model with Conda Runtime

Once you have ADS Model object, you can call deploy function to deploy the model and generate the endpoint.

Here is an example of deploying LightGBM model:

import lightgbm as lgb
import tempfile
from ads.common.model_metadata import UseCaseType
from ads.model.framework.lightgbm_model import LightGBMModel
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

# Load dataset and Prepare train and test split
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

# Train a LightGBM Classifier model
train = lgb.Dataset(X_train, label=y_train)
param = {
  'objective': 'multiclass', 'num_class': 3,
}
lightgbm_estimator = lgb.train(param, train)

# Instantiate ads.model.LightGBMModel using the trained LGBM Model
lightgbm_model = LightGBMModel(estimator=lightgbm_estimator, artifact_dir=tempfile.mkdtemp())

# Autogenerate score.py, pickled model, runtime.yaml, input_schema.json and output_schema.json
lightgbm_model.prepare(
    inference_conda_env="generalml_p38_cpu_v1",
    X_sample=X_train,
    y_sample=y_train,
    use_case_type=UseCaseType.BINARY_CLASSIFICATION,
)

# Verify generated artifacts
lightgbm_model.verify(X_test)

# Register LightGBM model
model_id = lightgbm_model.save()

# Deploy LightGBM model
lightgbm_model.deploy(
        display_name="LightGBM Model",
        deployment_log_group_id="ocid1.loggroup.oc1.xxx.xxxxx",
        deployment_access_log_id="ocid1.log.oc1.xxx.xxxxx",
        deployment_predict_log_id="ocid1.log.oc1.xxx.xxxxx",
        # Shape config details mandatory for flexible shapes:
        # deployment_instance_shape="VM.Standard.E4.Flex",
        # deployment_ocpus=<number>,
        # deployment_memory_in_gbs=<number>,
    )

# Get endpoint of deployed model
model_deployment_url = lightgbm_model.model_deployment.url

# Generate prediction by invoking the deployed endpoint
lightgbm_model.predict(X_test)["prediction"]

Here example retrieve predictions from model deployment endpoint using oci-cli:

export model_deployment_url=<model_deployment_url>/predict
oci raw-request --http-method POST \
    --target-uri $model_deployment_url \
    --request-body '{"data": [[5.6, 2.7, 4.2, 1.3]]}'

Find more information about oci raw-request command here.

Deploy

You can use the .deploy() method to deploy a model. You must first save the model to the model catalog, and then deploy it.

Deployment with Flex shape

It is mandatory to provide ocpus and memory_in_gb values, when deploy with Flex instance shapes. They are set in deployment_ocpus and deployment_memory_in_gbs of the deploy() method.

The .deploy() method returns a ModelDeployment object. Specify deployment attributes such as display name, instance type, number of instances, maximum router bandwidth, and logging groups. The API takes the following parameters:

See deploy() for more details about the parameters.

Tips

Predict

To invoke the endpoint of your deployed model, call the .predict() method. The .predict() method sends a request to the deployed endpoint, and computes the inference values based on the data that you input in the .predict() method.

See how to deploy and invoke deployed endpoint for different frameworks here.

See API documentation for more details about the parameters.

Observability

tail or head logs generated by the model deployment instances -

lightgbm_model.model_deployment.logs().tail()

You cal also call the .watch() from model deployment instance to stream the logs

lightgbm_model.model_deployment.watch()

Update Model Deployment

You can update the existing Model Deployment by using .update_deployment() method. See API documentation for more details.

lightgbm_model.update_deployment(
  wait_for_completion = True,
  access_log={
    log_id="ocid1.log.oc1.xxx.xxxxx",
  },
  description="Description for Custom Model",
  freeform_tags={"key": "value"},
)