ads.model.deployment package

Submodules

ads.model.deployment.model_deployer module

APIs to interact with Oracle’s Model Deployment service.

There are three main classes: ModelDeployment, ModelDeploymentDetails, ModelDeployer.

One creates a ModelDeployment and deploys it under the umbrella of the ModelDeployer class. This way multiple ModelDeployments can be unified with one ModelDeployer. The ModelDeployer class also serves as the interface to all the deployments. ModelDeploymentDetails holds information about the particular details of a particular deployment, such as how many instances, etc. In this way multiple, independent ModelDeployments with the same details can be created using the ModelDeployer class.

Examples

>>> from model_deploy.model_deployer import ModelDeployer, ModelDeploymentDetails
>>> deployer = ModelDeployer("model_dep_conf.yaml")
>>> deployment_properties = ModelDeploymentProperties(
...             'ocid1.datasciencemodel.ocn.reg.xxxxxxxxxxxxxxxxxxxxxxxxx')
...                 .with_prop('display_name', "My model display name")
...                 .with_prop("project_id", project_id)
...                 .with_prop("compartment_id", compartment_id)
...                 .with_instance_configuration(
...                     config={"INSTANCE_SHAPE":"VM.Standard.E3.Flex",
...                             "INSTANCE_COUNT":"1",
...                             "bandwidth_mbps":10,
...                             "memory_in_gbs":10,
...                             "ocpus":2}
...                  ).build()
>>> deployment_info = deployer.deploy(deployment_properties,
...             max_wait_time=600, poll_interval=15)
>>> print(deployment_info.model_deployment_id)
>>> print(deployment_info.workflow_req_id)
>>> print(deployment_info.url)
>>> deployer.list_deployments() # Optionally pass in a status
class ads.model.deployment.model_deployer.ModelDeployer(config: Optional[dict] = None, ds_client: Optional[DataScienceClient] = None)

Bases: object

ModelDeployer is the class responsible for deploying the ModelDeployment

config

ADS auth dictionary for OCI authentication.

Type:

dict

ds_client

data science client

Type:

DataScienceClient

ds_composite_client

composite data science client

Type:

DataScienceCompositeClient

deploy(model_deployment_details, \*\*kwargs)

Deploy the model specified by model_deployment_details.

get_model_deployment(model_deployment_id: str)

Get the ModelDeployment specified by model_deployment_id.

get_model_deployment_state(model_deployment_id)

Get the state of the current deployment specified by id.

delete(model_deployment_id, \*\*kwargs)

Remove the model deployment specified by the id or Model Deployment Object

list_deployments(status)

lists the model deployments associated with current compartment and data science client

show_deployments(status)

shows the deployments filtered by status in a Dataframe

Initializes model deployer.

Parameters:

config (dict, optional) –

ADS auth dictionary for OCI authentication.

This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.

ds_client: oci.data_science.data_science_client.DataScienceClient

The Oracle DataScience client.

delete(model_deployment_id, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10) ModelDeployment

Deletes the model deployment specified by OCID.

Parameters:
  • model_deployment_id (str) – Model deployment OCID.

  • wait_for_completion (bool) – Wait for deletion to complete. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 60).

Return type:

A ModelDeployment instance that was deleted

deploy(properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs) ModelDeployment

Deploys a model.

Parameters:
  • properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Optional, defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds. Optional, defaults to 1200. Negative value implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds. Optional, defaults to 30.

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties. See ModelDeploymentProperties() for details.

Returns:

A ModelDeployment instance.

Return type:

ModelDeployment

deploy_from_model_uri(model_uri: str, properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs) ModelDeployment

Deploys a model.

Parameters:
  • model_uri (str) – uri to model files, can be local or in cloud storage

  • properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 30).

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties

Returns:

A ModelDeployment instance

Return type:

ModelDeployment

get_model_deployment(model_deployment_id: str) ModelDeployment

Gets a ModelDeployment by OCID.

Parameters:

model_deployment_id (str) – Model deployment OCID

Returns:

A ModelDeployment instance

Return type:

ModelDeployment

get_model_deployment_state(model_deployment_id: str) State

Gets the state of a deployment specified by OCID

Parameters:

model_deployment_id (str) – Model deployment OCID

Returns:

The state of the deployment

Return type:

str

list_deployments(status=None, compartment_id=None, **kwargs) list

Lists the model deployments associated with current compartment and data science client

Parameters:
  • status (str) – Status of deployment. Defaults to None.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

  • kwargs – The values are passed to oci.data_science.DataScienceClient.list_model_deployments.

Returns:

A list of ModelDeployment objects.

Return type:

list

Raises:

ValueError – If compartment_id is not specified and cannot be determined from the environment.

show_deployments(status=None, compartment_id=None) DataFrame
Returns the model deployments associated with current compartment and data science client

as a Dataframe that can be easily visualized

Parameters:
  • status (str) – Status of deployment. Defaults to None.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

Returns:

pandas Dataframe containing information about the ModelDeployments

Return type:

DataFrame

Raises:

ValueError – If compartment_id is not specified and cannot be determined from the environment.

update(model_deployment_id: str, properties: Optional[ModelDeploymentProperties] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs) ModelDeployment

Updates an existing model deployment.

Parameters:
  • model_deployment_id (str) – Model deployment OCID.

  • properties (ModelDeploymentProperties) – An instance of ModelDeploymentProperties or dict to initialize the ModelDeploymentProperties. Defaults to None.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200).

  • poll_interval (int) – Poll interval in seconds (Defaults to 30).

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties.

Returns:

A ModelDeployment instance

Return type:

ModelDeployment

ads.model.deployment.model_deployment module

exception ads.model.deployment.model_deployment.LogNotConfiguredError

Bases: Exception

class ads.model.deployment.model_deployment.ModelDeployment(properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, config: Optional[Dict] = None, workflow_req_id: Optional[str] = None, model_deployment_id: Optional[str] = None, model_deployment_url: str = '', **kwargs)

Bases: object

A class used to represent a Model Deployment.

config

Deployment configuration parameters

Type:

(dict)

properties

ModelDeploymentProperties object

Type:

(ModelDeploymentProperties)

workflow_state_progress

Workflow request id

Type:

(str)

workflow_steps

The number of steps in the workflow

Type:

(int)

url

The model deployment url endpoint

Type:

(str)

ds_client

The data science client used by model deployment

Type:

(DataScienceClient)

ds_composite_client

The composite data science client used by the model deployment

Type:

(DataScienceCompositeClient)

workflow_req_id

Workflow request id

Type:

(str)

model_deployment_id

model deployment id

Type:

(str)

state

Returns the deployment state of the current Model Deployment object

Type:

(State)

deploy(wait_for_completion, \*\*kwargs)

Deploy the current Model Deployment object

delete(wait_for_completion, \*\*kwargs)

Deletes the current Model Deployment object

update(wait_for_completion, \*\*kwargs)

Updates a model deployment

activate(wait_for_completion, max_wait_time, poll_interval)

Activates a model deployment

deactivate(wait_for_completion, max_wait_time, poll_interval)

Deactivates a model deployment

list_workflow_logs()

Returns a list of the steps involved in deploying a model

Initializes a ModelDeployment object.

Parameters:
  • properties ((Union[ModelDeploymentProperties, Dict], optional). Defaults to None.) – Object containing deployment properties. The properties can be None when kwargs are used for specifying properties.

  • config ((Dict, optional). Defaults to None.) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None then the ads.common.default_signer(client_kwargs) will be used.

  • workflow_req_id ((str, optional). Defaults to None.) – Workflow request id.

  • model_deployment_id ((str, optional). Defaults to None.) – Model deployment OCID.

  • model_deployment_url ((str, optional). Defaults to empty string.) – Model deployment url.

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties.

property access_log: OCILog

Gets the model deployment access logs object.

Returns:

The OCILog object containing the access logs.

Return type:

OCILog

activate(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10) ModelDeployment

Activates a model deployment

Parameters:
  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be activated before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

deactivate(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10) ModelDeployment

Deactivates a model deployment

Parameters:
  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be deactivated before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

delete(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10)

Deletes the ModelDeployment

Parameters:
  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be deleted before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

deploy(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10)

deploy deploys the current ModelDeployment object

Parameters:
  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be deployed before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

list_workflow_logs() list

Returns a list of the steps involved in deploying a model

Returns:

List of dictionaries detailing the status of each step in the deployment process.

Return type:

list

logs(log_type: Optional[str] = None) ConsolidatedLog

Gets the access or predict logs.

Parameters:

log_type ((str, optional). Defaults to None.) – The log type. Can be “access”, “predict” or None.

Returns:

The ConsolidatedLog object containing the logs.

Return type:

ConsolidatedLog

predict(json_input=None, data: Optional[Any] = None, auto_serialize_data: bool = False, **kwargs) dict

Returns prediction of input data run against the model deployment endpoint

Parameters:
  • json_input (Json serializable) – Json payload for the prediction.

  • data (Any) – Data for the prediction.

  • auto_serialize_data (bool.) – Whether to auto serialize input data. Defauls to False. If auto_serialize_data=False, data required to be bytes or json serializable and json_input required to be json serializable. If auto_serialize_data set to True, data will be serialized before sending to model deployment endpoint.

  • kwargs

    content_type: str

    Used to indicate the media type of the resource. By default, it will be application/octet-stream for bytes input and application/json otherwise. The content-type header will be set to this value when calling the model deployment endpoint.

Returns:

Prediction results.

Return type:

dict

property predict_log: OCILog

Gets the model deployment predict logs object.

Returns:

The OCILog object containing the predict logs.

Return type:

OCILog

show_logs(time_start: Optional[datetime] = None, time_end: Optional[datetime] = None, limit: int = 100, log_type: Optional[str] = None)

Shows deployment logs as a pandas dataframe.

Parameters:
  • time_start ((datetime.datetime, optional). Defaults to None.) – Starting date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved 14 days from now.

  • time_end ((datetime.datetime, optional). Defaults to None.) – Ending date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved until now.

  • limit ((int, optional). Defaults to 100.) – The maximum number of items to return.

  • log_type ((str, optional). Defaults to None.) – The log type. Can be “access”, “predict” or None.

Return type:

A pandas DataFrame containing logs.

property state: State

Returns the deployment state of the current Model Deployment object

property status: State

Returns the deployment state of the current Model Deployment object

update(properties: Optional[Union[ModelDeploymentProperties, dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs)

Updates a model deployment

You can update model_deployment_configuration_details and change instance_shape and model_id when the model deployment is in the ACTIVE lifecycle state. The bandwidth_mbps or instance_count can only be updated while the model deployment is in the INACTIVE state. Changes to the bandwidth_mbps or instance_count will take effect the next time the ActivateModelDeployment action is invoked on the model deployment resource.

Parameters:
  • properties (ModelDeploymentProperties or dict) – The properties for updating the deployment.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be updated before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

  • kwargs – dict

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

class ads.model.deployment.model_deployment.ModelDeploymentLogType

Bases: object

ACCESS = 'access'
PREDICT = 'predict'

ads.model.deployment.model_deployment_properties module

class ads.model.deployment.model_deployment_properties.ModelDeploymentProperties(model_id: Optional[str] = None, model_uri: Optional[str] = None, oci_model_deployment: Optional[Union[ModelDeployment, CreateModelDeploymentDetails, UpdateModelDeploymentDetails, Dict]] = None, config: Optional[dict] = None, **kwargs)

Bases: OCIDataScienceMixin, ModelDeployment

Represents the details for a model deployment

swagger_types

The property names and the corresponding types of OCI ModelDeployment model.

Type:

dict

model_id

The model artifact OCID in model catalog.

Type:

str

model_uri

uri to model files, can be local or in cloud storage.

Type:

str

with_prop(property_name, value)

Set the model deployment details property_name attribute to value

with_instance_configuration(config)

Set the configuration of VM instance.

with_access_log(log_group_id, log_id)

Config the access log with OCI logging service

with_predict_log(log_group_id, log_id)

Config the predict log with OCI logging service

build()

Return an instance of CreateModelDeploymentDetails for creating the deployment.

Initialize a ModelDeploymentProperties object by specifying one of the followings:

Parameters:
  • model_id ((str, optiona). Defaults to None.) – Model Artifact OCID. The model_id must be specified either explicitly or as an attribute of the OCI object.

  • model_uri ((str, optiona). Defaults to None.) – Uri to model files, can be local or in cloud storage.

  • oci_model_deployment ((Union[ModelDeployment, CreateModelDeploymentDetails, UpdateModelDeploymentDetails, Dict], optional). Defaults to None.) – An OCI model or Dict containing model deployment details. The OCI model can be an instance of either ModelDeployment, CreateModelDeploymentDetails or UpdateModelConfigurationDetails.

  • config ((Dict, optional). Defaults to None.) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.

  • kwargs

    Users can also initialize the object by using keyword arguments. The following keyword arguments are supported by oci.data_science.models.data_science_models.ModelDeployment:

    • display_name,

    • description,

    • project_id,

    • compartment_id,

    • model_deployment_configuration_details,

    • category_log_details,

    • freeform_tags,

    • defined_tags.

    If display_name is not specified, a randomly generated easy to remember name will be generated, like ‘strange-spider-2022-08-17-23:55.02’.

    ModelDeploymentProperties also supports the following additional keyward arguments:

    • instance_shape,

    • instance_count,

    • bandwidth_mbps,

    • access_log_group_id,

    • access_log_id,

    • predict_log_group_id,

    • predict_log_id,

    • memory_in_gbs,

    • ocpus.

    These additional arguments will be saved into appropriate properties in the OCI model.

Raises:

ValueError – model_id is None AND not specified in oci_model_deployment.model_deployment_configuration_details.model_configuration_details.

build() CreateModelDeploymentDetails

Converts the deployment properties to OCI CreateModelDeploymentDetails object. Converts a model URI into a model OCID if user passed in a URI.

Returns:

A CreateModelDeploymentDetails instance ready for OCI API.

Return type:

CreateModelDeploymentDetails

sub_properties = ['instance_shape', 'instance_count', 'bandwidth_mbps', 'access_log_group_id', 'access_log_id', 'predict_log_group_id', 'predict_log_id', 'memory_in_gbs', 'ocpus']
to_oci_model(oci_model)

Convert properties into an OCI data model

Parameters:

oci_model (class) – The class of OCI data model, e.g., oci.data_science_models.CreateModelDeploymentDetails

to_update_deployment() UpdateModelDeploymentDetails

Converts the deployment properties to OCI UpdateModelDeploymentDetails object.

Returns:

An UpdateModelDeploymentDetails instance ready for OCI API.

Return type:

CreateModelDeploymentDetails

with_access_log(log_group_id: str, log_id: str)

Adds access log config

Parameters:
  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns:

self

Return type:

ModelDeploymentProperties

with_category_log(log_type: str, group_id: str, log_id: str)

Adds category log configuration

Parameters:
  • log_type (str) – The type of logging to be configured. Must be “access” or “predict”

  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns:

self

Return type:

ModelDeploymentProperties

Raises:

ValueError – When log_type is invalid

with_instance_configuration(config)

with_instance_configuration creates a ModelDeploymentDetails object with a specific config

Parameters:

config (dict) –

dictionary containing instance configuration about the deployment. The following keys are supported:

  • instance_shape: str,

  • instance_count: int,

  • bandwidth_mbps: int,

  • memory_in_gbs: float,

  • ocpus: float

The instance_shape and instance_count are required when creating a new deployment. They are optional when updating an existing deployment.

Returns:

self

Return type:

ModelDeploymentProperties

with_logging_configuration(access_log_group_id: str, access_log_id: str, predict_log_group_id: Optional[str] = None, predict_log_id: Optional[str] = None)

Adds OCI logging configurations for OCI logging service

Parameters:
  • access_log_group_id (str) – Log group ID of OCI logging service for access log

  • access_log_id (str) – Log ID of OCI logging service for access log

  • predict_log_group_id (str) – Log group ID of OCI logging service for predict log

  • predict_log_id (str) – Log ID of OCI logging service for predict log

Returns:

self

Return type:

ModelDeploymentProperties

with_predict_log(log_group_id: str, log_id: str)

Adds predict log config

Parameters:
  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns:

self

Return type:

ModelDeploymentProperties

with_prop(property_name: str, value: Any)

Sets model deployment’s property_name attribute to value

Parameters:
  • property_name (str) – Name of a model deployment property.

  • value – New value for property attribute.

Returns:

self

Return type:

ModelDeploymentProperties

Module contents