ads.model.deployment package

Submodules

ads.model.deployment.model_deployer module

APIs to interact with Oracle’s Model Deployment service.

There are three main classes: ModelDeployment, ModelDeploymentDetails, ModelDeployer.

One creates a ModelDeployment and deploys it under the umbrella of the ModelDeployer class. This way multiple ModelDeployments can be unified with one ModelDeployer. The ModelDeployer class also serves as the interface to all the deployments. ModelDeploymentDetails holds information about the particular details of a particular deployment, such as how many instances, etc. In this way multiple, independent ModelDeployments with the same details can be created using the ModelDeployer class.

Examples

>>> from model_deploy.model_deployer import ModelDeployer, ModelDeploymentDetails
>>> deployer = ModelDeployer("model_dep_conf.yaml")
>>> deployment_properties = ModelDeploymentProperties(
...             'ocid1.datasciencemodel.ocn.reg.xxxxxxxxxxxxxxxxxxxxxxxxx')
...                 .with_prop('display_name', "My model display name")
...                 .with_prop("project_id", project_id)
...                 .with_prop("compartment_id", compartment_id)
...                 .with_instance_configuration(
...                     config={"INSTANCE_SHAPE":"VM.Standard2.1",
...                             "INSTANCE_COUNT":"1",
...                             'bandwidth_mbps':10})
...                      .build()
>>> deployment_info = deployer.deploy(deployment_properties,
...             max_wait_time=600, poll_interval=15)
>>> print(deployment_info.model_deployment_id)
>>> print(deployment_info.workflow_req_id)
>>> print(deployment_info.url)
>>> deployer.list_deployments() # Optionally pass in a status
class ads.model.deployment.model_deployer.ModelDeployer(config: Optional[dict] = None)

Bases: object

ModelDeployer is the class responsible for deploying the ModelDeployment

config

ADS auth dictionary for OCI authentication.

Type

dict

ds_client

data science client

Type

DataScienceClient

ds_composite_client

composite data science client

Type

DataScienceCompositeClient

deploy(model_deployment_details, \*\*kwargs)

Deploy the model specified by model_deployment_details.

get_model_deployment(model_deployment_id: str)

Get the ModelDeployment specified by model_deployment_id.

get_model_deployment_state(model_deployment_id)

Get the state of the current deployment specified by id.

delete(model_deployment_id, \*\*kwargs)

Remove the model deployment specified by the id or Model Deployment Object

list_deployments(status)

lists the model deployments associated with current compartment and data science client

show_deployments(status)

shows the deployments filtered by status in a Dataframe

Initializes model deployer.

Parameters

config (dict, optional) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.

delete(model_deployment_id, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30) → ModelDeployment

Deletes the model deployment specified by OCID.

Parameters

  • model_deployment_id (str) – Model deployment OCID.

  • wait_for_completion (bool) – Wait for deletion to complete. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 60).

Return type

A ModelDeployment instance that was deleted

deploy(properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs) → ModelDeployment

Deploys a model.

Parameters

  • properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Optional, defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds. Optional, defaults to 1200. Negative value implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds. Optional, defaults to 30.

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties. See ModelDeploymentProperties() for details.

Returns

A ModelDeployment instance.

Return type

ModelDeployment

deploy_from_model_uri(model_uri: str, properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs) → ModelDeployment

Deploys a model.

Parameters

  • model_uri (str) – uri to model files, can be local or in cloud storage

  • properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 30).

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties

Returns

A ModelDeployment instance

Return type

ModelDeployment

get_model_deployment(model_deployment_id: str) → ModelDeployment

Gets a ModelDeployment by OCID.

Parameters

model_deployment_id (str) – Model deployment OCID

Returns

A ModelDeployment instance

Return type

ModelDeployment

get_model_deployment_state(model_deployment_id: str) → State

Gets the state of a deployment specified by OCID

Parameters

model_deployment_id (str) – Model deployment OCID

Returns

The state of the deployment

Return type

str

list_deployments(status=None, compartment_id=None, **kwargs) → list

Lists the model deployments associated with current compartment and data science client

Parameters

  • status (str) – Status of deployment. Defaults to None.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

  • kwargs – The values are passed to oci.data_science.DataScienceClient.list_model_deployments.

Returns

A list of ModelDeployment objects.

Return type

list

Raises

ValueError – If compartment_id is not specified and cannot be determined from the environment.

show_deployments(status=None, compartment_id=None) → DataFrame
Returns the model deployments associated with current compartment and data science client

as a Dataframe that can be easily visualized

Parameters

  • status (str) – Status of deployment. Defaults to None.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

Returns

pandas Dataframe containing information about the ModelDeployments

Return type

DataFrame

Raises

ValueError – If compartment_id is not specified and cannot be determined from the environment.

update(model_deployment_id: str, properties: Optional[ModelDeploymentProperties] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs) → ModelDeployment

Updates an existing model deployment.

Parameters

  • model_deployment_id (str) – Model deployment OCID.

  • properties (ModelDeploymentProperties) – An instance of ModelDeploymentProperties or dict to initialize the ModelDeploymentProperties. Defaults to None.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200).

  • poll_interval (int) – Poll interval in seconds (Defaults to 30).

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties.

Returns

A ModelDeployment instance

Return type

ModelDeployment

ads.model.deployment.model_deployment module

class ads.model.deployment.model_deployment.ModelDeployment(properties=None, config=None, workflow_req_id=None, model_deployment_id=None, model_deployment_url='', **kwargs)

Bases: object

A class used to represent a Model Deployment.

config

Deployment configuration parameters

Type

(dict)

deployment_properties

ModelDeploymentProperties object

Type

(ModelDeploymentProperties)

workflow_state_progress

Workflow request id

Type

(str)

workflow_steps

The number of steps in the workflow

Type

(int)

url

The model deployment url endpoint

Type

(str)

ds_client

The data science client used by model deployment

Type

(DataScienceClient)

ds_composite_client

The composite data science client used by the model deployment

Type

(DataScienceCompositeClient)

workflow_req_id

Workflow request id

Type

(str)

model_deployment_id

model deployment id

Type

(str)

state

Returns the deployment state of the current Model Deployment object

Type

(State)

deploy(wait_for_completion, \*\*kwargs)

Deploy the current Model Deployment object

delete(wait_for_completion, \*\*kwargs)

Deletes the current Model Deployment object

update(wait_for_completion, \*\*kwargs)

Updates a model deployment

list_workflow_logs()

Returns a list of the steps involved in deploying a model

Initializes a ModelDeployment

Parameters

  • properties (ModelDeploymentProperties or dict) – Object containing deployment properties. properties can be None when kwargs are used for specifying properties.

  • config (dict) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.

  • workflow_req_id (str) – Workflow request id. Defaults to “”

  • model_deployment_id (str) – Model deployment OCID. Defaults to “”

  • model_deployment_url (str) – Model deployment url. Defaults to “”

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties

property access_log: ModelDeploymentLog

Gets the model deployment predict logs object.

Returns

The ModelDeploymentLog object containing the predict logs.

Return type

ModelDeploymentLog

delete(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30)

Deletes the ModelDeployment

Parameters

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 60).

Returns

The instance of ModelDeployment.

Return type

ModelDeployment

deploy(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30)

deploy deploys the current ModelDeployment object

Parameters

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 60).

Returns

The instance of ModelDeployment.

Return type

ModelDeployment

list_workflow_logs() → list

Returns a list of the steps involved in deploying a model

Returns

List of dictionaries detailing the status of each step in the deployment process.

Return type

list

logs(log_type: str = 'access', **kwargs)

Gets the access or predict logs.

Parameters

  • log_type ((str, optional). Defaults to "access".) – The log type. Can be “access” or “predict”.

  • kwargs (dict) – Back compatability arguments.

Returns

The ModelDeploymentLog object containing the logs.

Return type

ModelDeploymentLog

predict(json_input: Optional[dict] = None, data: Optional[Union[bytes, dict]] = None, **kwargs) → dict

Returns prediction of input data run against the model deployment endpoint

Parameters

  • json_input (dict) – Json payload for the prediction.

  • data (Union[bytes, dict]) – Payload for the prediction.

  • kwargs –

    content_type: str

    Used to indicate the media type of the resource. By default, it will be application/octet-stream for bytes input and application/json otherwise. The content-type header will be set to this value when calling the model deployment endpoint.

Returns

Prediction results.

Return type

dict

property predict_log: ModelDeploymentLog

Gets the model deployment predict logs object.

Returns

The ModelDeploymentLog object containing the predict logs.

Return type

ModelDeploymentLog

show_logs(time_start: Optional[datetime] = None, time_end: Optional[datetime] = None, limit=100, log_type='access')

Shows deployment logs as a pandas dataframe.

Parameters

  • time_start ((datetime.datetime, optional). Defaults to None.) – Starting date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved 14 days from now.

  • time_end ((datetime.datetime, optional). Defaults to None.) – Ending date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved until now.

  • limit ((int, optional). Defaults to 100.) – The maximum number of items to return.

  • log_type ((str, optional). Defaults to "access".) – The log type. Can be “access” or “predict”.

Return type

A pandas DataFrame containing logs.

property state: State

Returns the deployment state of the current Model Deployment object

property status: State

Returns the deployment state of the current Model Deployment object

update(properties: Optional[Union[ModelDeploymentProperties, dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs)

Updates a model deployment

You can update model_deployment_configuration_details and change instance_shape and model_id when the model deployment is in the ACTIVE lifecycle state. The bandwidth_mbps or instance_count can only be updated while the model deployment is in the INACTIVE state. Changes to the bandwidth_mbps or instance_count will take effect the next time the ActivateModelDeployment action is invoked on the model deployment resource.

Parameters

  • properties (ModelDeploymentProperties or dict) – The properties for updating the deployment.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 60).

  • kwargs – dict

Returns

The instance of ModelDeployment.

Return type

ModelDeployment

class ads.model.deployment.model_deployment.ModelDeploymentLog(model_deployment_id: str, **kwargs)

Bases: OCILog

The class representing model deployment logs.

Initializes an OCI log model for the model deployment.

Parameters

  • model_deployment_id (str) – The OCID of the model deployment. This parameter will be used as a source field to filter the log records.

  • kwargs (dict) – Keyword arguments for initializing ModelDeploymentLog.

head(limit=100, time_start: Optional[datetime] = None) → None

Prints the preceding log records.

Parameters

  • limit ((int, optional). Defaults to 100.) – Maximum number of records to be returned.

  • time_start ((datetime.datetime, optional)) – Starting time for the log query. Defaults to None. Logs up to 14 days from now will be returned.

Returns

Nothing

Return type

None

stream(interval: int = 3, stop_condition: Optional[callable] = None, time_start: Optional[datetime] = None) → None

Streams logs to console/terminal until stop_condition() returns true.

Parameters

  • interval ((int, optional). Defaults to 3 seconds.) – The time interval between sending each request to pull logs from OCI.

  • stop_condition ((callable, optional). Defaults to None.) – A function to determine if the streaming should stop. The log streaming will stop if the function returns true.

  • time_start (datetime.datetime) – Starting time for the log query. Defaults to None. Logs up to 14 days from now will be returned.

Returns

Nothing

Return type

None

tail(limit=100, time_start: Optional[datetime] = None) → None

Prints the most recent log records.

Parameters

  • limit ((int, optional). Defaults to 100.) – Maximum number of records to be returned.

  • time_start ((datetime.datetime, optional)) – Starting time for the log query. Defaults to None. Logs up to 14 days from now will be returned.

Returns

Nothing

Return type

None

class ads.model.deployment.model_deployment.ModelDeploymentLogType

Bases: object

ACCESS = 'access'
PREDICT = 'predict'

ads.model.deployment.model_deployment_properties module

class ads.model.deployment.model_deployment_properties.ModelDeploymentProperties(model_id: Optional[str] = None, model_uri: Optional[str] = None, oci_model_deployment: Optional[Union[ModelDeployment, CreateModelDeploymentDetails, UpdateModelDeploymentDetails, dict]] = None, config: Optional[dict] = None, **kwargs)

Bases: OCIDataScienceMixin, ModelDeployment

Represents the details for a model deployment

swagger_types

The property names and the corresponding types of OCI ModelDeployment model.

Type

dict

model_id

The model artifact OCID in model catalog.

Type

str

model_uri

uri to model files, can be local or in cloud storage.

Type

str

with_prop(property_name, value)

Set the model deployment details property_name attribute to value

with_instance_configuration(config)

Set the configuration of VM instance.

with_access_log(log_group_id, log_id)

Config the access log with OCI logging service

with_predict_log(log_group_id, log_id)

Config the predict log with OCI logging service

build()

Return an instance of CreateModelDeploymentDetails for creating the deployment.

Initialize a ModelDeploymentProperties object by specifying one of the followings:

Parameters

  • model_id (str) – Model Artifact OCID. The model_id must be specified either explicitly or as an attribute of the OCI object.

  • model_uri (str) – uri to model files, can be local or in cloud storage.

  • oci_model_deployment (ModelDeployment or CreateModelDeploymentDetails or UpdateModelDeploymentDetails or dict) – An OCI model or dict containing model deployment details. The OCI model can be an instance of either ModelDeployment, CreateModelDeploymentDetails or UpdateModelConfigurationDetails.

  • config (dict) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.

  • kwargs –

    Users can also initialize the object by using keyword arguments. The following keyword arguments are supported by OCI models:

    • display_name,

    • description,

    • project_id,

    • compartment_id,

    • model_deployment_configuration_details,

    • category_log_details,

    • freeform_tags,

    • defined_tags.

    ModelDeploymentProperties also supports the following additional keyward arguments:

    • instance_shape,

    • instance_count,

    • bandwidth_mbps,

    • access_log_group_id,

    • access_log_id,

    • predict_log_group_id,

    • predict_log_id.

    These additional arguments will be saved into appropriate properties in the OCI model.

Raises

ValueError – model_id is None AND not specified in oci_model_deployment.model_deployment_configuration_details.model_configuration_details.

build() → CreateModelDeploymentDetails

Converts the deployment properties to OCI CreateModelDeploymentDetails object. Converts a model URI into a model OCID if user passed in a URI.

Returns

A CreateModelDeploymentDetails instance ready for OCI API.

Return type

CreateModelDeploymentDetails

sub_properties = ['instance_shape', 'instance_count', 'bandwidth_mbps', 'access_log_group_id', 'access_log_id', 'predict_log_group_id', 'predict_log_id']
to_oci_model(oci_model)

Convert properties into an OCI data model

Parameters

oci_model (class) – The class of OCI data model, e.g., oci.data_science_models.CreateModelDeploymentDetails

to_update_deployment() → UpdateModelDeploymentDetails

Converts the deployment properties to OCI UpdateModelDeploymentDetails object.

Returns

An UpdateModelDeploymentDetails instance ready for OCI API.

Return type

CreateModelDeploymentDetails

with_access_log(log_group_id: str, log_id: str)

Adds access log config

Parameters

  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns

self

Return type

ModelDeploymentProperties

with_category_log(log_type: str, group_id: str, log_id: str)

Adds category log configuration

Parameters

  • log_type (str) – The type of logging to be configured. Must be “access” or “predict”

  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns

self

Return type

ModelDeploymentProperties

Raises

ValueError – When log_type is invalid

with_instance_configuration(config)

with_instance_configuration creates a ModelDeploymentDetails object with a specific config

Parameters

config (dict) –

dictionary containing instance configuration about the deployment. The following keys are supported:

  • instance_shape,

  • instance_count,

  • bandwidth_mbps.

The instance_shape and instance_count are required when creating a new deployment. They are optional when updating an existing deployment.

Returns

self

Return type

ModelDeploymentProperties

with_logging_configuration(access_log_group_id: str, access_log_id: str, predict_log_group_id: Optional[str] = None, predict_log_id: Optional[str] = None)

Adds OCI logging configurations for OCI logging service

Parameters

  • access_log_group_id (str) – Log group ID of OCI logging service for access log

  • access_log_id (str) – Log ID of OCI logging service for access log

  • predict_log_group_id (str) – Log group ID of OCI logging service for predict log

  • predict_log_id (str) – Log ID of OCI logging service for predict log

Returns

self

Return type

ModelDeploymentProperties

with_predict_log(log_group_id: str, log_id: str)

Adds predict log config

Parameters

  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns

self

Return type

ModelDeploymentProperties

with_prop(property_name: str, value: Any)

Sets model deployment’s property_name attribute to value

Parameters

  • property_name (str) – Name of a model deployment property.

  • value – New value for property attribute.

Returns

self

Return type

ModelDeploymentProperties

Module contents