ads.model.deployment package

Subpackages

Submodules

ads.model.deployment.model_deployer module

APIs to interact with Oracle’s Model Deployment service.

There are three main classes: ModelDeployment, ModelDeploymentDetails, ModelDeployer.

One creates a ModelDeployment and deploys it under the umbrella of the ModelDeployer class. This way multiple ModelDeployments can be unified with one ModelDeployer. The ModelDeployer class also serves as the interface to all the deployments. ModelDeploymentDetails holds information about the particular details of a particular deployment, such as how many instances, etc. In this way multiple, independent ModelDeployments with the same details can be created using the ModelDeployer class.

Examples

>>> from model_deploy.model_deployer import ModelDeployer, ModelDeploymentDetails
>>> deployer = ModelDeployer("model_dep_conf.yaml")
>>> deployment_properties = ModelDeploymentProperties(
...             'ocid1.datasciencemodel.ocn.reg.xxxxxxxxxxxxxxxxxxxxxxxxx')
...                 .with_prop('display_name', "My model display name")
...                 .with_prop("project_id", project_id)
...                 .with_prop("compartment_id", compartment_id)
...                 .with_instance_configuration(
...                     config={"INSTANCE_SHAPE":"VM.Standard.E3.Flex",
...                             "INSTANCE_COUNT":"1",
...                             "bandwidth_mbps":10,
...                             "memory_in_gbs":10,
...                             "ocpus":2}
...                  ).build()
>>> deployment_info = deployer.deploy(deployment_properties,
...             max_wait_time=600, poll_interval=15)
>>> print(deployment_info.model_deployment_id)
>>> print(deployment_info.workflow_req_id)
>>> print(deployment_info.url)
>>> deployer.list_deployments() # Optionally pass in a status
class ads.model.deployment.model_deployer.ModelDeployer(config: dict | None = None, ds_client: DataScienceClient | None = None)[source]

Bases: object

ModelDeployer is the class responsible for deploying the ModelDeployment

config

ADS auth dictionary for OCI authentication.

Type:

dict

ds_client

data science client

Type:

DataScienceClient

ds_composite_client

composite data science client

Type:

DataScienceCompositeClient

deploy(model_deployment_details, \*\*kwargs)[source]

Deploy the model specified by model_deployment_details.

get_model_deployment(model_deployment_id: str)[source]

Get the ModelDeployment specified by model_deployment_id.

get_model_deployment_state(model_deployment_id)[source]

Get the state of the current deployment specified by id.

delete(model_deployment_id, \*\*kwargs)[source]

Remove the model deployment specified by the id or Model Deployment Object

list_deployments(status)[source]

lists the model deployments associated with current compartment and data science client

show_deployments(status)[source]

shows the deployments filtered by status in a Dataframe

Initializes model deployer.

Parameters:

config (dict, optional) –

ADS auth dictionary for OCI authentication.

This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.

ds_client: oci.data_science.data_science_client.DataScienceClient

The Oracle DataScience client.

delete(model_deployment_id, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10) → ModelDeployment[source]

Deletes the model deployment specified by OCID.

Parameters:

  • model_deployment_id (str) – Model deployment OCID.

  • wait_for_completion (bool) – Wait for deletion to complete. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 60).

Return type:

A ModelDeployment instance that was deleted

deploy(properties: ModelDeploymentProperties | Dict | None = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs) → ModelDeployment[source]

Deploys a model.

Parameters:

  • properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Optional, defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds. Optional, defaults to 1200. Negative value implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds. Optional, defaults to 30.

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties. See ModelDeploymentProperties() for details.

Returns:

A ModelDeployment instance.

Return type:

ModelDeployment

deploy_from_model_uri(model_uri: str, properties: ModelDeploymentProperties | Dict | None = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs) → ModelDeployment[source]

Deploys a model.

Parameters:

  • model_uri (str) – uri to model files, can be local or in cloud storage

  • properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 30).

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties

Returns:

A ModelDeployment instance

Return type:

ModelDeployment

get_model_deployment(model_deployment_id: str) → ModelDeployment[source]

Gets a ModelDeployment by OCID.

Parameters:

model_deployment_id (str) – Model deployment OCID

Returns:

A ModelDeployment instance

Return type:

ModelDeployment

get_model_deployment_state(model_deployment_id: str) → State[source]

Gets the state of a deployment specified by OCID

Parameters:

model_deployment_id (str) – Model deployment OCID

Returns:

The state of the deployment

Return type:

str

list_deployments(status=None, compartment_id=None, **kwargs) → list[source]

Lists the model deployments associated with current compartment and data science client

Parameters:

  • status (str) – Status of deployment. Defaults to None.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

  • kwargs – The values are passed to oci.data_science.DataScienceClient.list_model_deployments.

Returns:

A list of ModelDeployment objects.

Return type:

list

Raises:

ValueError – If compartment_id is not specified and cannot be determined from the environment.

show_deployments(status=None, compartment_id=None) → DataFrame[source]
Returns the model deployments associated with current compartment and data science client

as a Dataframe that can be easily visualized

Parameters:

  • status (str) – Status of deployment. Defaults to None.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

Returns:

pandas Dataframe containing information about the ModelDeployments

Return type:

DataFrame

Raises:

ValueError – If compartment_id is not specified and cannot be determined from the environment.

update(model_deployment_id: str, properties: ModelDeploymentProperties | None = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs) → ModelDeployment[source]

Updates an existing model deployment.

Parameters:

  • model_deployment_id (str) – Model deployment OCID.

  • properties (ModelDeploymentProperties) – An instance of ModelDeploymentProperties or dict to initialize the ModelDeploymentProperties. Defaults to None.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200).

  • poll_interval (int) – Poll interval in seconds (Defaults to 30).

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties.

Returns:

A ModelDeployment instance

Return type:

ModelDeployment

ads.model.deployment.model_deployment module

exception ads.model.deployment.model_deployment.LogNotConfiguredError[source]

Bases: Exception

class ads.model.deployment.model_deployment.ModelDeployment(properties: ModelDeploymentProperties | Dict | None = None, config: Dict | None = None, model_deployment_id: str | None = None, model_deployment_url: str = '', spec: Dict | None = None, **kwargs)[source]

Bases: Builder

A class used to represent a Model Deployment.

config

Deployment configuration parameters

Type:

(dict)

properties

ModelDeploymentProperties object

Type:

(ModelDeploymentProperties)

workflow_state_progress

Workflow request id

Type:

(str)

workflow_steps

The number of steps in the workflow

Type:

(int)

dsc_model_deployment

The OCIDataScienceModelDeployment instance.

Type:

(OCIDataScienceModelDeployment)

state

Returns the deployment state of the current Model Deployment object

Type:

(State)

created_by

The user that creates the model deployment

Type:

(str)

lifecycle_state

Model deployment lifecycle state

Type:

(str)

lifecycle_details

Model deployment lifecycle details

Type:

(str)

time_created

The time when the model deployment is created

Type:

(datetime)

display_name

Model deployment display name

Type:

(str)

description

Model deployment description

Type:

(str)

freeform_tags

Model deployment freeform tags

Type:

(dict)

defined_tags

Model deployment defined tags

Type:

(dict)

runtime

Model deployment runtime

Type:

(ModelDeploymentRuntime)

infrastructure

Model deployment infrastructure

Type:

(ModelDeploymentInfrastructure)

deploy(wait_for_completion, \*\*kwargs)[source]

Deploy the current Model Deployment object

delete(wait_for_completion, \*\*kwargs)[source]

Deletes the current Model Deployment object

update(wait_for_completion, \*\*kwargs)[source]

Updates a model deployment

activate(wait_for_completion, max_wait_time, poll_interval)[source]

Activates a model deployment

deactivate(wait_for_completion, max_wait_time, poll_interval)[source]

Deactivates a model deployment

list(status, compartment_id, project_id, \*\*kwargs)[source]

List model deployment within given compartment and project.

with_display_name(display_name)[source]

Sets model deployment display name

with_description(description)[source]

Sets model deployment description

with_freeform_tags(freeform_tags)[source]

Sets model deployment freeform tags

with_defined_tags(defined_tags)[source]

Sets model deployment defined tags

with_runtime(self, runtime)[source]

Sets model deployment runtime

with_infrastructure(self, infrastructure)[source]

Sets model deployment infrastructure

from_dict(obj_dict)[source]

Deserializes model deployment instance from dict

from_id(id)[source]

Loads model deployment instance from ocid

sync()[source]

Updates the model deployment instance from backend

Examples

>>> # Build model deployment from builder apis:
>>> ds_model_deployment = (ModelDeployment()
...    .with_display_name("TestModelDeployment")
...    .with_description("Testing the test model deployment")
...    .with_freeform_tags(tag1="val1", tag2="val2")
...    .with_infrastructure(
...        (ModelDeploymentInfrastructure()
...        .with_project_id(<project_id>)
...        .with_compartment_id(<compartment_id>)
...        .with_shape_name("VM.Standard.E4.Flex")
...        .with_shape_config_details(
...            ocpus=1,
...            memory_in_gbs=16
...        )
...        .with_replica(1)
...        .with_bandwidth_mbps(10)
...        .with_web_concurrency(10)
...        .with_access_log(
...            log_group_id=<log_group_id>,
...            log_id=<log_id>
...        )
...        .with_predict_log(
...            log_group_id=<log_group_id>,
...            log_id=<log_id>
...        ))
...    )
...    .with_runtime(
...        (ModelDeploymentContainerRuntime()
...        .with_image(<image>)
...        .with_image_digest(<image_digest>)
...        .with_entrypoint(<entrypoint>)
...        .with_server_port(<server_port>)
...        .with_health_check_port(<health_check_port>)
...        .with_env({"key":"value"})
...        .with_deployment_mode("HTTPS_ONLY")
...        .with_model_uri(<model_uri>)
...        .with_bucket_uri(<bucket_uri>)
...        .with_auth(<auth>)
...        .with_timeout(<time_out>))
...    )
... )
>>> ds_model_deployment.deploy()
>>> ds_model_deployment.status
>>> ds_model_deployment.with_display_name("new name").update()
>>> ds_model_deployment.deactivate()
>>> ds_model_deployment.sync()
>>> ds_model_deployment.list(status="ACTIVE")
>>> ds_model_deployment.delete()

>>> # Build model deployment from yaml
>>> ds_model_deployment = ModelDeployment.from_yaml(uri=<path_to_yaml>)

Initializes a ModelDeployment object.

Parameters:

  • properties ((Union[ModelDeploymentProperties, Dict], optional). Defaults to None.) – Object containing deployment properties. The properties can be None when kwargs are used for specifying properties.

  • config ((Dict, optional). Defaults to None.) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None then the ads.common.default_signer(client_kwargs) will be used.

  • model_deployment_id ((str, optional). Defaults to None.) – Model deployment OCID.

  • model_deployment_url ((str, optional). Defaults to empty string.) – Model deployment url.

  • spec ((dict, optional). Defaults to None.) – Model deployment spec.

  • kwargs – Keyword arguments for initializing ModelDeploymentProperties or ModelDeployment.

CONST_CREATED_BY = 'createdBy'
CONST_DEFINED_TAG = 'definedTags'
CONST_DESCRIPTION = 'description'
CONST_DISPLAY_NAME = 'displayName'
CONST_FREEFORM_TAG = 'freeformTags'
CONST_ID = 'id'
CONST_INFRASTRUCTURE = 'infrastructure'
CONST_LIFECYCLE_DETAILS = 'lifecycleDetails'
CONST_LIFECYCLE_STATE = 'lifecycleState'
CONST_MODEL_DEPLOYMENT_URL = 'modelDeploymentUrl'
CONST_RUNTIME = 'runtime'
CONST_TIME_CREATED = 'timeCreated'
property access_log: OCILog

Gets the model deployment access logs object.

Returns:

The OCILog object containing the access logs.

Return type:

OCILog

activate(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10) → ModelDeployment[source]

Activates a model deployment

Parameters:

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be activated before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

attribute_map = {'createdBy': 'created_by', 'definedTags': 'defined_tags', 'description': 'description', 'displayName': 'display_name', 'freeformTags': 'freeform_tags', 'id': 'id', 'infrastructure': 'infrastructure', 'lifecycleDetails': 'lifecycle_details', 'lifecycleState': 'lifecycle_state', 'modelDeploymentUrl': 'model_deployment_url', 'runtime': 'runtime', 'timeCreated': 'time_created'}
build() → ModelDeployment[source]

Load default values from the environment for the job infrastructure.

property created_by: str

The user that creates the model deployment.

Returns:

The user that creates the model deployment.

Return type:

str

deactivate(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10) → ModelDeployment[source]

Deactivates a model deployment

Parameters:

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be deactivated before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

property defined_tags: Dict

Model deployment defined tags.

Returns:

Model deployment defined tags.

Return type:

Dict

delete(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10)[source]

Deletes the ModelDeployment

Parameters:

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be deleted before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

deploy(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10)[source]

Deploys the current ModelDeployment object

Parameters:

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be deployed before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

property description: str

Model deployment description.

Returns:

Model deployment description.

Return type:

str

property display_name: str

Model deployment display name.

Returns:

Model deployment display name.

Return type:

str

property freeform_tags: Dict

Model deployment freeform tags.

Returns:

Model deployment freeform tags.

Return type:

Dict

classmethod from_dict(obj_dict: Dict) → ModelDeployment[source]

Loads model deployment instance from a dictionary of configurations.

Parameters:

obj_dict (Dict) – A dictionary of configurations.

Returns:

The model deployment instance.

Return type:

ModelDeployment

classmethod from_id(id: str) → ModelDeployment[source]

Loads the model deployment instance from ocid.

Parameters:

id (str) – The ocid of model deployment.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

property id: str

The model deployment ocid.

Returns:

The model deployment ocid.

Return type:

str

property infrastructure: ModelDeploymentInfrastructure

Model deployment infrastructure.

Returns:

Model deployment infrastructure.

Return type:

ModelDeploymentInfrastructure

initialize_spec_attributes = ['display_name', 'description', 'freeform_tags', 'defined_tags', 'infrastructure', 'runtime']
property kind: str

The kind of the object as showing in YAML.

Returns:

deployment

Return type:

str

property lifecycle_details: str

Model deployment lifecycle details.

Returns:

Model deployment lifecycle details.

Return type:

str

property lifecycle_state: str

Model deployment lifecycle state.

Returns:

Model deployment lifecycle state.

Return type:

str

classmethod list(status: str | None = None, compartment_id: str | None = None, project_id: str | None = None, **kwargs) → List[ModelDeployment][source]

Lists the model deployments associated with current compartment id and status

Parameters:

  • status (str) – Status of deployment. Defaults to None. Allowed values: ACTIVE, CREATING, DELETED, DELETING, FAILED, INACTIVE and UPDATING.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

  • project_id (str) – Target project to list deployments from. Defaults to the project id in the environment variable “PROJECT_OCID”.

  • kwargs – The values are passed to oci.data_science.DataScienceClient.list_model_deployments.

Returns:

A list of ModelDeployment objects.

Return type:

list

classmethod list_df(status: str | None = None, compartment_id: str | None = None, project_id: str | None = None) → DataFrame[source]
Returns the model deployments associated with current compartment and status

as a Dataframe that can be easily visualized

Parameters:

  • status (str) – Status of deployment. Defaults to None. Allowed values: ACTIVE, CREATING, DELETED, DELETING, FAILED, INACTIVE and UPDATING.

  • compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.

  • project_id (str) – Target project to list deployments from. Defaults to the project id in the environment variable “PROJECT_OCID”.

Returns:

pandas Dataframe containing information about the ModelDeployments

Return type:

DataFrame

logs(log_type: str | None = None) → ConsolidatedLog[source]

Gets the access or predict logs.

Parameters:

log_type ((str, optional). Defaults to None.) – The log type. Can be “access”, “predict” or None.

Returns:

The ConsolidatedLog object containing the logs.

Return type:

ConsolidatedLog

property model_deployment_id: str

The model deployment ocid.

Returns:

The model deployment ocid.

Return type:

str

model_input_serializer = <ads.model.serde.model_input.JsonModelInputSERDE object>
predict(json_input=None, data: ~typing.Any = None, serializer: ads.model.ModelInputSerializer = <ads.model.serde.model_input.JsonModelInputSERDE object>, auto_serialize_data: bool = False, model_name: str = None, model_version: str = None, **kwargs) → dict[source]

Returns prediction of input data run against the model deployment endpoint.

Examples

>>> import numpy as np
>>> from ads.model import ModelInputSerializer
>>> class MySerializer(ModelInputSerializer):
...     def serialize(self, data):
...         serialized_data = 1
...         return serialized_data
>>> model_deployment = ModelDeployment.from_id(<model_deployment_id>)
>>> prediction = model_deployment.predict(
...        data=np.array([1, 2, 3]),
...        serializer=MySerializer(),
...        auto_serialize_data=True,
... )['prediction']
Parameters:

  • json_input (Json serializable) – JSON payload for the prediction.

  • data (Any) – Data for the prediction.

  • serializer (ads.model.ModelInputSerializer) – Defaults to ads.model.JsonModelInputSerializer.

  • auto_serialize_data (bool) – Defaults to False. Indicate whether to auto serialize input data using serializer. If auto_serialize_data=False, data required to be bytes or json serializable and json_input required to be json serializable. If auto_serialize_data set to True, data will be serialized before sending to model deployment endpoint.

  • model_name (str) – Defaults to None. When the inference_server=”triton”, the name of the model to invoke.

  • model_version (str) – Defaults to None. When the inference_server=”triton”, the version of the model to invoke.

  • kwargs –

    content_type: str

    Used to indicate the media type of the resource. By default, it will be application/octet-stream for bytes input and application/json otherwise. The content-type header will be set to this value when calling the model deployment endpoint.

Returns:

Prediction results.

Return type:

dict

property predict_log: OCILog

Gets the model deployment predict logs object.

Returns:

The OCILog object containing the predict logs.

Return type:

OCILog

property runtime: ModelDeploymentRuntime

Model deployment runtime.

Returns:

Model deployment runtime.

Return type:

ModelDeploymentRuntime

show_logs(time_start: datetime | None = None, time_end: datetime | None = None, limit: int = 100, log_type: str | None = None)[source]

Shows deployment logs as a pandas dataframe.

Parameters:

  • time_start ((datetime.datetime, optional). Defaults to None.) – Starting date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved 14 days from now.

  • time_end ((datetime.datetime, optional). Defaults to None.) – Ending date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved until now.

  • limit ((int, optional). Defaults to 100.) – The maximum number of items to return.

  • log_type ((str, optional). Defaults to None.) – The log type. Can be “access”, “predict” or None.

Return type:

A pandas DataFrame containing logs.

property state: State

Returns the deployment state of the current Model Deployment object

property status: State

Returns the deployment state of the current Model Deployment object

sync() → ModelDeployment[source]

Updates the model deployment instance from backend.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

property time_created: <module 'datetime' from '/home/docs/.asdf/installs/python/3.9.19/lib/python3.9/datetime.py'>

The time when the model deployment is created.

Returns:

The time when the model deployment is created.

Return type:

datetime

to_dict(**kwargs) → Dict[source]

Serializes model deployment to a dictionary.

Returns:

The model deployment serialized as a dictionary.

Return type:

dict

property type: str

The type of the object as showing in YAML.

Returns:

deployment

Return type:

str

update(properties: ModelDeploymentProperties | dict | None = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 10, **kwargs)[source]

Updates a model deployment

You can update model_deployment_configuration_details and change instance_shape and model_id when the model deployment is in the ACTIVE lifecycle state. The bandwidth_mbps or instance_count can only be updated while the model deployment is in the INACTIVE state. Changes to the bandwidth_mbps or instance_count will take effect the next time the ActivateModelDeployment action is invoked on the model deployment resource.

Parameters:

  • properties (ModelDeploymentProperties or dict) – The properties for updating the deployment.

  • wait_for_completion (bool) – Flag set for whether to wait for deployment to be updated before proceeding. Defaults to True.

  • max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.

  • poll_interval (int) – Poll interval in seconds (Defaults to 10).

  • kwargs – dict

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

property url: str

Model deployment url.

Returns:

Model deployment url.

Return type:

str

watch(log_type: str = None, time_start: <module 'datetime' from '/home/docs/.asdf/installs/python/3.9.19/lib/python3.9/datetime.py'> = None, interval: int = 3, log_filter: str = None) → ModelDeployment[source]

Streams the access and/or predict logs of model deployment.

Parameters:

  • log_type (str, optional) – The log type. Can be access, predict or None. Defaults to None.

  • time_start (datetime.datetime, optional) – Starting time for the log query. Defaults to None.

  • interval (int, optional) – The time interval between sending each request to pull logs from OCI logging service. Defaults to 3.

  • log_filter (str, optional) – Expression for filtering the logs. This will be the WHERE clause of the query. Defaults to None.

Returns:

The instance of ModelDeployment.

Return type:

ModelDeployment

with_defined_tags(**kwargs) → ModelDeployment[source]

Sets the defined tags of model deployment.

Parameters:

kwargs – The defined tags of model deployment.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

with_description(description: str) → ModelDeployment[source]

Sets the description of model deployment.

Parameters:

description (str) – The description of model deployment.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

with_display_name(display_name: str) → ModelDeployment[source]

Sets the name of model deployment.

Parameters:

display_name (str) – The name of model deployment.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

with_freeform_tags(**kwargs) → ModelDeployment[source]

Sets the freeform tags of model deployment.

Parameters:

kwargs – The freeform tags of model deployment.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

with_infrastructure(infrastructure: ModelDeploymentInfrastructure) → ModelDeployment[source]

Sets the infrastructure of model deployment.

Parameters:

infrastructure (ModelDeploymentInfrastructure) – The infrastructure of model deployment.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

with_runtime(runtime: ModelDeploymentRuntime) → ModelDeployment[source]

Sets the runtime of model deployment.

Parameters:

runtime (ModelDeploymentRuntime) – The runtime of model deployment.

Returns:

The ModelDeployment instance (self).

Return type:

ModelDeployment

class ads.model.deployment.model_deployment.ModelDeploymentLogType[source]

Bases: object

ACCESS = 'access'
PREDICT = 'predict'
exception ads.model.deployment.model_deployment.ModelDeploymentPredictError[source]

Bases: Exception

ads.model.deployment.model_deployment_properties module

class ads.model.deployment.model_deployment_properties.ModelDeploymentProperties(model_id: str | None = None, model_uri: str | None = None, oci_model_deployment: ModelDeployment | CreateModelDeploymentDetails | UpdateModelDeploymentDetails | Dict | None = None, config: dict | None = None, **kwargs)[source]

Bases: OCIDataScienceMixin, ModelDeployment

Represents the details for a model deployment

swagger_types

The property names and the corresponding types of OCI ModelDeployment model.

Type:

dict

model_id

The model artifact OCID in model catalog.

Type:

str

model_uri

uri to model files, can be local or in cloud storage.

Type:

str

with_prop(property_name, value)[source]

Set the model deployment details property_name attribute to value

with_instance_configuration(config)[source]

Set the configuration of VM instance.

with_access_log(log_group_id, log_id)[source]

Config the access log with OCI logging service

with_predict_log(log_group_id, log_id)[source]

Config the predict log with OCI logging service

build()[source]

Return an instance of CreateModelDeploymentDetails for creating the deployment.

Initialize a ModelDeploymentProperties object by specifying one of the followings:

Parameters:

  • model_id ((str, optiona). Defaults to None.) – Model Artifact OCID. The model_id must be specified either explicitly or as an attribute of the OCI object.

  • model_uri ((str, optiona). Defaults to None.) – Uri to model files, can be local or in cloud storage.

  • oci_model_deployment ((Union[ModelDeployment, CreateModelDeploymentDetails, UpdateModelDeploymentDetails, Dict], optional). Defaults to None.) – An OCI model or Dict containing model deployment details. The OCI model can be an instance of either ModelDeployment, CreateModelDeploymentDetails or UpdateModelConfigurationDetails.

  • config ((Dict, optional). Defaults to None.) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.

  • kwargs –

    Users can also initialize the object by using keyword arguments. The following keyword arguments are supported by oci.data_science.models.data_science_models.ModelDeployment:

    • display_name,

    • description,

    • project_id,

    • compartment_id,

    • model_deployment_configuration_details,

    • category_log_details,

    • freeform_tags,

    • defined_tags.

    If display_name is not specified, a randomly generated easy to remember name will be generated, like ‘strange-spider-2022-08-17-23:55.02’.

    ModelDeploymentProperties also supports the following additional keyward arguments:

    • instance_shape,

    • instance_count,

    • bandwidth_mbps,

    • access_log_group_id,

    • access_log_id,

    • predict_log_group_id,

    • predict_log_id,

    • memory_in_gbs,

    • ocpus.

    These additional arguments will be saved into appropriate properties in the OCI model.

Raises:

ValueError – model_id is None AND not specified in oci_model_deployment.model_deployment_configuration_details.model_configuration_details.

build() → CreateModelDeploymentDetails[source]

Converts the deployment properties to OCI CreateModelDeploymentDetails object. Converts a model URI into a model OCID if user passed in a URI.

Returns:

A CreateModelDeploymentDetails instance ready for OCI API.

Return type:

CreateModelDeploymentDetails

sub_properties = ['instance_shape', 'instance_count', 'bandwidth_mbps', 'access_log_group_id', 'access_log_id', 'predict_log_group_id', 'predict_log_id', 'memory_in_gbs', 'ocpus']
to_oci_model(oci_model)[source]

Convert properties into an OCI data model

Parameters:

oci_model (class) – The class of OCI data model, e.g., oci.data_science_models.CreateModelDeploymentDetails

to_update_deployment() → UpdateModelDeploymentDetails[source]

Converts the deployment properties to OCI UpdateModelDeploymentDetails object.

Returns:

An UpdateModelDeploymentDetails instance ready for OCI API.

Return type:

CreateModelDeploymentDetails

with_access_log(log_group_id: str, log_id: str)[source]

Adds access log config

Parameters:

  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns:

self

Return type:

ModelDeploymentProperties

with_category_log(log_type: str, group_id: str, log_id: str)[source]

Adds category log configuration

Parameters:

  • log_type (str) – The type of logging to be configured. Must be “access” or “predict”

  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns:

self

Return type:

ModelDeploymentProperties

Raises:

ValueError – When log_type is invalid

with_instance_configuration(config)[source]

with_instance_configuration creates a ModelDeploymentDetails object with a specific config

Parameters:

config (dict) –

dictionary containing instance configuration about the deployment. The following keys are supported:

  • instance_shape: str,

  • instance_count: int,

  • bandwidth_mbps: int,

  • memory_in_gbs: float,

  • ocpus: float

The instance_shape and instance_count are required when creating a new deployment. They are optional when updating an existing deployment.

Returns:

self

Return type:

ModelDeploymentProperties

with_logging_configuration(access_log_group_id: str, access_log_id: str, predict_log_group_id: str | None = None, predict_log_id: str | None = None)[source]

Adds OCI logging configurations for OCI logging service

Parameters:

  • access_log_group_id (str) – Log group ID of OCI logging service for access log

  • access_log_id (str) – Log ID of OCI logging service for access log

  • predict_log_group_id (str) – Log group ID of OCI logging service for predict log

  • predict_log_id (str) – Log ID of OCI logging service for predict log

Returns:

self

Return type:

ModelDeploymentProperties

with_predict_log(log_group_id: str, log_id: str)[source]

Adds predict log config

Parameters:

  • group_id (str) – Log group ID of OCI logging service

  • log_id (str) – Log ID of OCI logging service

Returns:

self

Return type:

ModelDeploymentProperties

with_prop(property_name: str, value: Any)[source]

Sets model deployment’s property_name attribute to value

Parameters:

  • property_name (str) – Name of a model deployment property.

  • value – New value for property attribute.

Returns:

self

Return type:

ModelDeploymentProperties

Module contents