ads.model.deployment package
Submodules
ads.model.deployment.model_deployer module
APIs to interact with Oracle’s Model Deployment service.
There are three main classes: ModelDeployment, ModelDeploymentDetails, ModelDeployer.
One creates a ModelDeployment and deploys it under the umbrella of the ModelDeployer class. This way multiple ModelDeployments can be unified with one ModelDeployer. The ModelDeployer class also serves as the interface to all the deployments. ModelDeploymentDetails holds information about the particular details of a particular deployment, such as how many instances, etc. In this way multiple, independent ModelDeployments with the same details can be created using the ModelDeployer class.
Examples
>>> from model_deploy.model_deployer import ModelDeployer, ModelDeploymentDetails
>>> deployer = ModelDeployer("model_dep_conf.yaml")
>>> deployment_properties = ModelDeploymentProperties(
... 'ocid1.datasciencemodel.ocn.reg.xxxxxxxxxxxxxxxxxxxxxxxxx')
... .with_prop('display_name', "My model display name")
... .with_prop("project_id", project_id)
... .with_prop("compartment_id", compartment_id)
... .with_instance_configuration(
... config={"INSTANCE_SHAPE":"VM.Standard.E3.Flex",
... "INSTANCE_COUNT":"1",
... "bandwidth_mbps":10,
... "memory_in_gbs":10,
... "ocpus":2}
... ).build()
>>> deployment_info = deployer.deploy(deployment_properties,
... max_wait_time=600, poll_interval=15)
>>> print(deployment_info.model_deployment_id)
>>> print(deployment_info.workflow_req_id)
>>> print(deployment_info.url)
>>> deployer.list_deployments() # Optionally pass in a status
- class ads.model.deployment.model_deployer.ModelDeployer(config: Optional[dict] = None, ds_client: Optional[DataScienceClient] = None)
Bases:
object
ModelDeployer is the class responsible for deploying the ModelDeployment
- config
ADS auth dictionary for OCI authentication.
- Type
dict
- ds_client
data science client
- Type
DataScienceClient
- ds_composite_client
composite data science client
- Type
DataScienceCompositeClient
- deploy(model_deployment_details, \*\*kwargs)
Deploy the model specified by model_deployment_details.
- get_model_deployment(model_deployment_id: str)
Get the ModelDeployment specified by model_deployment_id.
- get_model_deployment_state(model_deployment_id)
Get the state of the current deployment specified by id.
- delete(model_deployment_id, \*\*kwargs)
Remove the model deployment specified by the id or Model Deployment Object
- list_deployments(status)
lists the model deployments associated with current compartment and data science client
- show_deployments(status)
shows the deployments filtered by status in a Dataframe
Initializes model deployer.
- Parameters
config (dict, optional) –
- ADS auth dictionary for OCI authentication.
This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.
- ds_client: oci.data_science.data_science_client.DataScienceClient
The Oracle DataScience client.
- delete(model_deployment_id, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30) ModelDeployment
Deletes the model deployment specified by OCID.
- Parameters
model_deployment_id (str) – Model deployment OCID.
wait_for_completion (bool) – Wait for deletion to complete. Defaults to True.
max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.
poll_interval (int) – Poll interval in seconds (Defaults to 60).
- Return type
A ModelDeployment instance that was deleted
- deploy(properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs) ModelDeployment
Deploys a model.
- Parameters
properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.
wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Optional, defaults to True.
max_wait_time (int) – Maximum amount of time to wait in seconds. Optional, defaults to 1200. Negative value implies infinite wait time.
poll_interval (int) – Poll interval in seconds. Optional, defaults to 30.
kwargs – Keyword arguments for initializing ModelDeploymentProperties. See ModelDeploymentProperties() for details.
- Returns
A ModelDeployment instance.
- Return type
- deploy_from_model_uri(model_uri: str, properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs) ModelDeployment
Deploys a model.
- Parameters
model_uri (str) – uri to model files, can be local or in cloud storage
properties (ModelDeploymentProperties or dict) – Properties to deploy the model. Properties can be None when kwargs are used for specifying properties.
wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True
max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.
poll_interval (int) – Poll interval in seconds (Defaults to 30).
kwargs – Keyword arguments for initializing ModelDeploymentProperties
- Returns
A ModelDeployment instance
- Return type
- get_model_deployment(model_deployment_id: str) ModelDeployment
Gets a ModelDeployment by OCID.
- Parameters
model_deployment_id (str) – Model deployment OCID
- Returns
A ModelDeployment instance
- Return type
- get_model_deployment_state(model_deployment_id: str) State
Gets the state of a deployment specified by OCID
- Parameters
model_deployment_id (str) – Model deployment OCID
- Returns
The state of the deployment
- Return type
str
- list_deployments(status=None, compartment_id=None, **kwargs) list
Lists the model deployments associated with current compartment and data science client
- Parameters
status (str) – Status of deployment. Defaults to None.
compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.
kwargs – The values are passed to oci.data_science.DataScienceClient.list_model_deployments.
- Returns
A list of ModelDeployment objects.
- Return type
list
- Raises
ValueError – If compartment_id is not specified and cannot be determined from the environment.
- show_deployments(status=None, compartment_id=None) DataFrame
- Returns the model deployments associated with current compartment and data science client
as a Dataframe that can be easily visualized
- Parameters
status (str) – Status of deployment. Defaults to None.
compartment_id (str) – Target compartment to list deployments from. Defaults to the compartment set in the environment variable “NB_SESSION_COMPARTMENT_OCID”. If “NB_SESSION_COMPARTMENT_OCID” is not set, the root compartment ID will be used. An ValueError will be raised if root compartment ID cannot be determined.
- Returns
pandas Dataframe containing information about the ModelDeployments
- Return type
DataFrame
- Raises
ValueError – If compartment_id is not specified and cannot be determined from the environment.
- update(model_deployment_id: str, properties: Optional[ModelDeploymentProperties] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs) ModelDeployment
Updates an existing model deployment.
- Parameters
model_deployment_id (str) – Model deployment OCID.
properties (ModelDeploymentProperties) – An instance of ModelDeploymentProperties or dict to initialize the ModelDeploymentProperties. Defaults to None.
wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.
max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200).
poll_interval (int) – Poll interval in seconds (Defaults to 30).
kwargs – Keyword arguments for initializing ModelDeploymentProperties.
- Returns
A ModelDeployment instance
- Return type
ads.model.deployment.model_deployment module
- exception ads.model.deployment.model_deployment.LogNotConfiguredError
Bases:
Exception
- class ads.model.deployment.model_deployment.ModelDeployment(properties: Optional[Union[ModelDeploymentProperties, Dict]] = None, config: Optional[Dict] = None, workflow_req_id: Optional[str] = None, model_deployment_id: Optional[str] = None, model_deployment_url: str = '', **kwargs)
Bases:
object
A class used to represent a Model Deployment.
- config
Deployment configuration parameters
- Type
(dict)
- properties
ModelDeploymentProperties object
- Type
- workflow_state_progress
Workflow request id
- Type
(str)
- workflow_steps
The number of steps in the workflow
- Type
(int)
- url
The model deployment url endpoint
- Type
(str)
- ds_client
The data science client used by model deployment
- Type
(DataScienceClient)
- ds_composite_client
The composite data science client used by the model deployment
- Type
(DataScienceCompositeClient)
- workflow_req_id
Workflow request id
- Type
(str)
- model_deployment_id
model deployment id
- Type
(str)
- deploy(wait_for_completion, \*\*kwargs)
Deploy the current Model Deployment object
- delete(wait_for_completion, \*\*kwargs)
Deletes the current Model Deployment object
- update(wait_for_completion, \*\*kwargs)
Updates a model deployment
- list_workflow_logs()
Returns a list of the steps involved in deploying a model
Initializes a ModelDeployment object.
- Parameters
properties ((Union[ModelDeploymentProperties, Dict], optional). Defaults to None.) – Object containing deployment properties. The properties can be None when kwargs are used for specifying properties.
config ((Dict, optional). Defaults to None.) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None then the ads.common.default_signer(client_kwargs) will be used.
workflow_req_id ((str, optional). Defaults to None.) – Workflow request id.
model_deployment_id ((str, optional). Defaults to None.) – Model deployment OCID.
model_deployment_url ((str, optional). Defaults to empty string.) – Model deployment url.
kwargs – Keyword arguments for initializing ModelDeploymentProperties.
- property access_log: OCILog
Gets the model deployment access logs object.
- Returns
The OCILog object containing the access logs.
- Return type
OCILog
- delete(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30)
Deletes the ModelDeployment
- Parameters
wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.
max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.
poll_interval (int) – Poll interval in seconds (Defaults to 60).
- Returns
The instance of ModelDeployment.
- Return type
- deploy(wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30)
deploy deploys the current ModelDeployment object
- Parameters
wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.
max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 600). Negative implies infinite wait time.
poll_interval (int) – Poll interval in seconds (Defaults to 60).
- Returns
The instance of ModelDeployment.
- Return type
- list_workflow_logs() list
Returns a list of the steps involved in deploying a model
- Returns
List of dictionaries detailing the status of each step in the deployment process.
- Return type
list
- logs(log_type: Optional[str] = None) ConsolidatedLog
Gets the access or predict logs.
- Parameters
log_type ((str, optional). Defaults to None.) – The log type. Can be “access”, “predict” or None.
- Returns
The ConsolidatedLog object containing the logs.
- Return type
ConsolidatedLog
- predict(json_input=None, data: Optional[Any] = None, auto_serialize_data: bool = False, **kwargs) dict
Returns prediction of input data run against the model deployment endpoint
- Parameters
json_input (Json serializable) – Json payload for the prediction.
data (Any) – Data for the prediction.
auto_serialize_data (bool.) – Whether to auto serialize input data. Defauls to False. If auto_serialize_data=False, data required to be bytes or json serializable and json_input required to be json serializable. If auto_serialize_data set to True, data will be serialized before sending to model deployment endpoint.
kwargs –
- content_type: str
Used to indicate the media type of the resource. By default, it will be application/octet-stream for bytes input and application/json otherwise. The content-type header will be set to this value when calling the model deployment endpoint.
- Returns
Prediction results.
- Return type
dict
- property predict_log: OCILog
Gets the model deployment predict logs object.
- Returns
The OCILog object containing the predict logs.
- Return type
OCILog
- show_logs(time_start: Optional[datetime] = None, time_end: Optional[datetime] = None, limit: int = 100, log_type: Optional[str] = None)
Shows deployment logs as a pandas dataframe.
- Parameters
time_start ((datetime.datetime, optional). Defaults to None.) – Starting date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved 14 days from now.
time_end ((datetime.datetime, optional). Defaults to None.) – Ending date and time in RFC3339 format for retrieving logs. Defaults to None. Logs will be retrieved until now.
limit ((int, optional). Defaults to 100.) – The maximum number of items to return.
log_type ((str, optional). Defaults to None.) – The log type. Can be “access”, “predict” or None.
- Return type
A pandas DataFrame containing logs.
- property state: State
Returns the deployment state of the current Model Deployment object
- property status: State
Returns the deployment state of the current Model Deployment object
- update(properties: Optional[Union[ModelDeploymentProperties, dict]] = None, wait_for_completion: bool = True, max_wait_time: int = 1200, poll_interval: int = 30, **kwargs)
Updates a model deployment
You can update model_deployment_configuration_details and change instance_shape and model_id when the model deployment is in the ACTIVE lifecycle state. The bandwidth_mbps or instance_count can only be updated while the model deployment is in the INACTIVE state. Changes to the bandwidth_mbps or instance_count will take effect the next time the ActivateModelDeployment action is invoked on the model deployment resource.
- Parameters
properties (ModelDeploymentProperties or dict) – The properties for updating the deployment.
wait_for_completion (bool) – Flag set for whether to wait for deployment to complete before proceeding. Defaults to True.
max_wait_time (int) – Maximum amount of time to wait in seconds (Defaults to 1200). Negative implies infinite wait time.
poll_interval (int) – Poll interval in seconds (Defaults to 60).
kwargs – dict
- Returns
The instance of ModelDeployment.
- Return type
ads.model.deployment.model_deployment_properties module
- class ads.model.deployment.model_deployment_properties.ModelDeploymentProperties(model_id: Optional[str] = None, model_uri: Optional[str] = None, oci_model_deployment: Optional[Union[ModelDeployment, CreateModelDeploymentDetails, UpdateModelDeploymentDetails, Dict]] = None, config: Optional[dict] = None, **kwargs)
Bases:
OCIDataScienceMixin
,ModelDeployment
Represents the details for a model deployment
- swagger_types
The property names and the corresponding types of OCI ModelDeployment model.
- Type
dict
- model_id
The model artifact OCID in model catalog.
- Type
str
- model_uri
uri to model files, can be local or in cloud storage.
- Type
str
- with_prop(property_name, value)
Set the model deployment details property_name attribute to value
- with_instance_configuration(config)
Set the configuration of VM instance.
- with_access_log(log_group_id, log_id)
Config the access log with OCI logging service
- with_predict_log(log_group_id, log_id)
Config the predict log with OCI logging service
- build()
Return an instance of CreateModelDeploymentDetails for creating the deployment.
Initialize a ModelDeploymentProperties object by specifying one of the followings:
- Parameters
model_id ((str, optiona). Defaults to None.) – Model Artifact OCID. The model_id must be specified either explicitly or as an attribute of the OCI object.
model_uri ((str, optiona). Defaults to None.) – Uri to model files, can be local or in cloud storage.
oci_model_deployment ((Union[ModelDeployment, CreateModelDeploymentDetails, UpdateModelDeploymentDetails, Dict], optional). Defaults to None.) – An OCI model or Dict containing model deployment details. The OCI model can be an instance of either ModelDeployment, CreateModelDeploymentDetails or UpdateModelConfigurationDetails.
config ((Dict, optional). Defaults to None.) – ADS auth dictionary for OCI authentication. This can be generated by calling ads.common.auth.api_keys() or ads.common.auth.resource_principal(). If this is None, ads.common.default_signer(client_kwargs) will be used.
kwargs –
Users can also initialize the object by using keyword arguments. The following keyword arguments are supported by oci.data_science.models.data_science_models.ModelDeployment:
display_name,
description,
project_id,
compartment_id,
model_deployment_configuration_details,
category_log_details,
freeform_tags,
defined_tags.
If display_name is not specified, a randomly generated easy to remember name will be generated, like ‘strange-spider-2022-08-17-23:55.02’.
ModelDeploymentProperties also supports the following additional keyward arguments:
instance_shape,
instance_count,
bandwidth_mbps,
access_log_group_id,
access_log_id,
predict_log_group_id,
predict_log_id,
memory_in_gbs,
ocpus.
These additional arguments will be saved into appropriate properties in the OCI model.
- Raises
ValueError – model_id is None AND not specified in oci_model_deployment.model_deployment_configuration_details.model_configuration_details.
- build() CreateModelDeploymentDetails
Converts the deployment properties to OCI CreateModelDeploymentDetails object. Converts a model URI into a model OCID if user passed in a URI.
- Returns
A CreateModelDeploymentDetails instance ready for OCI API.
- Return type
CreateModelDeploymentDetails
- sub_properties = ['instance_shape', 'instance_count', 'bandwidth_mbps', 'access_log_group_id', 'access_log_id', 'predict_log_group_id', 'predict_log_id', 'memory_in_gbs', 'ocpus']
- to_oci_model(oci_model)
Convert properties into an OCI data model
- Parameters
oci_model (class) – The class of OCI data model, e.g., oci.data_science_models.CreateModelDeploymentDetails
- to_update_deployment() UpdateModelDeploymentDetails
Converts the deployment properties to OCI UpdateModelDeploymentDetails object.
- Returns
An UpdateModelDeploymentDetails instance ready for OCI API.
- Return type
CreateModelDeploymentDetails
- with_access_log(log_group_id: str, log_id: str)
Adds access log config
- Parameters
group_id (str) – Log group ID of OCI logging service
log_id (str) – Log ID of OCI logging service
- Returns
self
- Return type
- with_category_log(log_type: str, group_id: str, log_id: str)
Adds category log configuration
- Parameters
log_type (str) – The type of logging to be configured. Must be “access” or “predict”
group_id (str) – Log group ID of OCI logging service
log_id (str) – Log ID of OCI logging service
- Returns
self
- Return type
- Raises
ValueError – When log_type is invalid
- with_instance_configuration(config)
with_instance_configuration creates a ModelDeploymentDetails object with a specific config
- Parameters
config (dict) –
dictionary containing instance configuration about the deployment. The following keys are supported:
instance_shape: str,
instance_count: int,
bandwidth_mbps: int,
memory_in_gbs: float,
ocpus: float
The instance_shape and instance_count are required when creating a new deployment. They are optional when updating an existing deployment.
- Returns
self
- Return type
- with_logging_configuration(access_log_group_id: str, access_log_id: str, predict_log_group_id: Optional[str] = None, predict_log_id: Optional[str] = None)
Adds OCI logging configurations for OCI logging service
- Parameters
access_log_group_id (str) – Log group ID of OCI logging service for access log
access_log_id (str) – Log ID of OCI logging service for access log
predict_log_group_id (str) – Log group ID of OCI logging service for predict log
predict_log_id (str) – Log ID of OCI logging service for predict log
- Returns
self
- Return type
- with_predict_log(log_group_id: str, log_id: str)
Adds predict log config
- Parameters
group_id (str) – Log group ID of OCI logging service
log_id (str) – Log ID of OCI logging service
- Returns
self
- Return type
- with_prop(property_name: str, value: Any)
Sets model deployment’s property_name attribute to value
- Parameters
property_name (str) – Name of a model deployment property.
value – New value for property attribute.
- Returns
self
- Return type