ads.catalog package

Submodules

ads.catalog.model module

class ads.catalog.model.Model(model: Model, model_etag: str, provenance_metadata: ModelProvenance, provenance_etag: str, ds_client: DataScienceClient, identity_client: IdentityClient)

Bases: object

Class that represents the ADS implementation of model catalog item. Converts the metadata and schema from OCI implememtation to ADS implementation.

to_dataframe()

Converts model to dataframe format.

show_in_notebook()

Shows model in the notebook in dataframe or YAML representation.

activate()

Activates model.

deactivate()

Deactivates model.

commit()

Commits the changes made to the model.

rollback()

Rollbacks the changes made to the model.

load_model()

Loads the model from the model catalog based on model ID.

Initializes the Model.

Parameters
  • model (OCIModel) – The OCI model object.

  • model_etag (str) – The model ETag.

  • provenance_metadata (ModelProvenance) – The model provenance metadata.

  • provenance_etag (str) – The model provenance metadata ETag.

  • ds_client (DataScienceClient) – The Oracle DataScience client.

  • identity_client (IdentityClient) – The Orcale Identity Service Client.

activate() None

Activates model.

Returns

Nothing.

Return type

None

commit(force: bool = True) None

Commits model changes.

Parameters

force (bool) – If True, any remote changes on this model would be lost.

Returns

Nothing.

Return type

None

deactivate() None

Deactivates model.

Returns

Nothing.

Return type

None

classmethod load_model(ds_client: DataScienceClient, identity_client: IdentityClient, model_id: str) Model

Loads the model from the model catalog based on model ID.

Parameters
  • ds_client (DataScienceClient) – The Oracle DataScience client.

  • identity_client (IdentityClient) – The Orcale Identity Service Client.

  • model_id (str) – The model ID.

Returns

The ADS model catalog item.

Return type

Model

Raises
  • ServiceError – If error occures while getting model from server.:

  • KeyError – If model not found.:

  • ValueError – If error occures while getting model provenance mettadata from server.:

rollback() None

Rollbacks the changes made to the model.

Returns

Nothing.

Return type

None

show_in_notebook(display_format: str = 'dataframe') None

Shows model in dataframe or yaml format. Supported formats: dataframe and yaml. Defaults to dataframe format.

Returns

Nothing.

Return type

None

to_dataframe() DataFrame

Converts the model to dataframe format.

Returns

Pandas dataframe.

Return type

panadas.DataFrame

exception ads.catalog.model.ModelArtifactSizeError(max_artifact_size: str)

Bases: Exception

class ads.catalog.model.ModelCatalog(compartment_id: Optional[str] = None, ds_client_auth: Optional[dict] = None, identity_client_auth: Optional[dict] = None, timeout: Optional[int] = None, ds_client: Optional[DataScienceClient] = None, identity_client: Optional[IdentityClient] = None)

Bases: object

Allows to list, load, update, download, upload and delete models from model catalog.

get_model(self, model_id)

Loads the model from the model catalog based on model_id.

list_models(self, project_id=None, include_deleted=False, datetime_format=utils.date_format, \*\*kwargs)

Lists all models in a given compartment, or in the current project if project_id is specified.

list_model_deployment(self, model_id, config=None, tenant_id=None, limit=500, page=None, \*\*kwargs)

Gets the list of model deployments by model Id across the compartments.

update_model(self, model_id, update_model_details=None, \*\*kwargs)

Updates a model with given model_id, using the provided update data.

delete_model(self, model, \*\*kwargs)

Deletes the model based on model_id.

download_model(self, model_id, target_dir, force_overwrite=False, install_libs=False, conflict_strategy=ConflictStrategy.IGNORE)

Downloads the model from model_dir to target_dir based on model_id.

upload_model(self, model_artifact, provenance_metadata=None, project_id=None, display_name=None, description=None)

Uploads the model artifact to cloud storage.

Initializes model catalog instance.

Parameters
  • compartment_id ((str, optional). Defaults to None.) – Model compartment OCID. If None, the config.NB_SESSION_COMPARTMENT_OCID would be used.

  • ds_client_auth ((dict, optional). Defaults to None.) – The default authetication is set using ads.set_auth API. If you need to override the default, use the ads.common.auth.api_keys or ads.common.auth.resource_principal to create appropriate authentication signer and kwargs required to instantiate DataScienceClient object.

  • identity_client_auth ((dict, optional). Defaults to None.) – The default authetication is set using ads.set_auth API. If you need to override the default, use the ads.common.auth.api_keys or ads.common.auth.resource_principal to create appropriate authentication signer and kwargs required to instantiate IdentityClient object.

  • timeout ((int, optional). Defaults to 10 seconds.) – The connection timeout in seconds for the client.

  • ds_client (DataScienceClient) – The Oracle DataScience client.

  • identity_client (IdentityClient) – The Orcale Identity Service Client.

Raises
  • ValueError – If compartment ID not specified.

  • TypeError – If timeout not an integer.

delete_model(model: Union[str, ads.catalog.Model], **kwargs) bool

Deletes the model from Model Catalog.

Parameters
  • model (Union[str, "ads.catalog.Model"]) – The OCID of the model to delete as a string, or a ads.catalog.Model instance.

  • kwargs

    delete_associated_model_deployment: (bool, optional). Defaults to False.

    Whether associated model deployments need to be deletet or not.

Returns

True if the model was successfully deleted.

Return type

bool

Raises

ModelWithActiveDeploymentError – If model has active model deployments ant inout attribute delete_associated_model_deployment set to False.

download_model(model_id: str, target_dir: str, force_overwrite: bool = False, install_libs: bool = False, conflict_strategy='IGNORE', bucket_uri: Optional[str] = None, remove_existing_artifact: Optional[bool] = True)

Downloads the model from model_dir to target_dir based on model_id.

Parameters
  • model_id (str) – The OCID of the model to download.

  • target_dir (str) – The target location of model after download.

  • force_overwrite (bool) – Overwrite target_dir if exists.

  • install_libs (bool, default: False) – Install the libraries specified in ds-requirements.txt which are missing in the current environment.

  • conflict_strategy (ConflictStrategy, default: IGNORE) – Determines how to handle version conflicts between the current environment and requirements of model artifact. Valid values: “IGNORE”, “UPDATE” or ConflictStrategy. IGNORE: Use the installed version in case of conflict UPDATE: Force update dependency to the version required by model artifact in case of conflict

  • bucket_uri ((str, optional). Defaults to None.) – The OCI Object Storage URI where model artifacts will be copied to. The bucket_uri is only necessary for downloading large artifacts with size is greater than 2GB. Example: oci://<bucket_name>@<namespace>/prefix/.

  • remove_existing_artifact ((bool, optional). Defaults to True.) – Whether artifacts uploaded to object storage bucket need to be removed or not.

Returns

A ModelArtifact instance.

Return type

ModelArtifact

get_model(model_id)

Loads the model from the model catalog based on model_id.

Parameters

model_id (str, required) – The model ID.

Returns

The ads.catalog.Model with the matching ID.

Return type

ads.catalog.Model

list_model_deployment(model_id: str, config: Optional[dict] = None, tenant_id: Optional[str] = None, limit: int = 500, page: Optional[str] = None, **kwargs)

Gets the list of model deployments by model Id across the compartments.

Parameters
  • model_id (str) – The model ID.

  • config (dict (optional)) – Configuration keys and values as per SDK and Tool Configuration. The from_file() method can be used to load configuration from a file. Alternatively, a dict can be passed. You can validate_config the dict using validate_config(). Defaults to None.

  • tenant_id (str (optional)) – The tenancy ID, which can be used to specify a different tenancy (for cross-tenancy authorization) when searching for resources in a different tenancy. Defaults to None.

  • limit (int (optional)) – The maximum number of items to return. The value must be between 1 and 1000. Defaults to 500.

  • page (str (optional)) – The page at which to start retrieving results.

Return type

The list of model deployments.

list_models(project_id: Optional[str] = None, include_deleted: bool = False, datetime_format: str = '%Y-%m-%d %H:%M:%S', **kwargs)

Lists all models in a given compartment, or in the current project if project_id is specified.

Parameters
  • project_id (str) – The project_id of model.

  • include_deleted (bool, optional, default=False) – Whether to include deleted models in the returned list.

  • datetime_format (str, optional, default: '%Y-%m-%d %H:%M:%S') – Change format for date time fields.

Returns

A list of models.

Return type

ModelSummaryList

update_model(model_id, update_model_details=None, **kwargs) Model

Updates a model with given model_id, using the provided update data.

Parameters
  • model_id (str) – The model ID.

  • update_model_details (UpdateModelDetails) – Contains the update model details data to apply. Mandatory unless kwargs are supplied.

  • kwargs (dict, optional) – Update model details can be supplied instead as kwargs.

Returns

The ads.catalog.Model with the matching ID.

Return type

Model

upload_model(model_artifact: ModelArtifact, provenance_metadata: Optional[ModelProvenance] = None, project_id: Optional[str] = None, display_name: Optional[str] = None, description: Optional[str] = None, freeform_tags: Optional[Dict[str, Dict[str, object]]] = None, defined_tags: Optional[Dict[str, Dict[str, object]]] = None, bucket_uri: Optional[str] = None, remove_existing_artifact: Optional[bool] = True, overwrite_existing_artifact: Optional[bool] = True, model_version_set: Optional[Union[str, ModelVersionSet]] = None, version_label: Optional[str] = None)

Uploads the model artifact to cloud storage.

Parameters
  • model_artifact (Union[ModelArtifact, GenericModel]) – The model artifacts or generic model instance.

  • provenance_metadata ((ModelProvenance, optional). Defaults to None.) – Model provenance gives data scientists information about the origin of their model. This information allows data scientists to reproduce the development environment in which the model was trained.

  • project_id ((str, optional). Defaults to None.) – The project_id of model.

  • display_name ((str, optional). Defaults to None.) – The name of model. If a display_name is not provided, a randomly generated easy to remember name with timestamp will be generated, like ‘strange-spider-2022-08-17-23:55.02’.

  • description ((str, optional). Defaults to None.) – The description of model.

  • freeform_tags ((Dict[str, str], optional). Defaults to None.) – Freeform tags for the model, by default None

  • defined_tags ((Dict[str, dict[str, object]], optional). Defaults to None.) – Defined tags for the model, by default None.

  • bucket_uri ((str, optional). Defaults to None.) – The OCI Object Storage URI where model artifacts will be copied to. The bucket_uri is only necessary for uploading large artifacts which size greater than 2GB. Example: oci://<bucket_name>@<namespace>/prefix/.

  • remove_existing_artifact ((bool, optional). Defaults to True.) – Whether artifacts uploaded to object storage bucket need to be removed or not.

  • overwrite_existing_artifact ((bool, optional). Defaults to True.) – Overwrite target bucket artifact if exists.

  • model_version_set ((Union[str, ModelVersionSet], optional). Defaults to None.) – The Model version set OCID, or name, or ModelVersionSet instance.

  • version_label ((str, optional). Defaults to None.) – The model version label.

Returns

The ads.catalog.Model with the matching ID.

Return type

ads.catalog.Model

class ads.catalog.model.ModelSummaryList(model_catalog, model_list, response=None, datetime_format='%Y-%m-%d %H:%M:%S')

Bases: SummaryList

Model Summary List which represents a list of Model Object.

sort_by(self, columns, reverse=False)

Performs a multi-key sort on a particular set of columns and returns the sorted ModelSummaryList. Results are listed in a descending order by default.

filter(self, selection, instance=None)

Filters the model list according to a lambda filter function, or list comprehension.

filter(selection, instance=None)

Filters the model list according to a lambda filter function, or list comprehension.

Parameters
  • selection (lambda function filtering model instances, or a list-comprehension) – function of list filtering projects

  • instance (list, optional) – list to filter, optional, defaults to self

Returns

ModelSummaryList

Return type

A filtered ModelSummaryList

sort_by(columns, reverse=False)

Performs a multi-key sort on a particular set of columns and returns the sorted ModelSummaryList. Results are listed in a descending order by default.

Parameters
  • columns (List of string) – A list of columns which are provided to sort on

  • reverse (Boolean (defaults to false)) – If you’d like to reverse the results (for example, to get ascending instead of descending results)

Returns

ModelSummaryList

Return type

A sorted ModelSummaryList

exception ads.catalog.model.ModelWithActiveDeploymentError

Bases: Exception

ads.catalog.notebook module

class ads.catalog.notebook.NotebookCatalog(compartment_id=None)

Bases: object

create_notebook_session(display_name=None, project_id=None, shape=None, block_storage_size_in_gbs=None, subnet_id=None, **kwargs)

Create a new notebook session with the supplied details.

Parameters
  • display_name (str, required) – The value to assign to the display_name property of this CreateNotebookSessionDetails.

  • project_id (str, required) – The value to assign to the project_id property of this CreateNotebookSessionDetails.

  • shape (str, required) – The value to assign to the shape property of this NotebookSessionConfigurationDetails. Allowed values for this property are: “VM.Standard.E2.2”, “VM.Standard.E2.4”, “VM.Standard.E2.8”, “VM.Standard2.1”, “VM.Standard2.2”, “VM.Standard2.4”, “VM.Standard2.8”, “VM.Standard2.16”,”VM.Standard2.24”.

  • block_storage_size_in_gbs (int, required) – Size of the block storage drive. Limited to values between 50 (GB) and 1024 (1024GB = 1TB)

  • subnet_id (str, required) – The OCID of the subnet resource where the notebook is to be created.

  • kwargs (dict, optional) – Additional kwargs passed to DataScienceClient.create_notebook_session()

Returns

oci.data_science.models.NotebookSession

Return type

A new notebook record.

Raises

KeyError – If the resource was not found or do not have authorization to access that resource.:

delete_notebook_session(notebook, **kwargs)

Deletes the notebook based on notebook_id.

Parameters

notebook (str ID or oci.data_science.models.NotebookSession,required) – The OCID of the notebook to delete as a string, or a Notebook Session instance

Returns

Bool

Return type

True if delete was successful, false otherwise

get_notebook_session(notebook_id)

Get the notebook based on notebook_id

Parameters

notebook_id (str, required) – The OCID of the notebook to get.

Returns

oci.data_science.models.NotebookSession

Return type

The oci.data_science.models.NotebookSession with the matching ID.

Raises

KeyError – If the resource was not found or do not have authorization to access that resource.:

list_notebook_session(include_deleted=False, datetime_format='%Y-%m-%d %H:%M:%S', **kwargs)

List all notebooks in a given compartment

Parameters
  • include_deleted (bool, optional, default=False) – Whether to include deleted notebooks in the returned list

  • datetime_format (str, optional, default: '%Y-%m-%d %H:%M:%S') – Change format for date time fields

Returns

NotebookSummaryList

Return type

A List of notebooks.

Raises

KeyError – If the resource was not found or do not have authorization to access that resource.:

update_notebook_session(notebook_id, update_notebook_details=None, **kwargs)

Updates a notebook with given notebook_id, using the provided update data

Parameters
  • notebook_id (str) – notebook_id OCID to update

  • update_notebook_details (oci.data_science.models.UpdateNotebookSessionDetails) – contains the new notebook details data to apply

  • kwargs (dict, optional) – Update notebook session details can be supplied instead as kwargs

Returns

oci.data_science.models.NotebookSession

Return type

The updated Notebook record

Raises

KeyError – If the resource was not found or do not have authorization to access that resource.:

class ads.catalog.notebook.NotebookSummaryList(notebook_list, response=None, datetime_format='%Y-%m-%d %H:%M:%S')

Bases: SummaryList

filter(selection, instance=None)

Filter the notebook list according to a lambda filter function, or list comprehension.

Parameters
  • selection (lambda function filtering notebook instances, or a list-comprehension) – function of list filtering notebooks

  • instance (list, optional) – list to filter, optional, defaults to self

Raises

ValueError – If selection passed is not correct. For example: selection=oci.data_science.models.NotebookSession.:

sort_by(columns, reverse=False)

Performs a multi-key sort on a particular set of columns and returns the sorted NotebookSummaryList Results are listed in a descending order by default.

Parameters
  • columns (List of string) – A list of columns which are provided to sort on

  • reverse (Boolean (defaults to false)) – If you’d like to reverse the results (for example, to get ascending instead of descending results)

Returns

NotebookSummaryList

Return type

A sorted NotebookSummaryList

ads.catalog.project module

class ads.catalog.project.ProjectCatalog(compartment_id=None, ds_client_auth=None, identity_client_auth=None)

Bases: Mapping

create_project(create_project_details=None, **kwargs)

Create a new project with the supplied details. create_project_details contains parameters needed to create a new project, according to oci.data_science.models.CreateProjectDetails.

Parameters
  • display_name (str) – The value to assign to the display_name property of this CreateProjectDetails.

  • description (str) – The value to assign to the description property of this CreateProjectDetails.

  • compartment_id (str) – The value to assign to the compartment_id property of this CreateProjectDetails.

  • freeform_tags (dict(str, str)) – The value to assign to the freeform_tags property of this CreateProjectDetails.

  • defined_tags (dict(str, dict(str, object))) – The value to assign to the defined_tags property of this CreateProjectDetails.

  • kwargs – New project details can be supplied instead as kwargs

Returns

oci.data_science.models.Project

Return type

A new Project record.

delete_project(project, **kwargs)

Deletes the project based on project_id.

Parameters

project (str ID or oci.data_science.models.Project,required) – The OCID of the project to delete as a string, or a Project instance

Returns

Bool

Return type

True if delete was succesful

get_project(project_id)

Get the Project based on project_id

Parameters

project_id (str, required) – The OCID of the project to get.

Return type

The oci.data_science.models.Project with the matching ID.

Raises

KeyError – If the resource was not found or do not have authorization to access that resource.:

list_projects(include_deleted=False, datetime_format='%Y-%m-%d %H:%M:%S', **kwargs)

List all projects in a given compartment, or in the current notebook session’s compartment

Parameters
  • include_deleted (bool, optional, default=False) – Whether to include deleted projects in the returned list

  • datetime_format (str, optional, default: '%Y-%m-%d %H:%M:%S') – Change format for date time fields

Returns

ProjectSummaryList

Return type

List of Projects.

Raises

KeyError – If the resource was not found or do not have authorization to access that resource.:

update_project(project_id, update_project_details=None, **kwargs)

Updates a project with given project_id, using the provided update data update_project_details contains the update project details data to apply, according to oci.data_science.models.UpdateProjectDetails

Parameters
  • project_id (str) – project_id OCID to update

  • display_name (str) – The value to assign to the display_name property of this UpdateProjectDetails.

  • description (str) – The value to assign to the description property of this UpdateProjectDetails.

  • freeform_tags (dict(str, str)) – The value to assign to the freeform_tags property of this UpdateProjectDetails.

  • defined_tags (dict(str, dict(str, object))) – The value to assign to the defined_tags property of this UpdateProjectDetails.

  • kwargs (dict, optional) – Update project details can be supplied instead as kwargs

Returns

oci.data_science.models.Project

Return type

The updated Project record

class ads.catalog.project.ProjectSummaryList(project_list, response=None, datetime_format='%Y-%m-%d %H:%M:%S')

Bases: SummaryList

A class used to represent Project Summary List.

df

Summary information for a project.

Type

data frame

datetime_format

Format used to describe time.

Type

str

response

A response object with data of type list of ProjectSummaryList.

Type

oci.response.Response

short_id_index

Mapping of short id and its value.

Type

(dict of str: str)

sort_by(self, columns, reverse=False):

Sort ProjectSummaryList by columns.

filter(self, selection, instance=None):

Filter the project list according to a lambda filter function, or list comprehension.

filter(selection, instance=None)

Filter the project list according to a lambda filter function, or list comprehension.

Parameters
  • selection (lambda function filtering Project instances, or a list-comprehension) – function of list filtering projects

  • instance (list, optional) – list to filter, optional, defaults to self

Returns

ProjectSummaryList

Return type

A filtered ProjectSummaryList

Raises

ValueError – If selection passed is not correct.:

sort_by(columns, reverse=False)

Sort ProjectSummaryList by columns.

Performs a multi-key sort on a particular set of columns and returns the sorted ProjectSummaryList Results are listed in a descending order by default.

Parameters
  • columns (List of string) – A list of columns which are provided to sort on

  • reverse (Boolean (defaults to false)) – If you’d like to reverse the results (for example, to get ascending instead of descending results)

Returns

ProjectSummaryList

Return type

A sorted ProjectSummaryList

ads.catalog.summary module

class ads.catalog.summary.SummaryList(entity_list, datetime_format='%Y-%m-%d %H:%M:%S')

Bases: list

abstract filter(selection, instance=None)

Abstract method for filtering, implemented by the derived class

show_in_notebook(datetime_format=None)

Displays the model catalog summary in a Jupyter Notebook cell

Parameters

date_format (like utils.date_format. Defaults to none.) –

Return type

None

abstract sort_by(columns, reverse=False)

Abstract method for sorting, implemented by the derived class

to_dataframe(datetime_format=None)

Returns the model catalog summary as a pandas dataframe

Parameters

datatime_format (date_format) – A datetime format, like utils.date_format. Defaults to none.

Returns

Dataframe

Return type

The pandas DataFrame repersentation of the model catalog summary

Module contents