ads.opctl package

Subpackages

• ads.opctl.backend package
  • Submodules
  • ads.opctl.backend.ads_dataflow module
    • DataFlowBackend
      • DataFlowBackend.apply()
      • DataFlowBackend.cancel()
      • DataFlowBackend.delete()
      • DataFlowBackend.init()
      • DataFlowBackend.run()
      • DataFlowBackend.watch()
    • DataFlowOperatorBackend
      • DataFlowOperatorBackend.runtime_config
      • DataFlowOperatorBackend.operator_config
      • DataFlowOperatorBackend.operator_type
      • DataFlowOperatorBackend.operator_version
      • DataFlowOperatorBackend.job
      • DataFlowOperatorBackend.run()
    • DataFlowRuntimeFactory
  • ads.opctl.backend.ads_ml_job module
    • JobRuntimeFactory
    • MLJobBackend
      • MLJobBackend.apply()
      • MLJobBackend.cancel()
      • MLJobBackend.delete()
      • MLJobBackend.init()
      • MLJobBackend.run()
      • MLJobBackend.watch()
    • MLJobDistributedBackend
      • MLJobDistributedBackend.DIAGNOSTIC_COMMAND
      • MLJobDistributedBackend.generate_worker_name()
      • MLJobDistributedBackend.prepare_job_config()
      • MLJobDistributedBackend.run()
      • MLJobDistributedBackend.run_diagnostics()
    • MLJobOperatorBackend
      • MLJobOperatorBackend.runtime_config
      • MLJobOperatorBackend.operator_config
      • MLJobOperatorBackend.operator_type
      • MLJobOperatorBackend.operator_version
      • MLJobOperatorBackend.operator_info
      • MLJobOperatorBackend.job
      • MLJobOperatorBackend.run()
  • ads.opctl.backend.ads_ml_pipeline module
    • PipelineBackend
      • PipelineBackend.apply()
      • PipelineBackend.cancel()
      • PipelineBackend.delete()
      • PipelineBackend.init()
      • PipelineBackend.run()
      • PipelineBackend.watch()
  • ads.opctl.backend.base module
    • Backend
      • Backend.activate()
      • Backend.apply()
      • Backend.cancel()
      • Backend.deactivate()
      • Backend.delete()
      • Backend.init()
      • Backend.predict()
      • Backend.run()
      • Backend.run_diagnostics()
      • Backend.watch()
    • RuntimeFactory
      • RuntimeFactory.get_runtime()
    • UnsupportedRuntime
  • ads.opctl.backend.local module
    • CondaPackNotFound
    • LocalBackend
      • LocalBackend.init_vscode_container()
      • LocalBackend.run()
    • LocalBackendDistributed
      • LocalBackendDistributed.run()
    • LocalModelDeploymentBackend
      • LocalModelDeploymentBackend.predict()
    • LocalOperatorBackend
      • LocalOperatorBackend.runtime_config
      • LocalOperatorBackend.operator_config
      • LocalOperatorBackend.operator_type
      • LocalOperatorBackend.operator_info
      • LocalOperatorBackend.init()
      • LocalOperatorBackend.run()
    • LocalPipelineBackend
      • LocalPipelineBackend.DEFAULT_PARALLEL_CONTAINER_MAXIMUM
      • LocalPipelineBackend.DEFAULT_STATUS_POLL_INTERVAL_SECONDS
      • LocalPipelineBackend.LOG_PREFIX
      • LocalPipelineBackend.run()
  • Module contents
• ads.opctl.conda package
  • Submodules
  • ads.opctl.conda.cli module
  • ads.opctl.conda.cmds module
    • create()
    • install()
    • publish()
  • ads.opctl.conda.multipart_uploader module
    • MultiPartUploader
      • MultiPartUploader.upload()
  • ads.opctl.conda.pack module
  • Module contents
• ads.opctl.config package
  • Subpackages
    • ads.opctl.config.diagnostics package
      • Module contents
    • ads.opctl.config.yaml_parsers package
      • Subpackages
      • Submodules
      • ads.opctl.config.yaml_parsers.base module
      • Module contents
  • Submodules
  • ads.opctl.config.base module
    • ConfigProcessor
      • ConfigProcessor.process()
      • ConfigProcessor.step()
  • ads.opctl.config.merger module
    • ConfigMerger
      • ConfigMerger.process()
  • ads.opctl.config.resolver module
    • ConfigResolver
      • ConfigResolver.process()
  • ads.opctl.config.utils module
    • CondaPackInfoNotProvided
    • NotSupportedError
    • OperatorNotFound
    • convert_notebook()
    • read_from_ini()
  • ads.opctl.config.validator module
    • ConfigValidator
      • ConfigValidator.process()
  • ads.opctl.config.versioner module
    • ConfigVersioner
      • ConfigVersioner.process()
  • Module contents
• ads.opctl.diagnostics package
  • Submodules
  • ads.opctl.diagnostics.check_distributed_job_requirements module
  • ads.opctl.diagnostics.check_requirements module
  • ads.opctl.diagnostics.requirement_exception module
    • RequirementException
  • Module contents
• ads.opctl.distributed package
  • Subpackages
    • ads.opctl.distributed.common package
      • Submodules
      • ads.opctl.distributed.common.abstract_cluster_provider module
      • ads.opctl.distributed.common.abstract_framework_spec_builder module
      • ads.opctl.distributed.common.cluster_config_helper module
      • ads.opctl.distributed.common.cluster_provider_factory module
      • ads.opctl.distributed.common.cluster_runner module
      • ads.opctl.distributed.common.framework_factory module
      • Module contents
  • Submodules
  • ads.opctl.distributed.certificates module
    • download_certificates()
  • ads.opctl.distributed.cli module
  • ads.opctl.distributed.cmds module
    • cancel_distributed_run()
    • dask_cmd()
    • dev_mode_base_uri()
    • docker_build_cmd()
    • get_cmd()
    • horovod_cmd()
    • increment_tag()
    • increment_tag_in_ini()
    • initialize_workspace()
    • load_ini()
    • local_run()
    • production_mode_base_uri()
    • pytorch_cmd()
    • refresh_ini()
    • run_cmd()
    • show_config_info()
    • tensorflow_cmd()
    • update_config_image()
    • update_image()
    • update_ini()
    • verify_and_publish_image()
    • verify_image()
  • Module contents
• ads.opctl.spark package
  • Submodules
  • ads.opctl.spark.cli module
  • ads.opctl.spark.cmds module
    • core_site()
    • generate_core_site_properties()
    • generate_core_site_properties_str()
  • Module contents

Submodules

ads.opctl.cli module

ads.opctl.cli.add_options(options)

ads.opctl.cmds module

class ads.opctl.cmds.DataScienceResource

Bases: str

DATAFLOW = 'dataflowapplication'

JOB = 'datasciencejob'

MODEL = 'datasciencemodel'

MODEL_DEPLOYMENT = 'datasciencemodeldeployment'

PIPELINE = 'datasciencepipeline'

class ads.opctl.cmds.DataScienceResourceRun

Bases: str

DATAFLOW_RUN = 'dataflowrun'

JOB_RUN = 'datasciencejobrun'

MODEL_DEPLOYMENT = 'datasciencemodeldeployment'

PIPELINE_RUN = 'datasciencepipelinerun'

ads.opctl.cmds.activate(**kwargs) → None

Activate a ModelDeployment.

Parameters:

kwargs (dict) – keyword argument, stores command line args

Return type:

None

ads.opctl.cmds.cancel(**kwargs) → None

Cancel a MLJob/DataFlow run.

Parameters:

kwargs (dict) – keyword argument, stores command line args

Return type:

None

ads.opctl.cmds.configure() → None

Save default configurations for opctl.

Parameters:

ml_job (bool) – turn on/off saving configurations for ML Job

Return type:

None

ads.opctl.cmds.deactivate(**kwargs) → None

Deactivate a ModelDeployment.

Parameters:

kwargs (dict) – keyword argument, stores command line args

Return type:

None

ads.opctl.cmds.delete(**kwargs) → None

Delete a MLJob/DataFlow run.

Parameters:

kwargs (dict) – keyword argument, stores command line args

Return type:

None

ads.opctl.cmds.init(resource_type: str, runtime_type: str | None = None, output: str | None = None, overwrite: bool = False, ads_config: str = '~/.ads_ops', **kwargs) → str | None

Generates a starter specification template YAML for the Data Science resource.

Parameters:

• resource_type (str) – The resource type to generate the specification YAML.

• runtime_type ((str, optional). Defaults to None.) – The resource runtime type.

• output ((str, optional). Defaults to None.) – The path to the file to save the resulting specification template YAML.

• overwrite ((bool, optional). Defaults to False.) – Whether to overwrite the result specification YAML if exists.

• ads_config ((str, optional)) – The folder where the ads opctl config located.

• kwargs ((Dict, optional).) – Any optional kwargs arguments.

Returns:

The YAML specification for the given resource if output was not provided, path to the specification otherwise.

Return type:

Union[str, None]

Raises:

ValueError – If resource_type not specified.

ads.opctl.cmds.init_vscode(**kwargs) → None

Create a .devcontainer.json file for local development.

Parameters:

kwargs – keyword arguments, stores command line args

Return type:

None

ads.opctl.cmds.predict(**kwargs) → None

Make prediction using the model with the payload.

Parameters:

kwargs (dict) – keyword argument, stores command line args

Return type:

None

ads.opctl.cmds.run(config: Dict, **kwargs) → Dict

Run a job given configuration and command line args passed in (kwargs).

Parameters:

• config (dict) – dictionary of configurations

• kwargs (dict) – keyword arguments, stores configuration from command line args

Returns:

dictionary of job id and run id in case of ML Job run, else empty if running locally

Return type:

Dict

ads.opctl.cmds.run_diagnostics(config: Dict, **kwargs) → Dict

Run a job given configuration and command line args passed in (kwargs).

Parameters:

• config (dict) – dictionary of configurations

• kwargs (dict) – keyword arguments, stores configuration from command line args

Returns:

dictionary of job id and run id in case of ML Job run, else empty if running locally

Return type:

Dict

ads.opctl.cmds.watch(**kwargs) → None

Watch a MLJob/DataFlow run.

Parameters:

kwargs (dict) – keyword argument, stores command line args

Return type:

None

ads.opctl.constants module

class ads.opctl.constants.BACKEND_NAME(value)

Bases: ExtendedEnum

An enumeration.

DATAFLOW = 'dataflow'

JOB = 'job'

LOCAL = 'local'

MARKETPLACE = 'marketplace'

MODEL_DEPLOYMENT = 'deployment'

OPERATOR_LOCAL = 'operator.local'

PIPELINE = 'pipeline'

class ads.opctl.constants.RESOURCE_TYPE(value)

Bases: ExtendedEnum

An enumeration.

DATAFLOW = 'dataflow'

JOB = 'job'

MARKETPLACE = 'marketplace'

MODEL_DEPLOYMENT = 'deployment'

OPERATOR = 'operator'

PIPELINE = 'pipeline'

class ads.opctl.constants.RUNTIME_TYPE(value)

Bases: ExtendedEnum

An enumeration.

CONDA = 'conda'

CONTAINER = 'container'

DATAFLOW = 'dataFlow'

DATAFLOWNOTEBOOK = 'dataFlowNotebook'

GITPYTHON = 'gitPython'

NOTEBOOK = 'notebook'

OPERATOR = 'operator'

PYTHON = 'python'

SCRIPT = 'script'

ads.opctl.utils module

ads.opctl.utils.build_image(image_type: str, gpu: bool = False) → None

Build an image for opctl.

Parameters:

• image_type (str) – specify the image to build, can take ‘job-local’, former for running job with conda pack locally, latter for running operators

• gpu (bool) – whether to use gpu version of image

• dst_image (str) – image to save as when building custom operator

Return type:

None

ads.opctl.utils.get_docker_client() → docker.client.DockerClient

ads.opctl.utils.get_namespace(auth: dict) → str

ads.opctl.utils.get_oci_region(auth: dict) → str

ads.opctl.utils.get_region_key(auth: dict) → str

ads.opctl.utils.get_service_pack_prefix() → Dict

ads.opctl.utils.is_in_notebook_session()

ads.opctl.utils.list_ads_operators() → dict

ads.opctl.utils.parse_conda_uri(uri: str) → Tuple[str, str, str, str]

ads.opctl.utils.publish_image(image: str, registry: str | None = None) → None

Publish an image.

Parameters:

• image (str) – image name

• registry (str) – registry to push to

Return type:

None

ads.opctl.utils.run_command(cmd: str | List[str], cwd: str | None = None, shell: bool = False) → Popen

ads.opctl.utils.run_container(image: str, bind_volumes: Dict, env_vars: Dict, command: str = None, entrypoint: str = None, verbose: bool = False) → int

ads.opctl.utils.suppress_traceback(debug: bool = True) → None

Decorator to suppress traceback when in debug mode.

Parameters:

debug (bool) – turn on debug mode or not

Return type:

None

Module contents

ads.opctl.set_log_level(level)