ads.opctl.config package

Subpackages

• ads.opctl.config.diagnostics package
  • Module contents
• ads.opctl.config.yaml_parsers package
  • Subpackages
    • ads.opctl.config.yaml_parsers.distributed package
      • Submodules
      • ads.opctl.config.yaml_parsers.distributed.yaml_parser module
      • Module contents
  • Submodules
  • ads.opctl.config.yaml_parsers.base module
    • YamlSpecParser
      • YamlSpecParser.parse()
      • YamlSpecParser.parse_content()
      • YamlSpecParser.translate_config()
  • Module contents

Submodules

ads.opctl.config.base module

class ads.opctl.config.base.ConfigProcessor(config: Dict | None = None)

Bases: object

Initializing a ConfigProcessor object given a configuration.

Parameters:

config (dict) – a dictionary of configurations

abstract process(**kwargs) → ConfigProcessor

Perform some processing on configuration associated with this object.

Parameters:

kwargs (dict) – keyword arguments passed to the function

Returns:

this instance itself

Return type:

ConfigProcessor

step(cls, **kwargs) → ConfigProcessor

Perform some processing according to the cls given.

Parameters:

• cls (ConfigProcessor) – a subclass of ConfigProcessor

• kwargs (dict) – keyword arguments passed to the process function of cls

Returns:

this instance itself

Return type:

ConfigProcessor

ads.opctl.config.merger module

class ads.opctl.config.merger.ConfigMerger(config: Dict | None = None)

Bases: ConfigProcessor

Merge configurations from command line args, YAML, ini and default configs. The order of precedence is command line args > YAML + conf.ini > .ads_ops/config.ini + .ads_ops/ml_job_config.ini or .ads_ops/dataflow_config.ini Detailed examples can be found at the last section on this page: https://bitbucket.oci.oraclecorp.com/projects/ODSC/repos/advanced-ds/browse/ads/opctl/README.md?at=refs%2Fheads%2Fads-ops-draft

Initializing a ConfigProcessor object given a configuration.

Parameters:

config (dict) – a dictionary of configurations

process(**kwargs) → None

Perform some processing on configuration associated with this object.

Parameters:

kwargs (dict) – keyword arguments passed to the function

Returns:

this instance itself

Return type:

ConfigProcessor

ads.opctl.config.resolver module

class ads.opctl.config.resolver.ConfigResolver(config: Dict | None = None)

Bases: ConfigProcessor

Infer and fill in fields necessary for running an operator based on user inputs. For instance, if user pass in an ADS operator name, we will fill in image name, command, volumes to mount for local run, source folder path, etc.

Please check functions with name _resolve_<thing>.

List of things being resolved (and brief description): - operator name (could be passed in directly or in YAML) - conda (type is determined on whether a slug or uri is passed in. slug -> service, uri -> published. also do some additional checks.) - source folder (mainly for ADS operator) - command (docker command) - image name (mainly for ADS operator) - env vars - mount volumes (for local runs)

There are additional comments in some of the functions below.

Initializing a ConfigProcessor object given a configuration.

Parameters:

config (dict) – a dictionary of configurations

process()

Perform some processing on configuration associated with this object.

Parameters:

kwargs (dict) – keyword arguments passed to the function

Returns:

this instance itself

Return type:

ConfigProcessor

ads.opctl.config.utils module

exception ads.opctl.config.utils.CondaPackInfoNotProvided

Bases: Exception

exception ads.opctl.config.utils.NotSupportedError

Bases: Exception

exception ads.opctl.config.utils.OperatorNotFound

Bases: Exception

ads.opctl.config.utils.convert_notebook(input_path, auth, exclude_tags: List = None, output_path: str = None, overwrite: bool = False) → str

ads.opctl.config.utils.read_from_ini(path: str) → ConfigParser

ads.opctl.config.validator module

class ads.opctl.config.validator.ConfigValidator(config: Dict | None = None)

Bases: ConfigProcessor

Initializing a ConfigProcessor object given a configuration.

Parameters:

config (dict) – a dictionary of configurations

process()

Perform some processing on configuration associated with this object.

Parameters:

kwargs (dict) – keyword arguments passed to the function

Returns:

this instance itself

Return type:

ConfigProcessor

ads.opctl.config.versioner module

class ads.opctl.config.versioner.ConfigVersioner(config: Dict | None = None)

Bases: ConfigProcessor

ads opctl supports multiple yaml file types, kinds and versions. Each has its own slightly different yaml structure. This goal of this class is to provide a translation from the generic term for a variable

to it’s specific path in it’s yaml.

Initializing a ConfigProcessor object given a configuration.

Parameters:

config (dict) – a dictionary of configurations

process()

Perform some processing on configuration associated with this object.

Parameters:

kwargs (dict) – keyword arguments passed to the function

Returns:

this instance itself

Return type:

ConfigProcessor

Module contents