ads.jobs.builders.runtimes package¶

Submodules¶

ads.jobs.builders.runtimes.artifact module¶

class ads.jobs.builders.runtimes.artifact.Artifact(source, runtime=None)[source]¶

Bases: object

Represents a OCI Data Science Job artifact. The Artifact class is designed to add an additional processing step on runtime/source code. before uploading it as data science job artifact.

A sub-class should implement the build() method to do the additional processing. A sub-class is designed to be used with context manager so that the temporary files are cleaned up properly.

For example, the NotebookArtifact implements the build() method to convert the notebook to python script. with NotebookArtifact(runtime) as artifact:

The build() method will be called when entering the context manager
The final artifact for the job will be stored in artifact.path upload_artifact(artifact.path)
Files are cleaned up when exit or if there is an exception.

CONST_DRIVER_NOTEBOOK = 'driver_notebook.py'¶

CONST_DRIVER_UTILS = 'driver_utils.py'¶

build()[source]¶

Builds the runtime artifact in the temporary directory. Subclass should implement this method to: 1. Process the runtime 2. Set the self.path to the final artifact path

Raises:: NotImplementedError – When this method is not implemented in the subclass.

static copy_from_uri(uri, to_path, unpack=False, **storage_options)[source]¶

Copy file(s) to local path

Parameters:

uri (str) – The URI of the source file or directory, which can be local path of OCI object storage URI.
to_path (path-like object) – The local destination path. If this is a directory, the source file/directory will be placed under it.
unpack (bool) – Indicate if zip or tar.gz file specified by the uri should be unpacked. This option has no effect on other files.
storage_options – Storage options for fsspec. For OCI object storage, the default_signer from ads.common.auth will be used if storage option is not specified.

Returns:

The actual path of file/directory at destination.

For copying a single file and to_path is a filename, this will be the same as to_path.
For copying a single file and to_path is a directory, this will be to_path + filename.
For copying a directory, this will be to_path + directory name.

Return type:

str or path-like object

class ads.jobs.builders.runtimes.artifact.GitPythonArtifact[source]¶

Bases: Artifact

CONST_DRIVER_SCRIPT = 'driver_oci.py'¶

build()[source]¶: Prepares job artifact for GitPythonRuntime.

class ads.jobs.builders.runtimes.artifact.NotebookArtifact(source, runtime=None)[source]¶

Bases: PythonArtifact

Represents a NotebookRuntime job artifact

CONST_DRIVER_SCRIPT = 'driver_notebook.py'¶

build()[source]¶: Prepares job artifact for notebook runtime

class ads.jobs.builders.runtimes.artifact.PythonArtifact(source, runtime=None)[source]¶

Bases: Artifact

Represents a PythonRuntime job artifact

CONST_DRIVER_SCRIPT = 'driver_python.py'¶

DEFAULT_BASENAME = 'artifact'¶

USER_CODE_DIR = 'code'¶

build()[source]¶: Prepares job artifact for PythonRuntime.

class ads.jobs.builders.runtimes.artifact.ScriptArtifact(source, runtime=None)[source]¶

Bases: Artifact

Represents a ScriptRuntime job artifact

build()[source]¶: Prepares job artifact for script runtime. If the source is a file, it will be returned as is. If the source is a directory, it will be compressed as a zip file.

ads.jobs.builders.runtimes.base module¶

class ads.jobs.builders.runtimes.base.MultiNodeRuntime(spec: Dict | None = None, **kwargs)[source]¶

Bases: Runtime

Represents runtime supporting multi-node jobs.

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

CONST_REPLICA = 'replicas'¶

property replica: int¶: The number of nodes (job runs).

run(dsc_job, **kwargs)[source]¶: Starts the job runs

with_replica(count: int)[source]¶

Specifies the number of nodes (job runs) for the job.

Parameters:: count (int) – Number of nodes (job runs)
Returns:: The runtime instance.
Return type:: self

class ads.jobs.builders.runtimes.base.Runtime(spec: Dict | None = None, **kwargs)[source]¶

Bases: Builder

Base class for job runtime

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

CONST_ARGS = 'args'¶

CONST_DEFINED_TAGS = 'definedTags'¶

CONST_ENV_VAR = 'env'¶

CONST_FREEFORM_TAGS = 'freeformTags'¶

CONST_MAXIMUM_RUNTIME_IN_MINUTES = 'maximumRuntimeInMinutes'¶

property args: list¶: Command line arguments

attribute_map = {'args': 'args', 'definedTags': 'defined_tags', 'env': 'env', 'freeformTags': 'freeform_tags'}¶

property defined_tags: dict¶: Defined tags

property environment_variables: dict¶

Environment variables

Returns:: The runtime environment variables. The returned dictionary is a copy.
Return type:: dict

property envs: dict¶: Environment variables

property freeform_tags: dict¶: Freeform tags

init(**kwargs) → Self[source]¶

Initializes a starter specification for the runtime.

Returns:: This method returns self to support chaining methods.
Return type:: Self

property kind: str¶: Kind of the object to be stored in YAML. All runtime implementations will have “runtime” as kind. Subclass will have different types.

property maximum_runtime_in_minutes: int¶: Maximum runtime in minutes

property type: str¶: The type of the object as showing in YAML

with_argument(*args, **kwargs) → Self[source]¶

Adds command line arguments to the runtime.

This method can be called (chained) multiple times to add various arguments.

Parameters:

args – Positional arguments. In a single method call, positional arguments are always added before keyword arguments. You can call with_argument() to add positional arguments after keyword arguments.
kwargs – Keyword arguments. To add a keyword argument without value, set the value to None.

Returns:

This method returns self to support chaining methods.

Return type:

Self

Raises:

ValueError – Keyword arguments with space in a key.

Examples

>>> runtime = Runtime().with_argument(key1="val1", key2="val2").with_argument("pos1")
>>> print(runtime.args)
["--key1", "val1", "--key2", "val2", "pos1"]

>>> runtime = Runtime()
>>> runtime.with_argument("pos1")
>>> runtime.with_argument(key1="val1", key2="val2.1 val2.2")
>>> runtime.with_argument("pos2")
>>> print(runtime.args)
['pos1', '--key1', 'val1', '--key2', 'val2.1 val2.2', 'pos2']

>>> runtime = Runtime()
>>> runtime.with_argument("pos1")
>>> runtime.with_argument(key1=None, key2="val2")
>>> runtime.with_argument("pos2")
>>> print(runtime.args)
["pos1", "--key1", "--key2", "val2", "pos2"]

with_defined_tag(**kwargs) → Self[source]¶

Sets defined tags

Returns:: This method returns self to support chaining methods.
Return type:: Self

with_environment_variable(**kwargs) → Self[source]¶

Sets environment variables

Environment variables enclosed by ${...} will be substituted.

You can use $$ to escape the substitution.
Undefined variable enclosed by ${} will be ignored.
Double dollar signs $$ will be substituted by a single one $.

Returns:: This method returns self to support chaining methods.
Return type:: Self

Examples

>>> runtime = (
...     PythonRuntime()
...     .with_environment_variable(
...         HOST="10.0.0.1",
...         PORT="443",
...         URL="http://${HOST}:${PORT}/path/",
...         ESCAPED_URL="http://$${HOST}:$${PORT}/path/",
...         MISSING_VAR="This is ${UNDEFINED}",
...         VAR_WITH_DOLLAR="$10",
...         DOUBLE_DOLLAR="$$10"
...     )
... )
>>> for k, v in runtime.environment_variables.items():
...     print(f"{k}: {v}")
HOST: 10.0.0.1
PORT: 443
URL: http://10.0.0.1:443/path/
ESCAPED_URL: http://${HOST}:${PORT}/path/
MISSING_VAR: This is ${UNDEFINED}
VAR_WITH_DOLLAR: $10
DOUBLE_DOLLAR: $10

with_freeform_tag(**kwargs) → Self[source]¶

Sets freeform tags

Returns:: This method returns self to support chaining methods.
Return type:: Self

with_maximum_runtime_in_minutes(maximum_runtime_in_minutes: int) → Self[source]¶

Sets maximum runtime in minutes

Returns:: This method returns self to support chaining methods.
Return type:: Self

ads.jobs.builders.runtimes.container_runtime module¶

class ads.jobs.builders.runtimes.container_runtime.ContainerRuntime(spec: Dict | None = None, **kwargs)[source]¶

Bases: MultiNodeRuntime

Represents a container job runtime

To define container runtime:

>>> ContainerRuntime()
>>> .with_image("iad.ocir.io/<your_tenancy>/<your_image>:<tag>")
>>> .with_cmd("sleep 5 && echo Hello World")
>>> .with_entrypoint(["/bin/sh", "-c"])
>>> .with_image_digest("<image_digest>")
>>> .with_image_signature_id("<image_signature_id>")
>>> .with_environment_variable(MY_ENV="MY_VALUE")

Alternatively, you can define the entrypoint, cmd, image_digest``and ``image_signature_id along with the image.

>>> ContainerRuntime()
>>> .with_image(
>>>     "iad.ocir.io/<your_tenancy>/<your_image>:<tag>",
>>>     entrypoint=["/bin/sh", "-c"],
>>>     cmd="sleep 5 && echo Hello World",
>>>     image_digest="<image_digest>",
>>>     image_signature_id="<image_signature_id>",
>>> )
>>> .with_environment_variable(MY_ENV="MY_VALUE")

The entrypoint and cmd can be either “exec form” or “shell form” (See references). The exec form is used when a list is passed in. The shell form is used when a space separated string is passed in.

When using the ContainerRuntime with OCI Data Science Job, the exec form is recommended. For most images, when the entrypoint is set to ["/bin/sh", "-c"], cmd can be a string as if you are running shell command.

References

https://docs.docker.com/engine/reference/builder/#entrypoint https://docs.docker.com/engine/reference/builder/#cmd

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

CONST_CMD = 'cmd'¶

CONST_ENTRYPOINT = 'entrypoint'¶

CONST_IMAGE = 'image'¶

CONST_IMAGE_DIGEST = 'imageDigest'¶

CONST_IMAGE_SIGNATURE_ID = 'imageSignatureId'¶

CONST_SCRIPT_PATH = 'scriptPathURI'¶

property artifact_uri: str¶: The URI of the source code

attribute_map = {'args': 'args', 'cmd': 'cmd', 'definedTags': 'defined_tags', 'entrypoint': 'entrypoint', 'env': 'env', 'freeformTags': 'freeform_tags', 'image': 'image', 'imageDigest': 'image_digest', 'imageSignatureId': 'image_signature_id'}¶

property cmd: str¶: Command of the container job

property entrypoint: str¶: Entrypoint of the container job

property image: str¶: The container image

property image_digest: str¶: The container image digest.

property image_signature_id: str¶: The container image signature id.

init(**kwargs) → ContainerRuntime[source]¶

Initializes a starter specification for the runtime.

Returns:: The runtime instance.
Return type:: ContainerRuntime

property job_env_type: str¶: The container type

with_artifact(uri: str)[source]¶

Specifies the artifact to be added to the container.

Parameters:: uri (str) – URI to the source code script, which can be any URI supported by fsspec, including http://, https:// and OCI object storage. For example: oci://your_bucket@your_namespace/path/to/script.py
Returns:: The runtime instance.
Return type:: self

with_cmd(cmd: str) → ContainerRuntime[source]¶

Specifies the command for the container job.

Parameters:: cmd (str) – Command for the container job
Returns:: The runtime instance.
Return type:: ContainerRuntime

with_entrypoint(entrypoint: str | list) → ContainerRuntime[source]¶

Specifies the entrypoint for the container job.

Parameters:: entrypoint (str or list) – Entrypoint for the container job
Returns:: The runtime instance.
Return type:: ContainerRuntime

Specify the image for the container job.

Parameters:

image (str) – The container image, e.g. iad.ocir.io/<your_tenancy>/<your_image>:<your_tag>
entrypoint (str or list, optional) – Entrypoint for the job, by default None (the entrypoint defined in the image will be used).
cmd (str, optional) – Command for the job, by default None.
image_digest (str, optional) – The image digest, by default None.
image_signature_id (str, optional) – The image signature id, by default None.

Returns:

The runtime instance.

Return type:

ContainerRuntime

with_image_digest(image_digest: str) → ContainerRuntime[source]¶

Sets the digest of custom image.

Parameters:: image_digest (str) – The image digest.
Returns:: The runtime instance.
Return type:: ContainerRuntime

with_image_signature_id(image_signature_id: str) → ContainerRuntime[source]¶

Sets the signature id of custom image.

Parameters:: image_signature_id (str) – The image signature id.
Returns:: The runtime instance.
Return type:: ContainerRuntime

ads.jobs.builders.runtimes.python_runtime module¶

class ads.jobs.builders.runtimes.python_runtime.CondaRuntime(spec: Dict | None = None, **kwargs)[source]¶

Bases: Runtime

Represents a job runtime with conda pack This is the base class for Runtime using conda environment. The CondaRuntime is not designed to be used directly when creating a job.

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

CONST_CONDA = 'conda'¶

CONST_CONDA_REGION = 'region'¶

CONST_CONDA_SLUG = 'slug'¶

CONST_CONDA_TYPE = 'type'¶

CONST_CONDA_TYPE_CUSTOM = 'published'¶

CONST_CONDA_TYPE_SERVICE = 'service'¶

CONST_CONDA_URI = 'uri'¶

attribute_map = {'args': 'args', 'conda': 'conda', 'definedTags': 'defined_tags', 'env': 'env', 'freeformTags': 'freeform_tags'}¶

property conda: dict¶

The conda environment specification.

For service conda environment, the specification contains:

type, the type of the conda environment. This is always service for service conda environment.
slug, the slug of the conda environment.

For custom conda environment, the specification contains:

type, the type of the conda environment. This is always published for custom conda environment.
uri, the uri of the conda environment, e.g. oci://bucket@namespace/prefix/to/conda
region, the region of the bucket in which the conda environment is stored. By default, ADS will determine the region based on the authenticated API key or resource principal. This is only needed if your conda environment is stored in a different region.

Returns:: A dictionary containing the conda environment specifications.
Return type:: dict

init(**kwargs) → CondaRuntime[source]¶

Initializes a starter specification for the runtime.

Parameters:

**kwargs (Dict) –

conda_slug: str
The conda environment slug. If it contains ‘/’, then the assumption that this is a custom conda environment.

Returns:

The runtime instance.

Return type:

CondaRuntime

with_custom_conda(uri: str, region: str | None = None)[source]¶

Specifies the custom conda pack for running the job Make sure you have configured the IAM policy for the job run to access the conda environment.

Parameters:

uri (str) – The OCI object storage URI for the conda pack, e.g. “oci://your_bucket@namespace/object_name.” In the Environment Explorer of an OCI notebook session, this is shown as the “source” of the conda pack.
region (str, optional) –
The region of the bucket storing the custom conda pack, by default None. If region is not specified, ADS will use the region from your authentication credentials:
- For API Key, config[“region”] is used.
- For Resource Principal, signer.region is used.
This is required if the conda pack is stored in a different region.

Returns:

The runtime instance.

Return type:

self

See also

https: //docs.oracle.com/en-us/iaas/data-science/using/conda_publishs_object.htm

with_overwrite(overwrite: bool) → DataFlowRuntime[source]¶

Whether to overwrite the existing script in object storage (script bucket). If the Object Storage bucket already contains a script with the same name, then it will be overwritten with the new one if the overwrite flag equal to True.

Parameters:: overwrite (bool) – Whether to overwrite the existing script in object storage (script bucket).
Returns:: The DataFlowRuntime instance (self).
Return type:: DataFlowRuntime

with_script_bucket(bucket) → DataFlowRuntime[source]¶

Set object storage bucket to save the script, in case script uri given is local.

Parameters:: bucket (str) – name of the bucket
Returns:: runtime instance itself
Return type:: DataFlowRuntime

with_script_uri(path: str) → DataFlowRuntime[source]¶

Set script uri.

Parameters:: path (str) – uri to the script
Returns:: runtime instance itself
Return type:: DataFlowRuntime

with_service_conda(slug: str)[source]¶

Specifies the service conda pack for running the job

Parameters:: slug (str) – The slug name of the service conda pack
Returns:: The runtime instance.
Return type:: self

class ads.jobs.builders.runtimes.python_runtime.GitPythonRuntime(spec: Dict | None = None, **kwargs)[source]¶

Bases: CondaRuntime, _PythonRuntimeMixin

Represents a job runtime with source code from git repository

Example:

runtime = (
    GitPythonRuntime()
    .with_environment_variable(GREETINGS="Welcome to OCI Data Science")
    # Specify the service conda environment by slug name.
    .with_service_conda("pytorch19_p37_gpu_v1")
    # Specify the git repository
    # Optionally, you can specify the branch or commit
    .with_source("https://github.com/pytorch/tutorials.git")
    # Entrypoint is a relative path from the root of the git repo.
    .with_entrypoint("beginner_source/examples_nn/polynomial_nn.py")
    # Copy files in "beginner_source/examples_nn" to object storage after job finishes.
    .with_output(
      output_dir="beginner_source/examples_nn",
      output_uri="oci://bucket_name@namespace/path/to/dir"
    )
)

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

CONST_BRANCH = 'branch'¶

CONST_COMMIT = 'commit'¶

CONST_GIT_SSH_SECRET_ID = 'gitSecretId'¶

CONST_GIT_URL = 'url'¶

CONST_SKIP_METADATA = 'skipMetadataUpdate'¶

attribute_map = {'args': 'args', 'branch': 'branch', 'commit': 'commit', 'conda': 'conda', 'definedTags': 'defined_tags', 'entryFunction': 'entry_function', 'entrypoint': 'entrypoint', 'env': 'env', 'freeformTags': 'freeform_tags', 'gitSecretId': 'git_secret_id', 'outputDir': 'output_dir', 'outputUri': 'output_uri', 'pythonPath': 'python_path', 'skipMetadataUpdate': 'skip_metadata_update', 'url': 'url', 'workingDir': 'working_dir'}¶

property branch: str¶: Git branch name.

property commit: str¶: Git commit ID (SHA1 hash)

init(**kwargs) → GitPythonRuntime[source]¶

Initializes a starter specification for the runtime.

Returns:: The runtime instance.
Return type:: GitPythonRuntime

property skip_metadata_update¶

Indicate if the metadata update should be skipped after the job run

By default, the job run metadata will be updated with the following freeform tags: * repo: The URL of the Git repository * commit: The Git commit ID * module: The entry script/module * method: The entry function/method * outputs. The prefix of the output files in object storage.

This update step also requires resource principals to have the permission to update the job run.

Returns:: True if the metadata update will be skipped. Otherwise False.
Return type:: bool

property ssh_secret_ocid: str¶: The OCID of the OCI Vault secret storing the Git SSH key.

property url: str¶: URL of the Git repository.

with_source(url: str, branch: str | None = None, commit: str | None = None, secret_ocid: str | None = None)[source]¶

Specifies the Git repository and branch/commit for the job source code.

Parameters:

url (str) – URL of the Git repository.
branch (str, optional) – Git branch name, by default None, the default branch will be used.
commit (str, optional) – Git commit ID (SHA1 hash), by default None, the most recent commit will be used.
secret_ocid (str) – The secret OCID storing the SSH key content for checking out the Git repository.

Returns:

The runtime instance.

Return type:

self

class ads.jobs.builders.runtimes.python_runtime.NotebookRuntime(spec: Dict | None = None, **kwargs)[source]¶

Bases: CondaRuntime

Represents a job runtime with Jupyter notebook

To run a job with a single Jupyter notebook, you can define the run time as:

runtime = (
    NotebookRuntime()
    .with_notebook(
        path="https://raw.githubusercontent.com/tensorflow/docs/master/site/en/tutorials/customization/basics.ipynb",
        encoding='utf-8'
    )
    .with_service_conda("tensorflow28_p38_cpu_v1")
    .with_environment_variable(GREETINGS="Welcome to OCI Data Science")
    .with_exclude_tag(["ignore", "remove"])
    .with_output("oci://bucket_name@namespace/path/to/dir")
)

Note that the notebook path can be local or remote path supported by fsspec, including OCI object storage path like oci://bucket@namespace/path/to/notebook

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

CONST_ENTRYPOINT = 'entrypoint'¶

CONST_EXCLUDE_TAG = 'excludeTags'¶

CONST_NOTEBOOK_ENCODING = 'notebookEncoding'¶

CONST_NOTEBOOK_PATH = 'notebookPathURI'¶

CONST_OUTPUT_URI = 'outputUri'¶

CONST_OUTPUT_URI_ALT = 'outputURI'¶

CONST_SOURCE = 'source'¶

attribute_map = {'args': 'args', 'conda': 'conda', 'definedTags': 'defined_tags', 'entrypoint': 'entrypoint', 'env': 'env', 'excludeTags': 'exclude_tags', 'freeformTags': 'freeform_tags', 'notebookEncoding': 'notebook_encoding', 'notebookPathURI': 'notebook_path_uri', 'outputUri': 'output_uri', 'source': 'source'}¶

property exclude_tag: list¶: A list of cell tags indicating cells to be excluded from the job

init(**kwargs) → NotebookRuntime[source]¶

Initializes a starter specification for the runtime.

Returns:: The runtime instance.
Return type:: NotebookRuntime

property notebook: str¶: The path of the notebook relative to the source.

property notebook_encoding: str¶: The encoding of the notebook

property notebook_uri: str¶: The URI of the notebook

property output_uri: list¶: URI for storing the output notebook and files

property source: str¶: The source code location.

with_exclude_tag(*tags) → NotebookRuntime[source]¶

Specifies the cell tags in the notebook to exclude cells from the job script.

Parameters:: *tags (list) – A list of tags (strings).
Returns:: The runtime instance.
Return type:: self

with_notebook(path: str, encoding='utf-8') → NotebookRuntime[source]¶

Specifies the notebook to be run as a job. Use this method if you would like to run a single notebook. Use with_source() method if you would like to run a notebook with additional dependency files.

Parameters:

path (str) – The path of the Jupyter notebook
encoding (str) – The encoding for opening the notebook. Defaults to utf-8.

Returns:

The runtime instance.

Return type:

self

with_output(output_uri: str) → NotebookRuntime[source]¶

Specifies the output URI for storing the output notebook and files. All files in the directory containing the notebook will be saved.

Parameters:: output_uri (str) – URI for a directory storing the output notebook and files. For example, oci://bucket@namespace/path/to/dir
Returns:: The runtime instance.
Return type:: self

with_source(uri: str, notebook: str, encoding='utf-8')[source]¶

Specify source code directory containing the notebook and dependencies for the job. Use this method if you would like to run a notebook with additional dependency files. Use the with_notebook() method if you would like to run a single notebook.

In the following example, local folder “path/to/source” contains the notebook and dependencies, The local path of the notebook is “path/to/source/relative/path/to/notebook.ipynb”:

runtime.with_source(uri="path/to/source", notebook="relative/path/to/notebook.ipynb")

Parameters:

uri (str) – URI of the source code directory. This can be local or on OCI object storage.
notebook (str) – The relative path of the notebook from the source URI.
encoding (str) – The encoding for opening the notebook. Defaults to utf-8.

Returns:

The runtime instance.

Return type:

Self

class ads.jobs.builders.runtimes.python_runtime.PythonRuntime(spec: Dict | None = None, **kwargs)[source]¶

Bases: ScriptRuntime, _PythonRuntimeMixin

Represents a job runtime using ADS driver script to run Python code

Example:

runtime = (
    PythonRuntime()
    # Specify the service conda environment by slug name.
    .with_service_conda("pytorch110_p38_cpu_v1")
    # The job artifact can be a single Python script, a directory or a zip file.
    .with_source("local/path/to/code_dir")
    # Environment variable
    .with_environment_variable(NAME="Welcome to OCI Data Science.")
    # Command line argument, arg1 --key arg2
    .with_argument("arg1", key="arg2")
    # Set the working directory
    # When using a directory as source, the default working dir is the parent of code_dir.
    # Working dir should be a relative path beginning from the source directory (code_dir)
    .with_working_dir("code_dir")
    # The entrypoint is applicable only to directory or zip file as source
    # The entrypoint should be a path relative to the working dir.
    # Here my_script.py is a file in the code_dir/my_package directory
    .with_entrypoint("my_package/my_script.py")
    # Add an additional Python path, relative to the working dir (code_dir/other_packages).
    .with_python_path("other_packages")
    # Copy files in "code_dir/output" to object storage after job finishes.
    .with_output("output", "oci://bucket_name@namespace/path/to/dir")
)

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

attribute_map = {'args': 'args', 'conda': 'conda', 'definedTags': 'defined_tags', 'entryFunction': 'entry_function', 'entrypoint': 'entrypoint', 'env': 'env', 'freeformTags': 'freeform_tags', 'outputDir': 'output_dir', 'outputUri': 'output_uri', 'pythonPath': 'python_path', 'scriptPathURI': 'script_path_uri', 'workingDir': 'working_dir'}¶

init(**kwargs) → PythonRuntime[source]¶

Initializes a starter specification for the runtime.

Returns:: The runtime instance.
Return type:: PythonRuntime

class ads.jobs.builders.runtimes.python_runtime.ScriptRuntime(spec: Dict | None = None, **kwargs)[source]¶

Bases: CondaRuntime

Represents job runtime with scripts and conda pack.

This runtime is designed to define job artifacts and configurations supported by OCI Data Science Jobs natively. It can be used with any script types that is supported by the OCI Data Science Jobs, including shell scripts and python scripts.

To run a script with all dependencies contained in a local folder:

runtime = (
    ScriptRuntime()
    # Specify the service conda environment by slug name.
    .with_service_conda("pytorch110_p38_cpu_v1")
    # The job artifact can be a single Python script, a directory or a zip file.
    .with_source("local/path/to/code_dir")
    # Environment variable
    .with_environment_variable(NAME="Welcome to OCI Data Science.")
    # Command line argument
    .with_argument("100 linux 'hi there'")
    # The entrypoint is applicable only to directory or zip file as source
    # The entrypoint should be a path relative to the working dir.
    # Here my_script.sh is a file in the code_dir/my_package directory
    .with_entrypoint("my_package/my_script.sh")
)

References

https://docs.oracle.com/en-us/iaas/data-science/using/jobs-artifact.htm

To initialize the object, user can either pass in the specification as a dictionary or through keyword arguments.

Parameters:

spec (dict, optional) – Object specification, by default None
kwargs (dict) – Specification as keyword arguments. If spec contains the same key as the one in kwargs, the value from kwargs will be used.

CONST_ENTRYPOINT = 'entrypoint'¶

CONST_SCRIPT_PATH = 'scriptPathURI'¶

attribute_map = {'args': 'args', 'conda': 'conda', 'definedTags': 'defined_tags', 'entrypoint': 'entrypoint', 'env': 'env', 'freeformTags': 'freeform_tags', 'scriptPathURI': 'script_path_uri'}¶

property entrypoint: str¶: The relative path of the script to be set as entrypoint when source is a zip/tar/directory.

init(**kwargs) → ScriptRuntime[source]¶

Initializes a starter specification for the runtime.

Returns:: The runtime instance.
Return type:: ScriptRuntime

property script_uri: str¶: The URI of the source code

property source_uri: str¶: The URI of the source code

with_entrypoint(entrypoint: str)[source]¶

Specify the entrypoint for the job

Parameters:: entrypoint (str) – The relative path of the script to be set as entrypoint when source is a zip/tar/directory.
Returns:: The runtime instance.
Return type:: self

with_script(uri: str)[source]¶

Specifies the source code script for the job

Parameters:: uri (str) – URI to the source code script, which can be any URI supported by fsspec, including http://, https:// and OCI object storage. For example: oci://your_bucket@your_namespace/path/to/script.py
Returns:: The runtime instance.
Return type:: self

with_source(uri: str, entrypoint: str | None = None)[source]¶

Specifies the source code for the job

Parameters:

uri (str) – URI to the source code, which can be a (.py/.sh) script, a zip/tar file or directory containing the scripts/modules If the source code is a single file, URI can be any URI supported by fsspec, including http://, https:// and OCI object storage. For example: oci://your_bucket@your_namespace/path/to/script.py URI can also be a folder or a zip file containing the source code. In that case, entrypoint is required.
entrypoint (str, optional) – The relative path of the script to be set as entrypoint when source is a zip/tar/directory. By default None. This is not needed when the source is a single script.

Returns:

The runtime instance.

Return type:

self

ads.jobs.builders.runtimes package¶

Submodules¶

ads.jobs.builders.runtimes.artifact module¶

ads.jobs.builders.runtimes.base module¶

ads.jobs.builders.runtimes.container_runtime module¶

ads.jobs.builders.runtimes.python_runtime module¶

Module contents¶