Model Registration

Model Artifact

To save a trained model on OCI Data Science, prepare a Model Artifact.

Model Artifact is a zip file which contains the following artifacts -

  • Serialized model or models

  • runtime.yaml - This yaml captures provenance information and deployment conda environment

  • score.py - Entry module which is used by the model deployment server to load the model and run prediction

  • input_schema.json - Describes the schema of the features that will be used within predict function

  • output_schema.json - Describes the schem of the prediction values

  • Any other artifcat that are required during inference time.

ADS can auto generate all the mandatory files to help save the models that are compliant with the OCI Data Science Model Deployment service.

Auto generation of score.py with framework specific code for loading models and fetching prediction is available for following frameworks-

  • scikit-learn

  • XGBoost

  • LightGBM

  • PyTorch

  • SparkPipelineModel

  • TensorFlow

To accomodate for other frameworks that are unknown to ADS, a template code for score.py is generated in the provided artificat directory location.

Prepare the Model Artifact

To prepare the model artifact -

  • Train a model using the framework of your choice

  • Create a Model object from one of the framework specific Models available under ads.model.framework.*. The Model class takes two parameters - estimator object and a directory location to store autogenerated artifacts.

  • call prepare() to generate all the files.

See API documentation for more details about the parameters.

Here is an example for preparing a model artifact for TensorFlow model.

from ads.catalog.model import ModelCatalog
from ads.model.framework.tensorflow_model import TensorFlowModel
import tempfile
import tensorflow as tf
from ads.common.model_metadata import UseCaseType

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

tf_estimator = tf.keras.models.Sequential(
        [
            tf.keras.layers.Flatten(input_shape=(28, 28)),
            tf.keras.layers.Dense(128, activation="relu"),
            tf.keras.layers.Dropout(0.2),
            tf.keras.layers.Dense(10),
        ]
    )
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
tf_estimator.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
tf_estimator.fit(x_train, y_train, epochs=1)

tf_model = TensorFlowModel(tf_estimator, artifact_dir=tempfile.mkdtemp())

# Autogenerate score.py, pickled model, runtime.yaml, input_schema.json and output_schema.json
tf_model.prepare(inference_conda_env="generalml_p37_cpu_v1",
                    use_case_type=UseCaseType.MULTINOMIAL_CLASSIFICATION,
                    X_sample=trainx,
                    y_sample=trainy
                )

# Verify generated artifacts
tf_model.verify(x_test[:1])

# Register TensorFlow model
model_id = tf_model.save()
['output_schema.json', 'score.py', 'runtime.yaml',  'model.h5', '.model-ignore', 'input_schema.json']

ADS automatically captures:

  • Provenance metadata - commit id, git branch, etc

  • Taxonomy metadata such as model hyperparameters, framework name.

  • Custom metadata such as Data science conda environment when available, Model artifact inventory, model serialization format, etc.

  • Schema of input and target variables. This requires input sample and target sample to be passed while calling prepare

Note:

  • UseCaseType in metadata_taxonomy cannot be automatically populated. One way to populate the use case is to pass use_case_type to the prepare method.

  • Model introspection is automatically triggered.

score.py

In the prepare step, the service automatically generates a score.py file in the artifact directory.

score.py is used by the Data Science Model Deployment service to generate predictions in the input feature. Here is a minimal score.py implementation -

import joblib
model_name = "model.joblib"

def load_model(): # load_model must mandatorily return ``not None`` object.

  model = None
  with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), model_file_name), "rb") as mfile:
    model = joblib.load(mfile)
  return model

def predict(data, model=load_model()):
  return model.predict(data).tolist()

ADS autogenerates framework specific score.py which provides following functionality -

  • Parse the input data and convert to pandas dataframe/numpy array/list

  • Ensure the data type after converting to pandas dataframe matches the training time. This is achieved using the schema definition generated during prepare step.

  • Serialize prediction generated by the model such that it is json serializable to avoid deployment runtime errors

You could customize the score.py to fit your use case. The most common use case for changing the score.py file is to add preprocessing and postprocessing steps to the predict() method.

Refer Cusotmization section for how to change and verify the model artifacts.

The score.py consists of multiple functions among which the load_model and predict are most important.

load_model

During deployment, the load_model method loads the serialized model. The load_model method is always fully populated, except when you set serialize=False for GenericModel.

  • For the GenericModel class, if you choose serialize=True in the init function, the model is pickled and the score.py is fully auto-populated to support loading the pickled model. Otherwise, the user is responsible to fill the load_model.

  • For other frameworks, this part is fully populated.

Note: load_model should return not None value for successful deployment.

predict

The predict method is triggered every time a payload is sent to the model deployment endpoint. The method takes the payload and the loaded model as inputs. Based on the payload, the method returns the predicted results output by the model.

pre_inference

If the payload passed to the endpoint needs preprocessing, this function does the preprocessing step. The user is fully responsible for the preprocessing step.

post_inference

If the predicted result from the model needs some postprocessing, the user can put the logic in this function.

deserialize

When you use the .verify() or .predict() methods from model classes such as GenericModel or SklearnModel, if the data passed in is not in bytes or JsonSerializable, the models try to serialize the data. For example, if a pandas dataframe is passed and not accepted by the deployment endpoint, the pandas dataframe is converted to JSON internally. When the X_sample variable is passed into the .prepare() function, the data type of pandas dataframe is passed to the endpoint, and the schema of the dataframe is recorded in the input_schema.json file. Then, the JSON payload is sent to the endpoint. Because the model expects to take a pandas dataframe, the .deserialize() method converts the JSON back to the pandas dataframe using the schema and the data type. For all frameworks except for the GenericModel class, the .deserialize() method is auto-populated. Note that for each framework, only specific data types are supported.

Starting from .. versionadded:: 2.6.3, you can send the bytes to the endpoint directly. If the bytes payload is sent to the endpoint, bytes are passed directly to the model. If the model expects a specific data format, you need to write the conversion logic yourself.

fetch_data_type_from_schema

This function is used to load the schema from the input_schema.json when needed.

Model Introspection

The .intropect() method runs some sanity checks on the runtime.yaml, and score.py files. This is to help you identify potential errors that might occur during model deployment. It checks fields such as environment path, validates the path’s existence on the Object Storage, checks if the .load_model(), and .predict() functions are defined in score.py, and so on. The result of model introspection is automatically saved to the taxonomy metadata and model artifacts.

tf_model.introspect()
['output_schema.json', 'runtime.yaml', 'model.joblib', 'input_schema.json', 'score.py']
../../_images/introspection1.png

Reloading model artifacts automatically invokes model introspection. However, you can invoke introspection manually by calling tf_model.introspect():

The ArtifactTestResults field is populated in metadata_taxonomy when instrospect is triggered:

tf_model.metadata_taxonomy['ArtifactTestResults']
key: ArtifactTestResults
value:
  runtime_env_path:
    category: conda_env
    description: Check that field MODEL_DEPLOYMENT.INFERENCE_ENV_PATH is set
  ...

Save Model

To .save() method saves the model, introspection results, schema, metadata, etc on OCI Data Science Service and returs the model ocid.

See API documentation for more details.