Pipeline comparison – Pipelines using Kubeflow for Custom Models

Users may run multiple pipelines with different models or with a different sample of data. GCP provides users an option to compare the performance of different pipelines.

In this exercise, we had built a pipeline to train random forest classifier model. Change random forest to any other model of your choice and run the pipeline (decision tree classifier can be used in this scenario with minimal changes in the code). Import the required package and make necessary changes in the code (at third and fourth component of the pipeline). Change the name of the JSON file, and the pipeline name. Compile and submit the pipeline job to execute.

Once the pipeline is executed successfully follow these steps to compare the pipelines.

Step 1: Pipelines module of Vertex AI

All the pipelines executed will be listed under pipeline section as shown in Figure 7.17:

Figure 7.17: Pipeline module of Vertex AI

1. Go to the Pipelines module of Vertex AI.
2. Select the pipeline to be compared.
3. Click COMPARE.

Step 2: Pipeline comparison

Using the pipeline comparison, users can compare the parameters of the pipeline used, the output metrics and KPIs of the pipelines as shown in Figure 7.18:

Figure 7.18: Pipeline comparison

Deletion of resources

We have utilized workbench, and BigQuery to store the data and cloud storage to store the artifacts of the pipeline. Ensure to delete the workbench, clear the data stored in the cloud storage and BigQuery.

Differences between Vertex AI and Kubeflow pipelines

Few important differences between Vertex AI pipelines and Kubeflow pipelines are listed below:

• Domain-Specific Language (DSL) versions: Pipelines created using TFX v0.30.0 or later and Kubeflow Pipelines SDK v2 domain-specific language (DSL) can be executed on the Vertex AI pipelines. Only the Kubeflow Pipelines SDK are executed using Kubeflow pipelines.
• Storage: Kubernetes resources like persistent volume claims can be used in Kubeflow Pipelines, whereas in Vertex AI cloud storage is used for data storage (Cloud Storage FUSE is used for mounting to the components)
• Recursion: Pipeline components that are called recursively are not supported by Vertex AI whereas Kubeflow supports it.

Conclusion

We learnt how to construct pipeline using custom model using Kubeflow SDK, compile and submit the pipeline job. The use of lineage in the pipeline, comparing different pipelines and its uses were also covered in this chapter.

In the next chapter, we will construct the pipeline using TensorFlow Extended (TFX) for custom model training.

Questions

1. What changes are needed if we have to execute the pipeline using xgboost/pytorch /TensorFlow?
2. Why is Pipeline comparison helpful?
3. How are the artifacts and the parameters of the pipeline tracked?