Pipeline – Pipelines using Kubeflow for Custom Models

Follow these steps to analyze the status of the pipeline job, artifacts, lineage and output:

Step 1: Pipeline of custom model

Open the link as shown in Figure 7.10 to navigate to the pipelines of Vertex AI. The pipeline will start executing and will take about 5 to 10 mins. All the four tasks in the exercise will be executed sequentially. Once the complete pipeline is executed, you will see green color tick mark on all the components as shown in Figure 7.11:

Figure 7.11: Pipeline created & executed successfully

1. Indicating all the steps in the pipelines are executed successfully.
2. Click SUMMARY section of the pipeline
3. Basic info provides the information of pipeline run, region, link to view the logs, and so on.
4. Run parameters provides information regarding the parameters that are used for the pipeline. Scroll down to get the run metrics in the summary section.
5. Click the toggle button to expand the artifacts

Step 2: Expand artifacts

Artifacts will be expanded in the pipeline as shown in Figure 7.12:

Figure 7.12: Components and artifacts of the pipeline

1. Boxes with a tick on it indicates the components of the pipeline. Others are the artifacts of the pipeline. Click on it.
2. Click NODE INFO.
3. Location of the cloud storage where the artifacts are stored.
4. Click VIEW LINEAGE.

Step 3: Lineage of the pipeline

The artifacts and parameters of a pipeline run are recorded in Vertex ML Metadata when you use Vertex AI Pipelines to execute the pipeline. By relieving users from the burden of maintaining pipeline’s information, Vertex ML Metadata facilitates analysis of the items’ provenance. Everything that went into making an artifact, as well as any derivative objects or information, are all part of its lineage.

Lineage for the pipeline will open in a separate tab as shown in Figure 7.13:

Figure 7.13: Lineage of the pipeline

Step 4: Metrices and KPIs recorded in the pipeline

Metrices of trained model will be available in the artifacts as shown in Figure 7.14:

Figure 7.14: Metrics of the model

1. Click on the metrices artifacts of the pipeline.
2. In the last component of the pipeline, we have recorded Confusion matrix and ROC curve (scroll down to see it) are displayed.

KPIs of trained model will be available in the artifacts as shown in Figure 7.15:

Figure 7.15: KPIs of the model

1. Click the kpi artifacts of the pipeline.
2. In the last component of the pipeline, we have recorded accuracy and other KPIs of the model which are displayed.

Step 5: Pipeline artifacts stored in cloud storage

Pipeline artifacts are stored in the cloud storage as shown in the following figure. Separate folders are created for the artifacts from different components of the pipeline:

Figure 7.16: Artifacts stored in the cloud storage