Functionalities of custom components – Pipelines using TensorFlow Extended

Let us understand the functionalities of these components.

• Examplegen: The ExampleGen TFX pipeline component acts as the entry point and ingests data. ExampleGen scan automatically consumes external data sources, including CSV, TF Records, Avro, and Parquet, when used as inputs. ExampleGen generates TF examples, also known as TF sequence examples, as its outputs. These examples are particularly effective in performing data set representations and can be read reliably by subsequent components.
• StatisticsGen: Other pipeline components can make use of the statistics generated by the StatisticsGen TFX pipeline component. These statistics may be generated over both training data and serving data.
• SchemaGen: A description of the data that you enter, known as a schema, is used by the TFX components. An instance of the schema.proto class is being used here. It can specify the data types for feature values, the ranges of acceptable values, whether a feature must be present in all samples, and other features. In addition to this, a component of the SchemaGen pipeline will automatically construct a schema by deducing types, categories, and ranges from the training data.
• ExampleValidator: A component of the pipeline called ExampleValidator looks for irregularities in the data used for training and serving. It can identify several categories of abnormalities within the data. In addition to that, it can:
  • By comparing data statistics to a schema that details the user’s expectations, validity checks can be carried out.
  • Compares the data from training and serving to look for any differences between the two.
  • Examining a data series can allow you to identify any instances of data drift.
• Transform: The ExampleGen component is responsible for emitting data artifacts, and the Transform TFX pipeline component is responsible for performing feature engineering on those artifacts. Transform component imports the data schema artifact from external sources or using the data schema artifact that was generated by SchemaGen.
• Trainer: A component of a pipeline called Trainer TFX is responsible for training a TensorFlow model. The trainer component generates at least one model for inference and stores it in a format that is compatible with TensorFlow saved models. A trustworthy model will include an entire TensorFlow program, with all its weights and computations included.
• Tuner: The Tuner component is responsible for fine-tuning the model’s hyperparameters. The Tuner component is the most recent addition to the TFX effects component family and relies heavily on the Python Keras tuner API to tune hyperparameters. The transformed data is provided in the form of transform graph artifacts for the tuner component to take in as inputs, and the tuner component produces a hyperparameter artifact as an output.
• Evaluator: The model that was developed by the trainer will be used by the evaluator component which will use the original input data artifact. In addition to this, it will conduct a comprehensive analysis by making use of the TensorFlow model analysis library.
• InfraValidator: Before sending a model into production, the InfraValidator, which is a TFX component, serves as an early warning layer to catch any potential issues. The validation of the model takes place within the actual infrastructure that serves the model, which is where the name InfraValidator comes from. If the evaluator guarantees that the model will perform as expected, then InfraValidator will also guarantee that the model will not have any mechanical issues.
• Pusher: During the process of model training or re-training, the Pusher component is used to send a model that has been validated to a deployment target.