Skip to content

Latest commit

 

History

History
660 lines (554 loc) · 37 KB

RELEASE.md

File metadata and controls

660 lines (554 loc) · 37 KB

Upcoming release

  • Bump image for Structured Data pipelines.

Release 2.19.0

  • Add max_wait_duration to v1 GCPC custom job components/utils
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.18.0

  • Remove default prediction column names in v1.model_evaluation.regression_component component to fix pipeline errors when using bigquery data source.
  • Add reservation_affinition support in v1.create_custom_training_job_from_component.
  • Deprecate preview.custom_job module.
  • Fix default location in v1.create_custom_training_job_from_component.
  • Update Docker image.
  • Bump supported KFP versions to kfp>=2.6.0,<2.11.0.
  • Support Python versions 3.12 and 3.13.
  • Bump image for Structured Data pipelines.
  • Add strategy to v1 GCPC custom job components/utils
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.17.0

  • Fix Gemini batch prediction support to v1.model_evaluation.autosxs_pipeline after output schema change.
  • Modify the code to support latest bp result format
  • Update the StarryNet package metadata.
  • Use instance.target_field_name format for text-bison models only, use target_field_name for gemini models.
  • Pass model name to eval_runner to process batch prediction's output as per the output schema of model used.
  • Use LLM Model Evaluation image version v0.7
  • Update AutoSxS and RLHF image tags

Release 2.16.1

  • Fix to model batch explanation component for Structured Data pipelines; image bump.
  • Add dynamic support for boot_disk_type, boot_disk_size in preview.custom_job.utils.create_custom_training_job_from_component.
  • Remove preflight validations temporarily.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.16.0

  • Updated the Starry Net pipeline's template gallery description, and added dataprep_nan_threshold and dataprep_zero_threshold args to the Starry Net pipeline.
  • Fix bug in Starry Net's upload decomposition plot step due to protobuf upgrade, by pinning protobuf library to 3.20.*.
  • Bump Starry Net image tags.
  • In the Starry-Net pipeline, enforce that TF Record generation always runs before test set generation to speed up pipelines runs.
  • Add support for running tasks on a PersistentResource (see CustomJobSpec) via persistent_resource_id parameter on v1.custom_job.CustomTrainingJobOp and v1.custom_job.create_custom_training_job_from_component
  • Bump image for Structured Data pipelines.
  • Add check that component in preview.custom_job.utils.create_custom_training_job_from_component doesn't have any parameters that share names with any custom job fields
  • Add dynamic machine spec support for preview.custom_job.utils.create_custom_training_job_from_component.
  • Add preflight validations for LLM text generation pipeline.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.15.0

  • Add Gemini batch prediction support to v1.model_evaluation.autosxs_pipeline.
  • Add Starry Net forecasting pipeline to preview.starry_net.starry_net_pipeline
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.14.1

  • Add staging and temp location parameters to prophet trainer component.
  • Add input parameter autorater_prompt_parameters to _implementation.llm.online_evaluation_pairwise component.
  • Mitigate bug in v1.model_evaluation.autosxs_pipeline where batch prediction would fail the first time it is run in a project by retrying.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.14.0

  • Use larger base reward model when tuning text-bison@001, chat-bison@001 and t5-xxl with the preview.llm.rlhf_pipeline.
  • Move preview.model_evaluation.autosxs_pipeline to v1.model_evaluation.autosxs_pipeline.
  • Remove default prediction column names in v1.model_evaluation.classification_component component to fix pipeline errors when using bigquery data source.
  • Move _implementation.model_evaluation.ModelImportEvaluationOp component to preview namespace preview.model_evaluation.ModelImportEvaluationOp.
  • Drop support for Python 3.7 since it has reached end-of-life.
  • Expand number of regions supported by preview.llm.rlhf_pipeline.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.13.1

  • Fix model name preprocess error, pass correct model to ModelImportEvaluationOp component in v1.model_evaluation.evaluation_llm_text_generation_pipeline and v1.model_evaluation.evaluation_llm_classification_pipeline.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.13.0

  • Add support for text-bison@002 to preview.llm.rlhf_pipeline.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).
  • Fix preview.model_evaluation.autosxs_pipeline documentation to show autorater_prompt_parameters as required.
  • Introduce placeholders: SERVICE_ACCOUNT_PLACEHOLDER, NETWORK_PLACEHOLDER, PERSISTENT_RESOURCE_ID_PLACEHOLDER and ENCRYPTION_SPEC_KMS_KEY_NAME_PLACEHOLDER
  • Use PERSISTENT_RESOURCE_ID_PLACEHOLDER as the default value of persistent_resource_id for CustomTrainingJobOp and create_custom_training_job_op_from_component. With this change, custom job created without explicitly setting persistent_resource_id will inherit job level persistent_resource_id, if Persistent Resource is set as job level runtime.

Release 2.12.0

  • Log TensorBoard metrics from the preview.llm.rlhf_pipeline in real time.
  • Add task_type parameter to preview.llm.rlaif_pipeline.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.11.0

  • Fix bug in preview.llm.rlhf_pipeline that caused wrong output artifact to be used for inference after training.
  • Fix issue where AutoSxS was not propagating location to all sub-components.
  • Add CMEK support to preview.llm.infer_pipeline.
  • Use eval_dataset for train-time evalutation when training a reward model. Requires eval_dataset to contain the same fields as the preference dataset.
  • Update the documentation of GetModel.
  • Add CMEK support to preview.model_evaluation.autosxs_pipeline.
  • Updated component and pipeline inputs/outputs to support creating ModelEvaluations for ModelRegistry models in the AutoSxS pipeline.
  • Add DRZ-at-rest to preview.llm.rlhf_pipeline.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.10.0

  • Fix the missing output of pipeline remote runner. AutoMLImageTrainingJobRunOp now passes the model artifacts correctly to downstream components.
  • Fix the metadata of Model Evaluation resource when row based metrics is disabled in preview.model_evaluation.evaluation_llm_text_generation_pipeline.
  • Support Jinja2>=3.1.2,<4.
  • Support custom AutoSxS tasks.
  • Bump supported KFP versions to kfp>=2.6.0,<=2.7.0.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).
  • Add CMEK support to preview.llm.rlhf_pipeline when tuning in us-central1 with GPUs.

Release 2.9.0

  • Use large_model_reference for model_reference_name when uploading models from preview.llm.rlhf_pipeline instead of hardcoding value as text-bison@001.
  • Disable caching when resolving model display names for RLHF-tuned models so a unique name is generated on each preview.llm.rlhf_pipeline run.
  • Upload the tuned adapter to Model Registry instead of model checkpoint from preview.llm.rlhf_pipeline.
  • Fix the naming of AutoSxS's question answering task. "question_answer" -> "question_answering".
  • Add Vertex model get component (v1.model.ModelGetOp).
  • Migrate to Protobuf 4 (protobuf>=4.21.1,<5). Require kfp>=2.6.0.
  • Support setting version aliases in (v1.model.ModelUploadOp).
  • Only run preview.llm.bulk_inference pipeline after RLHF tuning for third-party models when eval_dataset is provided.
  • Update LLM Evaluation Pipelines to use text-bison@002 model by default.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).
  • Add preview.llm.rlaif_pipeline that tunes large-language models from AI feedback.

Release 2.8.0

  • Release AutoSxS pipeline to preview.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.7.0

  • Fix v1.automl.training_job.AutoMLImageTrainingJobRunOp ModuleNotFoundError.
  • Append tune-type to existing labels when uploading models tuned by preview.llm.rlhf_pipeline instead of overriding them.
  • Use llama-2-7b for the base reward model when tuning llama-2-13b with the preview.llm.rlhf_pipeline
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.6.0

  • Bump supported KFP versions to kfp>=2.0.0b10,<=2.4.0
  • Add LLM Eval pipeline parameter for customizing eval dataset reference ground truth field
  • Create new eval dataset preprocessor for formatting eval dataset in tuning dataset format.
  • Support customizing eval dataset format in Eval LLM Text Generation Pipeline (preview.model_evaluation.evaluation_llm_text_generation_pipeline) and LLM Text Classification Pipeline (preview.model_evaluation.evaluation_llm_classification_pipeline). Include new LLM Eval Preprocessor component in both pipelines.
  • Fix the output parameter output_dir of preview.automl.vision.DataConverterJobOp.
  • Fix batch prediction model parameters payload sanitization error .
  • Add ability to perform inference with chat datasets to preview.llm.infer_pipeline.
  • Add ability to tune chat models with preview.llm.rlhf_pipeline.
  • Group preview.llm.rlhf_pipeline components for better readability.
  • Add environment variable support to GCPC's create_custom_training_job_from_component (both v1 and preview namespaces)
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.5.0

  • Upload tensorboard metrics from preview.llm.rlhf_pipeline if a tensorboard_resource_id is provided at runtime.
  • Support incremental_train_base_model, parent_model, is_default_version, model_version_aliases, model_version_description in AutoMLImageTrainingJobRunOp.
  • Add preview.automl.vision and DataConverterJobOp.
  • Set display names for preview.llm pipelines.
  • Add sliced evaluation metrics support for custom and unstructured AutoML models in evaluation pipeline and evaluation pipeline with feature attribution.
  • Support service_account in ModelBatchPredictOp.
  • Release DataflowFlexTemplateJobOp to GA namespace (v1.dataflow.DataflowFlexTemplateJobOp).
  • Make model_checkpoint optional for preview.llm.infer_pipeline. If not provided, the base model associated with the large_model_reference will be used.
  • Bump apache_beam[gcp] version in GCPC container image from <2.34.0 to ==2.50.0 for compatibility with google-cloud-aiplatform, which depends on shapely<3.0.0dev. Note: upgrades to google-cloud-pipeline-components>=2.5.0 and later may require using a Dataflow worker image with apache_beam==2.50.0.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)
  • Add support for customizing model_parameters (maxOutputTokens, topK, topP, and temperature) in LLM eval text generation and LLM eval text classification pipelines.

Release 2.4.1

  • Disable caching for LLM pipeline tasks that store temporary artifacts.
  • Fix the mismatched arguments in 2.4.0 for the Feature Transform Engine component.
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates).

Release 2.4.0

  • Add support for running tasks on a PersistentResource (see CustomJobSpec) via persistent_resource_id parameter on preview.custom_job.CustomTrainingJobOp and preview.custom_job.create_custom_training_job_from_component
  • Fix use of encryption_spec_key_name in v1.custom_job.CustomTrainingJobOp and v1.custom_job.create_custom_training_job_from_component
  • Add feature_selection_pipeline to preview.automl.tabular.
  • Bump supported KFP versions to kfp>=2.0.0b10,<=2.2.0
  • Add time_series_dense_encoder_forecasting_pipeline, learn_to_learn_forecasting_pipeline, sequence_to_sequence_forecasting_pipeline, and temporal_fusion_transformer_forecasting_pipeline to preview.automl.forecasting.
  • Add support for customizing evaluation display name on v1 and preview model_evaluation pipelines.
  • Include model version ID in v1.model.upload_model.ModelUploadOp's VertexModel output (key: model). The URI and metadata resourceName field in the outputted VertexModel now have @<model_version_id> appended, corresponding to the model that was just created. Downstream components DeleteModel and UndeployModel will respect the model version if provided.
  • Bump KFP SDK upper bound to 2.3.0
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.3.1

  • Make LLM pipelines compatible with KFP SDK 2.1.3
  • Require KFP SDK <=2.1.3
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.3.0

  • Add preview.llm.infer_pipeline and preview.llm.rlhf_pipeline
  • Add automl_tabular_tabnet_trainer and automl_tabular_wide_and_deep_trainer to preview.automl.tabular and v1.automl.tabular
  • Minor feature additions to AutoML components
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.2.0

  • Add preview.model_evaluation.evaluation_llm_classification_pipeline.evaluation_llm_classification_pipeline
  • Change AutoML Vision Error Analysis pipeline names (`v1.model_evaluation.vision_model_error_analysis_pipeline' and 'v1.model_evaluation.evaluated_annotation_pipeline')
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.1.1

  • Add preview.model_evaluation.FeatureAttributionGraphComponentOp pipeline
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.1.0

  • Add AutoML tabular and forecasting components to preview namespace
  • Fix bug where parent_model parameter of ModelUploadOp ignored
  • Fix circular import bug for model evaluation components
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.0.0

Google Cloud Pipeline Components v2 is generally available!

Structure

Major changes

  • Migrate many components to the v1 GA namespace
  • Migrate components to the preview namespace
    • preview.model_evaluation.ModelEvaluationFeatureAttributionOp
    • preview.model_evaluation.DetectModelBiasOp
    • preview.model_evaluation.DetectDataBiasOp
    • preview.dataflow.DataflowFlexTemplateJobOp
  • Add many new components:
    • v1.dataflow.DataflowFlexTemplateJobOp
    • v1.model.evaluation.vision_model_error_analysis_pipeline
    • v1.model.evaluation.evaluated_annotation_pipeline
    • v1.model.evaluation.evaluation_automl_tabular_feature_attribution_pipeline
    • v1.model.evaluation.evaluation_automl_tabular_pipeline
    • v1.model.evaluation.evaluation_automl_unstructure_data_pipeline
    • v1.model.evaluation.evaluation_feature_attribution_pipeline
  • Make GCPC artifacts usable in user-defined KFP SDK Python components (Containerized Python Components recommended)

Runtime

  • Change runtime base image to marketplace.gcr.io/google/ubuntu2004
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Dependencies

  • Depend on KFP SDK v2 (GCPC v2 is not compatible with KFP v1)
  • Set google-api-core<1.34.0 to avoid 900s timeout
  • Remove google-cloud-notebooks and google-cloud-storage dependencies

Documentation

  • Refresh GCPC v2 reference documentation

Other

  • Assorted minor component interface changes
  • Assorted bug fixes
  • Change force_direct_runner flag to force_direct_runner_mode in experimental evaluation components to allow users to choose the runner of the evaluation pipeline
  • Support upload model with pipeline job id in UploadModel GCPC component
  • Change default value of prediction_score_column for AutoML Forecasting & Regression components to prediction.value
  • Change dataflow_disk_size parameter to dataflow_disk_size_gb in all model evaluation components
  • Remove aiplatform.CustomContainerTrainingJobRunOp and aiplatform.CustomPythonPackageTrainingJobRunOp components

Upcoming changes

  • Additional migrations from the 1.x.x's experimental namespace to the v1 and preview namespaces

Release 2.0.0b5

  • Fix experimental evaluation component runtime bugs
  • Add model evaluation pipelines:
    • v1.model.evaluation.vision_model_error_analysis_pipeline
    • v1.model.evaluation.evaluated_annotation_pipeline
    • v1.model.evaluation.evaluation_automl_tabular_feature_attribution_pipeline
    • v1.model.evaluation.evaluation_automl_tabular_pipeline
    • v1.model.evaluation.evaluation_automl_unstructure_data_pipeline
    • v1.model.evaluation.evaluation_feature_attribution_pipeline
  • Make GCPC artifacts usable in user-defined KFP SDK Python Components and add documentation
  • Change force_direct_runner flag to force_direct_runner_mode in experimental evaluation components to allow users to choose the runner of the evaluation pipeline
  • Add experimental AutoML Forecasting Seq2Seq and Temporal Fusion Transformer pipelines
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 2.0.0b4

  • GCPC v2 reference documentation improvements
  • Change GCPC base image to marketplace.gcr.io/google/ubuntu2004
  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)
  • Fix dataset components
  • Fix payload sanitation bug in google_cloud_pipeline_components.v1.batch_predict_job.ModelBatchPredictOp
  • Assorted experimental component bug fixes (note: experimental namespace will be removed in a future pre-release)

Release 2.0.0b3

  • Support sparse layer masking feature selection for experimental.automl.tabular classification/regression components
  • Fixes for GCPC v2 reference documentation
  • Fix experimental.dataflow.DataflowFlexTemplateJobOp component
  • Remove unused SDK dependency on google-cloud-notebooks and google-cloud-storage

Release 2.0.0b2

  • Add experimental.dataflow.DataflowFlexTemplateJobOp component
  • Remove aiplatform.CustomContainerTrainingJobRunOp and aiplatform.CustomPythonPackageTrainingJobRunOp components
  • Migrate other aiplatform.automl_training_job, aiplatform.ModelUndeployOp, aiplatform.EndpointDeleteOp, and aiplatform.ModelDeleteOp components to the v1 namespace
  • Deduplicate component definitions between experimental and v1 namespaces

Release 2.0.0b1

  • Change base image to ubuntu OS
  • Set google-api-core<1.34.0 to avoid 900s timeout

Release 2.0.0b0

  • Release of GCPC v2 beta
  • Supports KFP v2 beta
  • Experimental components that already in v1 folder are removed
  • Experimental components that are not fully tested (e.g. AutoML, Model Evaluation) are excluded for now, will be added in future releases
  • Even though the GCPC package's version is v2, the components under v1 folder have no interface change, so the those components' version remain as v1, decoupled from package version.

Release 1.0.44

  • Apply latest GCPC image vulnerability resolutions (base OS and software updates)

Release 1.0.43

  • Patch 5de4d78: unpin google-api-core version

Release 1.0.42

  • Patch cb7d9a8: Update import_model_evaluation so models with 100+ labels will not import confusion matrices at every threshold

Release 1.0.41

  • Add data-filter-split feature back to the ImageTrainingJob component

Release 1.0.40

  • Change base image to ubuntu OS
  • Set google-api-core<1.34.0 to avoid 900s timeout

Release 1.0.39

  • Fix AutoML Table pipeline failing on importing model evaluation metrics

Release 1.0.38

  • Fix default value issue in bigquery query API

Release 1.0.36

  • Cherrypick e358dee2f8d5c01580438ee54988f01fc3f16a7c and snap a new release

Release 1.0.35

  • Fix images for BQML components

Release 1.0.34

  • Cherrypick d1f1ee9f2bbd09df7ea6ab51b21f07ba5f86c871 and snap a new release

Release 1.0.33

  • Fix aiplatform & v1 batch predict job to work with KFP v2
  • Release Structured Data team's updated components and pipelines

Release 1.0.32

  • Support a HyperparameterTuningJobWithMetrics type to take execution_metrics path

Release 1.0.31

  • Fix aiplatform serialization
  • Release Structured Data team's updated components and pipelines
  • Add components for natural language: training TFHub model and preprocessing component for batch prediction

Release 1.0.30

  • Fix aiplatform & v1 batch predict job to work with KFP v2
  • Fix serialization for aiplatform components
  • Update Dataproc doc links
  • Update tags in Structured Data team's forecasting pipelines

Release 1.0.29

  • Propagate vertex system labels to the downstream resources
  • Release Structured Data team's updated components and pipelines
  • Fix Dataproc component doc to indicate that batch_id is optional
  • Simplify create_custom_training_job_op_from_component
  • Fix list and dict types for converted aiplatform components

Release 1.0.28

  • Support uploading for model versions for ModelUploadOp
  • Add text classification data processing component and training component
  • Propagates vertex system labels to the downstream resources for batch prediction job

Release 1.0.27

  • Add DataprocBatch resource to gcp_resources output parameter
  • Support serving default in bq export model job op

Release 1.0.26

  • Temporary fix for artifact types
  • Sync GCPC staging to prod to include AutoML model comparison and prophet pipelines
  • Update documentation for Eval components
  • Update HP tuning sample notebook
  • Improve folder structure for evaluation components
  • Model Evaluation, rename EvaluationDataSplitterOp to TargetFieldDataRemoverOp, rename ground_truth_column to target_field, rename class_names to class_labels, and remove key_columns input
  • Add model input to vertex ai model evaluation component

Release 1.0.25

  • Bigquery: Update public doc for evaluate model per customer feedback
  • Add Infra Validation remote runner
  • Add notification v1 doc to the v1 page
  • AutoML: Sync GCPC staging to prod to include bug fix for built-in algorithms

Release 1.0.24

  • Add notification v1 doc
  • Convert all v1 components into individual launchers and remote runners
  • Update AutoML Tables components to have latest SDK features
  • Add support for staging Dataflow options (sdk_location and extra_package)

Release 1.0.23

  • AutoML: Sync GCPC staging to prod to include recent API changes
  • TensorBoard: Make some input parameters optional to provide better user experience

Release 1.0.22

  • TensorBoard: Make some input parameters optional to provide better user experience

Release 1.0.21

  • Fix input parameter in tensorboard experiment creator component
  • Convert bigquery components into individual launchers and remote runners
  • Model Evaluation: Add metadata field for pipeline resource name

Release 1.0.20

  • Add special case in json_util.py where explanation_spec metadata outputs can have empty values
  • Update the docstring for missing arguments on feature_importance component
  • Create new tensorboard experiment creator component
  • Remove unused input in evaluation classification yaml
  • Update the docstring for exported_model_path in export_model

Release 1.0.19

  • Propagating labels for explain_forecast_model component
  • Model Evaluation - Add evaluation forecasting default of 0.5 for quantiles
  • Dataproc - Fix missing error payload from logging
  • Added BigQuery input support to evaluation components
  • Model Evaluation - Allow dataset paths list
  • Fix the docstring for ml_advanced_weights component
  • Fix the duplicated arguments in bigquery_ml_global_explain_job
  • Import importer from dsl namespace instead
  • Convert batch_prediction_job_remote_runner into individual launcher

Release 1.0.18

  • Model Evaluation - Give evaluation preprocessing components unique dataflow job names
  • Add vertex_notification_email component on v1 folder

Release 1.0.17

  • Model Evaluation - Rearrange json and yaml files in e2e test to eliminate duplicate defining and reading
  • Model Evaluation - Update JSON templates for evaluation
  • Model Evaluation - Split evaluation component into classification, forecasting, and regression evaluation & create artifact types for google.__Metrics
  • Model Evaluation - Match predictions input argument name to other Evaluation components
  • Model Evaluation - Update import_model_evaluation component to accept new google.___Metrics artifact types
  • Model Evaluation - Update regression and forecasting to contain ground truth input fields
  • Reverse re.findall order of arguments to (pattern, string) in job_remote_runner
  • Model Evaluation - Update evaluation container to v0.5 for data sampler and splitter preprocessing components

Release 1.0.16

  • Evaluation - Separate feature attribution from evaluation component to its own component
  • AutoML Tables - Include fix AMI issues for criteo dataset
  • AutoML Tables - Change Vertex evaluation pipeline templates
  • Model Evaluation - Import model evaluation slices when available in the metrics
  • Model Evaluation - Add nargs to allow for empty string input by component

Release 1.0.15

  • Sync AutomL components' code to GCPC codebase to reflect bug fix in FTE component spec
  • Auto-generate batch id if none is specified in Dataproc components
  • Add ground_truth_column input argument to data splitter component

Release 1.0.14

  • Temporarily pin apache_beam version to <2.34.0 due to apache/beam#22208.
  • Remove kms key name from the drop model interface.
  • Move new BQ components from experimental to v1
  • Fix the problem that AutoML Tabular pipeline could fail when using large number of features

Release 1.0.13

  • AutoML Tables - Fix AutoML Tabular pipeline always running evaluation.
  • AutoML Tables - Fix AutoML Tabular pipeline when there are a large set of input features.
  • Model Evaluation - Evaluation preprocessing component change output GCS artifact to JsonArray.

Release 1.0.12

  • Move generating feature ranking to utils to be available in SDK
  • Change JSON to primitive types for Tables v1, built-in algorithm and internal pipelines
  • AutoML Tables - update Tabular workflow to reference 1.0.10 launcher image
  • AutoML Tables - Add dataflow_service_account to specify custom service account to run dataflow jobs for stats_and_example_gen and transform components.
  • AutoML Tables - Update skip_architecture_search pipeline
  • AutoML Tables - Add algorithm to pipeline, also switch the default algorithm to be AMI
  • AutoML Tables - Use feature transform engine docker image for related components
  • AutoML Tables - Make calculation logic in SDK helper function run inside a component for Tables v1 and skip_architecture_search pipelines
  • AutoML Tables - weight_column_name -> weight_column and target_column_name -> target_column for Tables v1 and skip_architecture_search pipelines
  • AutoML Tables - For built-in algorithms, the transform_config input is expected to be a GCS file path.
  • AutoML Tables - Make generate analyze/transform data and split materialized data as components
  • AutoML Tables - Add automl_tabular_pipeline pipeline for Tabular Workflow.
  • AutoML Tables - Use FTE image directly to launch FTE component
  • Model Evaluation - Add display name to import model evaluation component
  • Model Evaluation - Update default number of workers.

Release 1.0.11

  • Add custom component to automl_tabular default pipeline
  • Add transformations_path to stats_and_example_gen and enable for v1 default pipeline and testing pipeline
  • Use 'unmanaged_container_model' instead of 'model' in infra validator component for automl tabular
  • Update evaluation component to v0.3

Release 1.0.10

  • Add new Evaluation components 'evaluation_data_sampler' and 'evaluation_data_splitter'
  • Make AutoML Tables ensemble also output explanation_metadata artifact
  • AutoML Tables - decouple transform config planner from metadata
  • AutoML Tables - Feature transform engine config planner to generate training schema & instance baseline

Release 1.0.9

  • FTE transform config passed as path to config file instead of directly as string to FTE
  • Support BigQuery ML weights job component
  • FTE now outputs training schema.
  • Support BigQuery ML reconstruction loss and trial info job components
  • Adding ML.TRAINING_INFO KFP and ML.EXPLAIN_PREDICT BQ Component.
  • Add additional experiments in distillation pipeline.
  • Support BigQuery ML advanced weights job component.
  • Support BigQuery drop model job components.
  • Support BigQuery ML centroids job components.
  • Wide and Deep and Tabnet models both now use the Feature Transform Engine pipeline instead of the Transform component.
  • Adding ML.CONFUSION_MATRIX KFP BQ Component.
  • Adding ML.FEATURE_INFO KFP BQ Component.
  • Merge distill_skip_evaluation and skip_evaluation pipelines with default pipeline using dsl.Condition
  • Adding ML.ROC_CURVE KFP BQ Component.
  • Adding ML.PRINCIPAL_COMPONENTS and ML.PRINCIPAL_COMPONENT_INFO KFP BQ component.
  • Adding ML.FEATURE_IMPORTANCE KFP BQ Component.
  • Add ML.ARIMA_COEFFICIENTS in component.yaml
  • Adding ML.Recommend KFP BQ component.
  • Add ML.ARIMA_EVALUATE in component.yaml
  • KFP component for ml.explain_forecast
  • KFP component for ml.forecast
  • Add distill + evaluation pipeline for Tables
  • Adding ML.GLOBAL_EXPLAIN KFP BQ Component.
  • KFP component for ml.detect_anomalies
  • Make stats-gen component to support running with example-gen only mode
  • Fix AutoML Tables pipeline and builtin pipelines on VPC-SC environment.
  • Preserve empty features in explanation_spec

Release 1.0.8

  • Use BigQuery batch queries in ARIMA pipeline after first 50 queries
  • Stats Gen and Feature Transform Engine pipeline integration.
  • Add window config to ARIMA pipeline
  • Removed default location setting from AutoML components and documentation.
  • Update default machine type to c2-standard-16 for built-in algorithms Custom and HyperparameterTuning Jobs
  • Use float instead of int max windows, which caused ARIMA pipeline failure
  • Renamed "Feature Transform Engine Transform Configuration" component to "Transform Configuration Planner" for clarity.
  • Preserve empty features in explanation_spec
  • Change json util to not remove empty primitives in a list.
  • Add model eval component to built-in algorithm default pipelines
  • Quick fix to Batch Prediction component input "bigquery_source_input_uri"

Release 1.0.7

  • Allow metrics and evaluated examples tables to be overwritten.
  • Replace custom copy_table component with BQ first-party query component.
  • Support vpc in feature selection.
  • Add import eval metrics to model to AutoML Tables default pipeline.
  • Add default Wide & Deep study_spec_parameters configs and add helper function to utils.py to get parameters.

Release 1.0.6

  • Update import evaluation metrics component.
  • Support parameterized input for reserved_ip_range and other Vertex Training parameters in custom job utility.
  • Generate feature selection tuning pipeline and test utils.
  • Add retries to queries hitting BQ write quota on BQML Arima pipeline.
  • Minor changes to the feature transform engine and transform configuration component specs to support their integration.
  • Update Executor component for Pipeline to support kernel_spec.
  • Add default TabNet study_spec_parameters_override configs for different dataset sizes and search space modes and helper function to get the parameters.

Release 1.0.5

  • Add VPC-SC and CMEK support for the experimental evaluation component
  • Add an import evaluation metrics component
  • Modify AutoML Tables template JSON pipeline specs
  • Add feature transform engine AutoML Table component.

Release 1.0.4

  • Create alias for create_custom_training_job_op_from_component as create_custom_training_job_from_component
  • Add support for env variables in Custom_Job component.

Release 1.0.3

  • Add API docs for Vertex Notification Email
  • Add template JSON pipeline spec for running evaluation on a managed GCP Vertex model.
  • Update documentation for Dataproc Serverless components v1.0.
  • Use if:cond:then when specifying image name in built-in algorithm hyperparameter tuning job component and add separate hyperparameter tuning job default pipelines for TabNet and Wide & Deep
  • Add gcp_resources in the eval component output
  • Add downsampled_test_split_json to example_and_stats_gen component.

Release 1.0.2

  • Dataproc Serverless components v1.0 launch.
  • Bump google-cloud-aiplatform version
  • Fix HP Tuning documentation, fixes #7460
  • Use feature ranking and selected features in AutoML Tables stage 1 tuning component.
  • Update distill_skip_evaluation_pipeline for performance improvement.

Release 1.0.1

  • Add experimental email notification component
  • add docs for create_custom_training_job_op_from_component
  • Remove ForecastingTrainingWithExperimentsOp component.
  • Use unmanaged_container_model for model_upload for AutoML Tables pipelines
  • add nfs mount support for create_custom_training_job_op_from_component
  • Implement cancellation for dataproc components
  • bump google-api-core version to 2.0+
  • Add retry for batch prediction component

Release 1.0.0

  • add enable_web_access for create_custom_training_job_op_from_component
  • remove remove training_filter_split, validation_filter_split, test_filter_split from automl components
  • Update the dataproc component docs

Release 0.3.1

  • Implement cancellation propagation
  • Remove encryption key in input for BQ create model
  • Add Dataproc Batch components
  • Add AutoML Tables Wide & Deep trainer component and pipeline
  • Create GCPC v1 and readthedocs for v1
  • Fix bug when ExplanationMetadata.InputMetadata field is provided the batch prediction job component

Release 0.3.0

  • Update BQML export model input from string to artifact
  • Move model/endpoint/job/bqml compoennts to 1.0 namespace
  • Expose enable_web_access and reserved_ip_ranges for custom job component
  • Add delete model and undeploy model components
  • Add utility library for google artifacts

Release 0.2.2

  • Fixes for BQML components
  • Add util functions for HP tuning components and update samples

Release 0.2.1

  • Add BigqueryQueryJobOp, BigqueryCreateModelJobOp, BigqueryExportModelJobOp and BigqueryPredictModelJobOp components
  • Add ModelEvaluationOp component
  • Accept UnmanagedContainerModel artifact in Batch Prediction component
  • Add util components and fix YAML for HP Tuning Job component; delete lightweight python version
  • Add generic custom training job component
  • Fix Dataflow error log reporting and component sample

Release 0.2.0

  • Update custom job name to create_custom_training_job_op_from_component
  • Remove special handling for "=" in remote runner.
  • Bug fixes and documentation updates.

Release 0.1.9

  • Dataflow and wait components
  • Bug fixes

Release 0.1.8

  • Update the CustomJob component interface, and rename to custom_training_job_op
  • Define new artifact types for Google Cloud resources.
  • Update the AI Platform components. Added the component YAML and uses the new Google artifact types
  • Add Vertex notebook component
  • Various doc updates

Release 0.1.7

  • Add support for labels in custom_job wrapper.
  • Add a component that connects the forecasting preprocessing and training components.
  • Write GCP_RESOURCE proto for the custom_job output.
  • Expose Custom Job parameters Service Account, Network and CMEK via Custom Job wrapper.
  • Increase KFP min version dependency.
  • AUpdate documentations for GCPC components.
  • Update typing checks to include Python3.6 deprecated types.

Release 0.1.6

  • Experimental component for Model Forecast.
  • Fixed issue with parameter passing for Vertex AI components
  • Simplify auto generated API docs
  • Fix parameter passing for explainability on ModelUploadOp
  • Update naming of project and location parameters for all for GCPC components

Release 0.1.5

  • Experimental component for vertex forecasting preprocessing and validation

Release 0.1.4

  • Experimental component for tfp_anomaly_detection.
  • Experimental module for Custom Job Wrapper.
  • Fix to include YAML files in PyPI package.
  • Restructure the google_cloud_pipeline_components.

Release 0.1.3

  • Use correct dataset type when passing dataset to CustomTraining.
  • Bump google-cloud-aiplatform to 1.1.1.

Release 0.1.2

  • Add components for AutoMLForecasting.
  • Update API documentation.

Release 0.1.1

  • Fix issue with latest version of KFP not accepting pipeline_root in kfp.compile.
  • Fix Compatibility with latest AI Platform name change to replace resource name class with Vertex AI

Release 0.1.0

First release

  • Initial release of the Python SDK with data and model managemnet operations for Image, Text, Tabular, and Video Data.