diff --git a/docs/docusaurus/docs/cloud/expectations/manage_expectations.md b/docs/docusaurus/docs/cloud/expectations/manage_expectations.md index af655684c2c3..c6b2c68901d5 100644 --- a/docs/docusaurus/docs/cloud/expectations/manage_expectations.md +++ b/docs/docusaurus/docs/cloud/expectations/manage_expectations.md @@ -8,12 +8,6 @@ An Expectation is a verifiable assertion about your data. They make implicit ass -:::info Custom SQL Query Expectations - -To create custom SQL query Expectations, you'll need to use the GX API. See [Customize Expectations](/core/customize_expectations/customize_expectations.md). - -::: - ## Prerequisites - You have a [Data Asset](/cloud/data_assets/manage_data_assets.md#create-a-data-asset). @@ -23,7 +17,7 @@ To create custom SQL query Expectations, you'll need to use the GX API. See [Cus The following table lists the available GX Cloud Expectations. | Data Quality Issue | Expectation | Description | -| ------------------ | --------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +|--------------------|-----------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------| | Cardinality | `expect_column_values_to_be_unique` | Expect each column value to be unique. | | Cardinality | `expect_compound_columns_to_be_unique` | Expect the compound columns to be unique. | | Cardinality | `expect_select_column_values_to_be_unique_within_record` | Expect the values for each record to be unique across the columns listed. Note that records can be duplicated. | @@ -69,6 +63,18 @@ The following table lists the available GX Cloud Expectations. | Volume | `expect_table_row_count_to_equal` | Expect the number of rows to equal a value. | | Volume | `expect_table_row_count_to_equal_other_table` | Expect the number of rows to equal the number in another table within the same database. | +## Custom SQL Expectations + +GX Cloud also offers the ability to write a custom Expectation using SQL. It is designed to fail validation if the provided SQL query returns one or more rows. + +The provided query should be written in the dialect of the Data Source in which a given Data Asset lives. + +:::info Optional `{batch}` named query + +The optional `{batch}` named query references the Batch of data under test. When the Expectation is evaluated, the `{batch}` named query will be replaced with the Batch of data that is validated. + +::: + ## Add an Expectation 1. In GX Cloud, click **Data Assets**. diff --git a/docs/docusaurus/docs/core/customize_expectations/use_sql_to_define_a_custom_expectation.md b/docs/docusaurus/docs/core/customize_expectations/use_sql_to_define_a_custom_expectation.md index 8917bf4d6b6e..91028741bb14 100644 --- a/docs/docusaurus/docs/core/customize_expectations/use_sql_to_define_a_custom_expectation.md +++ b/docs/docusaurus/docs/core/customize_expectations/use_sql_to_define_a_custom_expectation.md @@ -11,7 +11,7 @@ import PrereqPreconfiguredDataSourceAndAsset from '../_core_components/prerequis Among the available Expectations, the `UnexpectedRowsExpectation` is designed to facilitate the execution of SQL or Spark-SQL queries as the core logic for an Expectation. By default, `UnexpectedRowsExpectation` considers validation successful when no rows are returned by the provided SQL query. -You customize an `UnexpectedRowsExpectation` in essentially the same manner as you would [define a custom Expectation](/core/customize_expectations/define_a_custom_expectation_class.md), by subclassing `UnexpectedRowsExpectation` and providing customized default attributes and text for Data Docs. However, there are some caveats around the `UnexpectedRowsExpectation`'s `unexpected_rows_query` attribute that deserve further detail. +Like any other Expectation, you can instantiate the `UnexpectedRowsExpectation` directly. You can also customize an `UnexpectedRowsExpectation` in essentially the same manner as you would [define a custom Expectation](/core/customize_expectations/define_a_custom_expectation_class.md), by subclassing `UnexpectedRowsExpectation` and providing customized default attributes and text for Data Docs. However, there are some caveats around the `UnexpectedRowsExpectation`'s `unexpected_rows_query` attribute that deserve further detail. @@ -48,7 +48,7 @@ You customize an `UnexpectedRowsExpectation` in essentially the same manner as y The `unexpected_rows_query` attribute is a SQL or Spark-SQL query that returns a selection of rows from the Batch of data being validated. By default, rows that are returned have failed the validation check. - Although the `unexpected_rows_query` should be written in standard SQL or Spark-SQL syntax, it must also contain the special `{batch}` placeholder. When the Expectation is evaluated, the `{batch}` placeholder will be replaced with the Batch of data that is validated. + The `unexpected_rows_query` should be written in standard SQL or Spark-SQL syntax, except that it can also contain the special `{batch}` named query. When the Expectation is evaluated, the `{batch}` keyword will be replaced with the Batch of data that is configured for your Data Asset. In this example, `unexpected_rows_query` will select any rows where the passenger count is greater than `6` or less than `0`. These rows will fail validation for this Expectation: diff --git a/docs/docusaurus/sidebars.js b/docs/docusaurus/sidebars.js index 4a9acfd1bb73..1f6a1a709386 100644 --- a/docs/docusaurus/sidebars.js +++ b/docs/docusaurus/sidebars.js @@ -252,6 +252,11 @@ module.exports = { label: 'Available Expectations', href: '/docs/cloud/expectations/manage_expectations#available-expectations', }, + { + type: 'link', + label: 'Custom SQL Expectations', + href: '/docs/cloud/expectations/manage_expectations#custom-sql-expectations', + }, { type: 'link', label: 'Add an Expectation',