-
- Downloads
[SPARK-16621][SQL] Generate stable SQLs in SQLBuilder
## What changes were proposed in this pull request? Currently, the generated SQLs have not-stable IDs for generated attributes. The stable generated SQL will give more benefit for understanding or testing the queries. This PR provides stable SQL generation by the followings. - Provide unique ids for generated subqueries, `gen_subquery_xxx`. - Provide unique and stable ids for generated attributes, `gen_attr_xxx`. **Before** ```scala scala> new org.apache.spark.sql.catalyst.SQLBuilder(sql("select 1")).toSQL res0: String = SELECT `gen_attr_0` AS `1` FROM (SELECT 1 AS `gen_attr_0`) AS gen_subquery_0 scala> new org.apache.spark.sql.catalyst.SQLBuilder(sql("select 1")).toSQL res1: String = SELECT `gen_attr_4` AS `1` FROM (SELECT 1 AS `gen_attr_4`) AS gen_subquery_0 ``` **After** ```scala scala> new org.apache.spark.sql.catalyst.SQLBuilder(sql("select 1")).toSQL res1: String = SELECT `gen_attr_0` AS `1` FROM (SELECT 1 AS `gen_attr_0`) AS gen_subquery_0 scala> new org.apache.spark.sql.catalyst.SQLBuilder(sql("select 1")).toSQL res2: String = SELECT `gen_attr_0` AS `1` FROM (SELECT 1 AS `gen_attr_0`) AS gen_subquery_0 ``` ## How was this patch tested? Pass the existing Jenkins tests. Author: Dongjoon Hyun <dongjoon@apache.org> Closes #14257 from dongjoon-hyun/SPARK-16621.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala 18 additions, 5 deletions...main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala
- sql/hive/src/test/resources/sqlgen/agg1.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/agg1.sql
- sql/hive/src/test/resources/sqlgen/agg2.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/agg2.sql
- sql/hive/src/test/resources/sqlgen/agg3.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/agg3.sql
- sql/hive/src/test/resources/sqlgen/aggregate_functions_and_window.sql 1 addition, 1 deletion.../test/resources/sqlgen/aggregate_functions_and_window.sql
- sql/hive/src/test/resources/sqlgen/case.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/case.sql
- sql/hive/src/test/resources/sqlgen/case_with_else.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/case_with_else.sql
- sql/hive/src/test/resources/sqlgen/case_with_key.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/case_with_key.sql
- sql/hive/src/test/resources/sqlgen/case_with_key_and_else.sql 1 addition, 1 deletion...hive/src/test/resources/sqlgen/case_with_key_and_else.sql
- sql/hive/src/test/resources/sqlgen/cluster_by.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/cluster_by.sql
- sql/hive/src/test/resources/sqlgen/data_source_json_parquet_t0.sql 1 addition, 1 deletion...src/test/resources/sqlgen/data_source_json_parquet_t0.sql
- sql/hive/src/test/resources/sqlgen/data_source_orc_parquet_t0.sql 1 addition, 1 deletion.../src/test/resources/sqlgen/data_source_orc_parquet_t0.sql
- sql/hive/src/test/resources/sqlgen/data_source_parquet_parquet_t0.sql 1 addition, 1 deletion.../test/resources/sqlgen/data_source_parquet_parquet_t0.sql
- sql/hive/src/test/resources/sqlgen/distinct_aggregation.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/distinct_aggregation.sql
- sql/hive/src/test/resources/sqlgen/distribute_by.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/distribute_by.sql
- sql/hive/src/test/resources/sqlgen/distribute_by_with_sort_by.sql 1 addition, 1 deletion.../src/test/resources/sqlgen/distribute_by_with_sort_by.sql
- sql/hive/src/test/resources/sqlgen/except.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/except.sql
- sql/hive/src/test/resources/sqlgen/filter_after_subquery.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/filter_after_subquery.sql
- sql/hive/src/test/resources/sqlgen/generate_with_other_1.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/generate_with_other_1.sql
- sql/hive/src/test/resources/sqlgen/generate_with_other_2.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/generate_with_other_2.sql
Loading
Please register or sign in to comment