Skip to content
Snippets Groups Projects
Commit 21eadd1d authored by gatorsmile's avatar gatorsmile Committed by Wenchen Fan
Browse files

[SPARK-16229][SQL] Drop Empty Table After CREATE TABLE AS SELECT fails

#### What changes were proposed in this pull request?
In `CREATE TABLE AS SELECT`, if the `SELECT` query failed, the table should not exist. For example,

```SQL
CREATE TABLE tab
STORED AS TEXTFILE
SELECT 1 AS a, (SELECT a FROM (SELECT 1 AS a UNION ALL SELECT 2 AS a) t) AS b
```
The above query failed as expected but an empty table `t` is created.

This PR is to drop the created table when hitting any non-fatal exception.

#### How was this patch tested?
Added a test case to verify the behavior

Author: gatorsmile <gatorsmile@gmail.com>

Closes #13926 from gatorsmile/dropTableAfterException.
parent 909c6d81
No related branches found
No related tags found
No related merge requests found
......@@ -17,6 +17,8 @@
package org.apache.spark.sql.hive.execution
import scala.util.control.NonFatal
import org.apache.spark.sql.{AnalysisException, Row, SparkSession}
import org.apache.spark.sql.catalyst.catalog.{CatalogColumn, CatalogTable}
import org.apache.spark.sql.catalyst.plans.logical.{InsertIntoTable, LogicalPlan}
......@@ -87,8 +89,15 @@ case class CreateHiveTableAsSelectCommand(
throw new AnalysisException(s"$tableIdentifier already exists.")
}
} else {
sparkSession.sessionState.executePlan(InsertIntoTable(
metastoreRelation, Map(), query, overwrite = true, ifNotExists = false)).toRdd
try {
sparkSession.sessionState.executePlan(InsertIntoTable(
metastoreRelation, Map(), query, overwrite = true, ifNotExists = false)).toRdd
} catch {
case NonFatal(e) =>
// drop the created table.
sparkSession.sessionState.catalog.dropTable(tableIdentifier, ignoreIfNotExists = true)
throw e
}
}
Seq.empty[Row]
......
......@@ -554,6 +554,21 @@ class HiveDDLSuite
}
}
test("Create Cataloged Table As Select - Drop Table After Runtime Exception") {
withTable("tab") {
intercept[RuntimeException] {
sql(
"""
|CREATE TABLE tab
|STORED AS TEXTFILE
|SELECT 1 AS a, (SELECT a FROM (SELECT 1 AS a UNION ALL SELECT 2 AS a) t) AS b
""".stripMargin)
}
// After hitting runtime exception, we should drop the created table.
assert(!spark.sessionState.catalog.tableExists(TableIdentifier("tab")))
}
}
test("desc table for data source table") {
withTable("tab1") {
val tabName = "tab1"
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment