Skip to content
Snippets Groups Projects
Commit bdd53716 authored by gatorsmile's avatar gatorsmile Committed by Wenchen Fan
Browse files

[SPARK-16959][SQL] Rebuild Table Comment when Retrieving Metadata from Hive Metastore

### What changes were proposed in this pull request?
The `comment` in `CatalogTable` returned from Hive is always empty. We store it in the table property when creating a table. However, when we try to retrieve the table metadata from Hive metastore, we do not rebuild it. The `comment` is always empty.

This PR is to fix the issue.

### How was this patch tested?
Fixed the test case to verify the change.

Author: gatorsmile <gatorsmile@gmail.com>

Closes #14550 from gatorsmile/tableComment.
parent 1203c841
No related branches found
No related tags found
No related merge requests found
......@@ -413,7 +413,8 @@ private[hive] class HiveClientImpl(
properties = Option(h.getTTable.getSd.getSerdeInfo.getParameters)
.map(_.asScala.toMap).orNull
),
properties = properties,
properties = properties.filter(kv => kv._1 != "comment"),
comment = properties.get("comment"),
viewOriginalText = Option(h.getViewOriginalText),
viewText = Option(h.getViewExpandedText),
unsupportedFeatures = unsupportedFeatures)
......
......@@ -135,8 +135,11 @@ class HiveDDLSuite
sql(s"CREATE VIEW $viewName COMMENT 'no comment' AS SELECT * FROM $tabName")
val tableMetadata = catalog.getTableMetadata(TableIdentifier(tabName, Some("default")))
val viewMetadata = catalog.getTableMetadata(TableIdentifier(viewName, Some("default")))
assert(tableMetadata.properties.get("comment") == Option("BLABLA"))
assert(viewMetadata.properties.get("comment") == Option("no comment"))
assert(tableMetadata.comment == Option("BLABLA"))
assert(viewMetadata.comment == Option("no comment"))
// Ensure that `comment` is removed from the table property
assert(tableMetadata.properties.get("comment").isEmpty)
assert(viewMetadata.properties.get("comment").isEmpty)
}
}
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment