Cheng Lian
authored
This PR adds a `private[sql]` method `metadata` to `SparkPlan`, which can be used to describe detail information about a physical plan during visualization. Specifically, this PR uses this method to provide details of `PhysicalRDD`s translated from a data source relation. For example, a `ParquetRelation` converted from Hive metastore table `default.psrc` is now shown as the following screenshot:  And here is the screenshot for a regular `ParquetRelation` (not converted from Hive metastore table) loaded from a really long path:  Author: Cheng Lian <lian@databricks.com> Closes #10004 from liancheng/spark-12012.physical-rdd-metadata.
Name | Last commit | Last update |
---|---|---|
.. | ||
__init__.py | ||
column.py | ||
context.py | ||
dataframe.py | ||
functions.py | ||
group.py | ||
readwriter.py | ||
tests.py | ||
types.py | ||
utils.py | ||
window.py |