Skip to content
Snippets Groups Projects
Commit 688b6ef9 authored by Wenchen Fan's avatar Wenchen Fan Committed by Reynold Xin
Browse files

[SPARK-15932][SQL][DOC] document the contract of encoder serializer expressions

## What changes were proposed in this pull request?

In our encoder framework, we imply that serializer expressions should use `BoundReference` to refer to the input object, and a lot of codes depend on this contract(e.g. ExpressionEncoder.tuple).  This PR adds some document and assert in `ExpressionEncoder` to make it clearer.

## How was this patch tested?

existing tests

Author: Wenchen Fan <wenchen@databricks.com>

Closes #13648 from cloud-fan/comment.
parent 1842cdd4
No related branches found
No related tags found
No related merge requests found
...@@ -197,6 +197,15 @@ case class ExpressionEncoder[T]( ...@@ -197,6 +197,15 @@ case class ExpressionEncoder[T](
if (flat) require(serializer.size == 1) if (flat) require(serializer.size == 1)
// serializer expressions are used to encode an object to a row, while the object is usually an
// intermediate value produced inside an operator, not from the output of the child operator. This
// is quite different from normal expressions, and `AttributeReference` doesn't work here
// (intermediate value is not an attribute). We assume that all serializer expressions use a same
// `BoundReference` to refer to the object, and throw exception if they don't.
assert(serializer.forall(_.references.isEmpty), "serializer cannot reference to any attributes.")
assert(serializer.flatMap(_.collect { case b: BoundReference => b}).distinct.length <= 1,
"all serializer expressions must use the same BoundReference.")
/** /**
* Returns a new copy of this encoder, where the `deserializer` is resolved and bound to the * Returns a new copy of this encoder, where the `deserializer` is resolved and bound to the
* given schema. * given schema.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment