[SPARK-21312][SQL] correct offsetInBytes in UnsafeRow.writeToStream
## What changes were proposed in this pull request? Corrects offsetInBytes calculation in UnsafeRow.writeToStream. Known failures include writes to some DataSources that have own SparkPlan implementations and cause EXCHANGE in writes. ## How was this patch tested? Extended UnsafeRowSuite.writeToStream to include an UnsafeRow over byte array having non-zero offset. Author: Sumedh Wale <swale@snappydata.io> Closes #18535 from sumwale/SPARK-21312. (cherry picked from commit 14a3bb3a) Signed-off-by:Wenchen Fan <wenchen@databricks.com>
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 1 addition, 1 deletion.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/core/src/test/scala/org/apache/spark/sql/UnsafeRowSuite.scala 13 additions, 0 deletions.../src/test/scala/org/apache/spark/sql/UnsafeRowSuite.scala
Loading
Please register or sign in to comment