Skip to content
Snippets Groups Projects
Commit 3708d13f authored by Nong Li's avatar Nong Li Committed by Davies Liu
Browse files

[SPARK-12992] [SQL] Support vectorized decoding in UnsafeRowParquetRecordReader.

WIP: running tests. Code needs a bit of clean up.

This patch completes the vectorized decoding with the goal of passing the existing
tests. There is still more patches to support the rest of the format spec, even
just for flat schemas.

This patch adds a new flag to enable the vectorized decoding. Tests were updated
to try with both modes where applicable.

Once this is working well, we can remove the previous code path.

Author: Nong Li <nong@databricks.com>

Closes #11055 from nongli/spark-12992-2.
parent eeaf45b9
No related branches found
No related tags found
No related merge requests found
Showing
with 549 additions and 90 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment