Skip to content
Snippets Groups Projects
Commit 20fd35df authored by Yash Datta's avatar Yash Datta Committed by Yin Huai
Browse files

[SPARK-10451] [SQL] Prevent unnecessary serializations in InMemoryColumnarTableScan 

Many of the fields in InMemoryColumnar scan and InMemoryRelation can be made transient.

This  reduces my 1000ms job to abt 700 ms . The task size reduces from 2.8 mb to ~1300kb

Author: Yash Datta <Yash.Datta@guavus.com>

Closes #8604 from saucam/serde.
parent e3b5d6cb
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment