-
- Downloads
[SPARK-13137][SQL] NullPoingException in schema inference for CSV when the first line is empty
https://issues.apache.org/jira/browse/SPARK-13137 This PR adds a filter in schema inference so that it does not emit NullPointException. Also, I removed `MAX_COMMENT_LINES_IN_HEADER `but instead used a monad chaining with `filter()` and `first()`. Lastly, I simply added a newline rather than adding a new file for this so that this is covered with the original tests. Author: hyukjinkwon <gurwls223@gmail.com> Closes #11023 from HyukjinKwon/SPARK-13137.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala 0 additions, 3 deletions...ache/spark/sql/execution/datasources/csv/CSVOptions.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVRelation.scala 7 additions, 5 deletions...che/spark/sql/execution/datasources/csv/CSVRelation.scala
- sql/core/src/test/resources/cars.csv 1 addition, 0 deletionssql/core/src/test/resources/cars.csv
Loading
Please register or sign in to comment