Skip to content
Snippets Groups Projects
Commit a7f8a4f5 authored by Michael Armbrust's avatar Michael Armbrust
Browse files

Revert [SPARK-3011][SQL] _temporary directory should be filtered out by sqlContext.parquetFile

Reverts #1924 due to build failures with hadoop 0.23.

Author: Michael Armbrust <michael@databricks.com>

Closes #1949 from marmbrus/revert1924 and squashes the following commits:

6bff940 [Michael Armbrust] Revert "[SPARK-3011][SQL] _temporary directory should be filtered out by sqlContext.parquetFile"
parent 96221067
No related branches found
No related tags found
No related merge requests found
......@@ -378,8 +378,7 @@ private[parquet] object ParquetTypesConverter extends Logging {
val children = fs.listStatus(path).filterNot { status =>
val name = status.getPath.getName
name(0) == '.' || name == FileOutputCommitter.SUCCEEDED_FILE_NAME ||
name == FileOutputCommitter.TEMP_DIR_NAME
name(0) == '.' || name == FileOutputCommitter.SUCCEEDED_FILE_NAME
}
// NOTE (lian): Parquet "_metadata" file can be very slow if the file consists of lots of row
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment