Skip to content
Snippets Groups Projects
Commit a71cbbde authored by Cheng Lian's avatar Cheng Lian
Browse files

[SPARK-8458] [SQL] Don't strip scheme part of output path when writing ORC files

`Path.toUri.getPath` strips scheme part of output path (from `file:///foo` to `/foo`), which causes ORC data source only writes to the file system configured in Hadoop configuration. Should use `Path.toString` instead.

Author: Cheng Lian <lian@databricks.com>

Closes #6892 from liancheng/spark-8458 and squashes the following commits:

87f8199 [Cheng Lian] Don't strip scheme of output path when writing ORC files
parent 3eaed876
No related branches found
No related tags found
No related merge requests found
......@@ -111,7 +111,7 @@ private[orc] class OrcOutputWriter(
new OrcOutputFormat().getRecordWriter(
new Path(path, filename).getFileSystem(conf),
conf.asInstanceOf[JobConf],
new Path(path, filename).toUri.getPath,
new Path(path, filename).toString,
Reporter.NULL
).asInstanceOf[RecordWriter[NullWritable, Writable]]
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment