Skip to content
Snippets Groups Projects
Commit de3f87fa authored by Shixiong Zhu's avatar Shixiong Zhu
Browse files

[SPARK-18030][TESTS] Fix flaky FileStreamSourceSuite by not deleting the files

## What changes were proposed in this pull request?

The test `when schema inference is turned on, should read partition data` should not delete files because the source maybe is listing files. This PR just removes the delete actions since they are not necessary.

## How was this patch tested?

Jenkins

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #15699 from zsxwing/SPARK-18030.
parent 8bfc3b7a
No related branches found
No related tags found
No related merge requests found
......@@ -102,12 +102,6 @@ class FileStreamSourceTest extends StreamTest with SharedSQLContext with Private
}
}
case class DeleteFile(file: File) extends ExternalAction {
def runAction(): Unit = {
Utils.deleteRecursively(file)
}
}
/** Use `format` and `path` to create FileStreamSource via DataFrameReader */
def createFileStream(
format: String,
......@@ -697,10 +691,6 @@ class FileStreamSourceSuite extends FileStreamSourceTest {
AddTextFileData("{'value': 'keep5'}", partitionBarSubDir, tmp),
CheckAnswer(("keep2", "foo"), ("keep3", "foo"), ("keep4", "bar"), ("keep5", "bar")),
// Delete the two partition dirs
DeleteFile(partitionFooSubDir),
DeleteFile(partitionBarSubDir),
AddTextFileData("{'value': 'keep6'}", partitionBarSubDir, tmp),
CheckAnswer(("keep2", "foo"), ("keep3", "foo"), ("keep4", "bar"), ("keep5", "bar"),
("keep6", "bar"))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment