-
- Downloads
[SPARK-17822][R] Make JVMObjectTracker a member variable of RBackend
## What changes were proposed in this pull request? * This PR changes `JVMObjectTracker` from `object` to `class` and let its instance associated with each RBackend. So we can manage the lifecycle of JVM objects when there are multiple `RBackend` sessions. `RBackend.close` will clear the object tracker explicitly. * I assume that `SQLUtils` and `RRunner` do not need to track JVM instances, which could be wrong. * Small refactor of `SerDe.sqlSerDe` to increase readability. ## How was this patch tested? * Added unit tests for `JVMObjectTracker`. * Wait for Jenkins to run full tests. Author: Xiangrui Meng <meng@databricks.com> Closes #16154 from mengxr/SPARK-17822.
Showing
- core/src/main/scala/org/apache/spark/api/r/JVMObjectTracker.scala 87 additions, 0 deletions.../main/scala/org/apache/spark/api/r/JVMObjectTracker.scala
- core/src/main/scala/org/apache/spark/api/r/RBackend.scala 5 additions, 1 deletioncore/src/main/scala/org/apache/spark/api/r/RBackend.scala
- core/src/main/scala/org/apache/spark/api/r/RBackendHandler.scala 8 additions, 46 deletions...c/main/scala/org/apache/spark/api/r/RBackendHandler.scala
- core/src/main/scala/org/apache/spark/api/r/RRunner.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/api/r/RRunner.scala
- core/src/main/scala/org/apache/spark/api/r/SerDe.scala 54 additions, 38 deletionscore/src/main/scala/org/apache/spark/api/r/SerDe.scala
- core/src/test/scala/org/apache/spark/api/r/JVMObjectTrackerSuite.scala 73 additions, 0 deletions.../scala/org/apache/spark/api/r/JVMObjectTrackerSuite.scala
- core/src/test/scala/org/apache/spark/api/r/RBackendSuite.scala 31 additions, 0 deletions...src/test/scala/org/apache/spark/api/r/RBackendSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala 6 additions, 6 deletions.../src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
Loading
Please register or sign in to comment