-
- Downloads
[SPARK-17456][CORE] Utility for parsing Spark versions
## What changes were proposed in this pull request? This patch adds methods for extracting major and minor versions as Int types in Scala from a Spark version string. Motivation: There are many hacks within Spark's codebase to identify and compare Spark versions. We should add a simple utility to standardize these code paths, especially since there have been mistakes made in the past. This will let us add unit tests as well. Currently, I want this functionality to check Spark versions to provide backwards compatibility for ML model persistence. ## How was this patch tested? Unit tests Author: Joseph K. Bradley <joseph@databricks.com> Closes #15017 from jkbradley/version-parsing.
Showing
- core/src/main/scala/org/apache/spark/util/VersionUtils.scala 52 additions, 0 deletionscore/src/main/scala/org/apache/spark/util/VersionUtils.scala
- core/src/test/scala/org/apache/spark/util/VersionUtilsSuite.scala 76 additions, 0 deletions.../test/scala/org/apache/spark/util/VersionUtilsSuite.scala
Loading
Please register or sign in to comment