-
- Downloads
[SPARK-5128][MLLib] Add common used log1pExp API in MLUtils
When `x` is positive and large, computing `math.log(1 + math.exp(x))` will lead to arithmetic overflow. This will happen when `x > 709.78` which is not a very large number. It can be addressed by rewriting the formula into `x + math.log1p(math.exp(-x))` when `x > 0`. Author: DB Tsai <dbtsai@alpinenow.com> Closes #3915 from dbtsai/mathutil and squashes the following commits: bec6a84 [DB Tsai] remove empty line 3239541 [DB Tsai] revert part of patch into another PR 23144f3 [DB Tsai] doc 49f3658 [DB Tsai] temp 6c29ed3 [DB Tsai] formating f8447f9 [DB Tsai] address another overflow issue in gradientMultiplier in LOR gradient code 64eefd0 [DB Tsai] first commit
Showing
- mllib/src/main/scala/org/apache/spark/mllib/optimization/Gradient.scala 8 additions, 11 deletions.../scala/org/apache/spark/mllib/optimization/Gradient.scala
- mllib/src/main/scala/org/apache/spark/mllib/tree/loss/LogLoss.scala 3 additions, 7 deletions...main/scala/org/apache/spark/mllib/tree/loss/LogLoss.scala
- mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala 16 additions, 0 deletions.../src/main/scala/org/apache/spark/mllib/util/MLUtils.scala
- mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala 10 additions, 3 deletions...test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala
Loading
Please register or sign in to comment