Skip to content
Snippets Groups Projects
Commit 76e20a0a authored by Xiangrui Meng's avatar Xiangrui Meng
Browse files

[SPARK-6141][MLlib] Upgrade Breeze from 0.10 to 0.11 to fix convergence bug

LBFGS and OWLQN in Breeze 0.10 has convergence check bug.
This is fixed in 0.11, see the description in Breeze project for detail:

https://github.com/scalanlp/breeze/pull/373#issuecomment-76879760

Author: Xiangrui Meng <meng@databricks.com>
Author: DB Tsai <dbtsai@alpinenow.com>
Author: DB Tsai <dbtsai@dbtsai.com>

Closes #4879 from dbtsai/breeze and squashes the following commits:

d848f65 [DB Tsai] Merge pull request #1 from mengxr/AlpineNow-breeze
c2ca6ac [Xiangrui Meng] upgrade to breeze-0.11.1
35c2f26 [Xiangrui Meng] fix LRSuite
397a208 [DB Tsai] upgrade breeze
parent d334bfbc
No related branches found
No related tags found
No related merge requests found
......@@ -63,7 +63,7 @@
<dependency>
<groupId>org.scalanlp</groupId>
<artifactId>breeze_${scala.binary.version}</artifactId>
<version>0.10</version>
<version>0.11.1</version>
<exclusions>
<!-- This is included as a compile-scoped dependency by jtransforms, which is
a dependency of breeze. -->
......
......@@ -372,8 +372,12 @@ class LogisticRegressionSuite extends FunSuite with MLlibTestSparkContext with M
testRDD2.cache()
testRDD3.cache()
val numIteration = 10
val lrA = new LogisticRegressionWithLBFGS().setIntercept(true)
lrA.optimizer.setNumIterations(numIteration)
val lrB = new LogisticRegressionWithLBFGS().setIntercept(true).setFeatureScaling(false)
lrB.optimizer.setNumIterations(numIteration)
val modelA1 = lrA.run(testRDD1, initialWeights)
val modelA2 = lrA.run(testRDD2, initialWeights)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment