Skip to content
Snippets Groups Projects
Commit 47a2940d authored by Zheng RuiFeng's avatar Zheng RuiFeng Committed by Xiangrui Meng
Browse files

[SPARK-15398][ML] Update the warning message to recommend ML usage

## What changes were proposed in this pull request?
MLlib are not recommended to use, and some methods are even deprecated.
Update the warning message to recommend ML usage.
```
  def showWarning() {
    System.err.println(
      """WARN: This is a naive implementation of Logistic Regression and is given as an example!
        |Please use either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or
        |org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS
        |for more conventional use.
      """.stripMargin)
  }
```
To
```
  def showWarning() {
    System.err.println(
      """WARN: This is a naive implementation of Logistic Regression and is given as an example!
        |Please use org.apache.spark.ml.classification.LogisticRegression
        |for more conventional use.
      """.stripMargin)
  }
```

## How was this patch tested?
local build

Author: Zheng RuiFeng <ruifengz@foxmail.com>

Closes #13190 from zhengruifeng/update_recd.
parent 4c7a6b38
No related branches found
No related tags found
No related merge requests found
Showing
with 28 additions and 37 deletions
...@@ -32,8 +32,7 @@ import java.util.regex.Pattern; ...@@ -32,8 +32,7 @@ import java.util.regex.Pattern;
* Logistic regression based classification. * Logistic regression based classification.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or * please refer to org.apache.spark.ml.classification.LogisticRegression.
* org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS based on your needs.
*/ */
public final class JavaHdfsLR { public final class JavaHdfsLR {
...@@ -43,8 +42,7 @@ public final class JavaHdfsLR { ...@@ -43,8 +42,7 @@ public final class JavaHdfsLR {
static void showWarning() { static void showWarning() {
String warning = "WARN: This is a naive implementation of Logistic Regression " + String warning = "WARN: This is a naive implementation of Logistic Regression " +
"and is given as an example!\n" + "and is given as an example!\n" +
"Please use either org.apache.spark.mllib.classification.LogisticRegressionWithSGD " + "Please use org.apache.spark.ml.classification.LogisticRegression " +
"or org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS " +
"for more conventional use."; "for more conventional use.";
System.err.println(warning); System.err.println(warning);
} }
......
...@@ -17,7 +17,7 @@ ...@@ -17,7 +17,7 @@
""" """
This is an example implementation of ALS for learning how to use Spark. Please refer to This is an example implementation of ALS for learning how to use Spark. Please refer to
ALS in pyspark.mllib.recommendation for more conventional use. pyspark.ml.recommendation.ALS for more conventional use.
This example requires numpy (http://www.numpy.org/) This example requires numpy (http://www.numpy.org/)
""" """
...@@ -59,7 +59,7 @@ if __name__ == "__main__": ...@@ -59,7 +59,7 @@ if __name__ == "__main__":
""" """
print("""WARN: This is a naive implementation of ALS and is given as an print("""WARN: This is a naive implementation of ALS and is given as an
example. Please use the ALS method found in pyspark.mllib.recommendation for more example. Please use pyspark.ml.recommendation.ALS for more
conventional use.""", file=sys.stderr) conventional use.""", file=sys.stderr)
sc = SparkContext(appName="PythonALS") sc = SparkContext(appName="PythonALS")
......
...@@ -17,8 +17,8 @@ ...@@ -17,8 +17,8 @@
""" """
The K-means algorithm written from scratch against PySpark. In practice, The K-means algorithm written from scratch against PySpark. In practice,
one may prefer to use the KMeans algorithm in MLlib, as shown in one may prefer to use the KMeans algorithm in ML, as shown in
examples/src/main/python/mllib/kmeans.py. examples/src/main/python/ml/kmeans_example.py.
This example requires NumPy (http://www.numpy.org/). This example requires NumPy (http://www.numpy.org/).
""" """
...@@ -52,8 +52,8 @@ if __name__ == "__main__": ...@@ -52,8 +52,8 @@ if __name__ == "__main__":
exit(-1) exit(-1)
print("""WARN: This is a naive implementation of KMeans Clustering and is given print("""WARN: This is a naive implementation of KMeans Clustering and is given
as an example! Please refer to examples/src/main/python/mllib/kmeans.py for an example on as an example! Please refer to examples/src/main/python/ml/kmeans_example.py for an
how to use MLlib's KMeans implementation.""", file=sys.stderr) example on how to use ML's KMeans implementation.""", file=sys.stderr)
sc = SparkContext(appName="PythonKMeans") sc = SparkContext(appName="PythonKMeans")
lines = sc.textFile(sys.argv[1]) lines = sc.textFile(sys.argv[1])
......
...@@ -20,7 +20,7 @@ A logistic regression implementation that uses NumPy (http://www.numpy.org) ...@@ -20,7 +20,7 @@ A logistic regression implementation that uses NumPy (http://www.numpy.org)
to act on batches of input data using efficient matrix operations. to act on batches of input data using efficient matrix operations.
In practice, one may prefer to use the LogisticRegression algorithm in In practice, one may prefer to use the LogisticRegression algorithm in
MLlib, as shown in examples/src/main/python/mllib/logistic_regression.py. ML, as shown in examples/src/main/python/ml/logistic_regression_with_elastic_net.py.
""" """
from __future__ import print_function from __future__ import print_function
...@@ -51,8 +51,9 @@ if __name__ == "__main__": ...@@ -51,8 +51,9 @@ if __name__ == "__main__":
exit(-1) exit(-1)
print("""WARN: This is a naive implementation of Logistic Regression and is print("""WARN: This is a naive implementation of Logistic Regression and is
given as an example! Please refer to examples/src/main/python/mllib/logistic_regression.py given as an example!
to see how MLlib's implementation is used.""", file=sys.stderr) Please refer to examples/src/main/python/ml/logistic_regression_with_elastic_net.py
to see how ML's implementation is used.""", file=sys.stderr)
sc = SparkContext(appName="PythonLR") sc = SparkContext(appName="PythonLR")
points = sc.textFile(sys.argv[1]).mapPartitions(readPointBatch).cache() points = sc.textFile(sys.argv[1]).mapPartitions(readPointBatch).cache()
......
...@@ -24,7 +24,7 @@ import org.apache.commons.math3.linear._ ...@@ -24,7 +24,7 @@ import org.apache.commons.math3.linear._
* Alternating least squares matrix factorization. * Alternating least squares matrix factorization.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to org.apache.spark.mllib.recommendation.ALS * please refer to org.apache.spark.ml.recommendation.ALS.
*/ */
object LocalALS { object LocalALS {
...@@ -96,7 +96,7 @@ object LocalALS { ...@@ -96,7 +96,7 @@ object LocalALS {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of ALS and is given as an example! """WARN: This is a naive implementation of ALS and is given as an example!
|Please use the ALS method found in org.apache.spark.mllib.recommendation |Please use org.apache.spark.ml.recommendation.ALS
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
...@@ -26,8 +26,7 @@ import breeze.linalg.{DenseVector, Vector} ...@@ -26,8 +26,7 @@ import breeze.linalg.{DenseVector, Vector}
* Logistic regression based classification. * Logistic regression based classification.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or * please refer to org.apache.spark.ml.classification.LogisticRegression.
* org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS based on your needs.
*/ */
object LocalFileLR { object LocalFileLR {
val D = 10 // Number of dimensions val D = 10 // Number of dimensions
...@@ -43,8 +42,7 @@ object LocalFileLR { ...@@ -43,8 +42,7 @@ object LocalFileLR {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of Logistic Regression and is given as an example! """WARN: This is a naive implementation of Logistic Regression and is given as an example!
|Please use either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or |Please use org.apache.spark.ml.classification.LogisticRegression
|org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
...@@ -29,7 +29,7 @@ import breeze.linalg.{squaredDistance, DenseVector, Vector} ...@@ -29,7 +29,7 @@ import breeze.linalg.{squaredDistance, DenseVector, Vector}
* K-means clustering. * K-means clustering.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to org.apache.spark.mllib.clustering.KMeans * please refer to org.apache.spark.ml.clustering.KMeans.
*/ */
object LocalKMeans { object LocalKMeans {
val N = 1000 val N = 1000
...@@ -66,7 +66,7 @@ object LocalKMeans { ...@@ -66,7 +66,7 @@ object LocalKMeans {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of KMeans Clustering and is given as an example! """WARN: This is a naive implementation of KMeans Clustering and is given as an example!
|Please use the KMeans method found in org.apache.spark.mllib.clustering |Please use org.apache.spark.ml.clustering.KMeans
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
...@@ -26,8 +26,7 @@ import breeze.linalg.{DenseVector, Vector} ...@@ -26,8 +26,7 @@ import breeze.linalg.{DenseVector, Vector}
* Logistic regression based classification. * Logistic regression based classification.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or * please refer to org.apache.spark.ml.classification.LogisticRegression.
* org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS based on your needs.
*/ */
object LocalLR { object LocalLR {
val N = 10000 // Number of data points val N = 10000 // Number of data points
...@@ -50,8 +49,7 @@ object LocalLR { ...@@ -50,8 +49,7 @@ object LocalLR {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of Logistic Regression and is given as an example! """WARN: This is a naive implementation of Logistic Regression and is given as an example!
|Please use either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or |Please use org.apache.spark.ml.classification.LogisticRegression
|org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
...@@ -26,7 +26,7 @@ import org.apache.spark._ ...@@ -26,7 +26,7 @@ import org.apache.spark._
* Alternating least squares matrix factorization. * Alternating least squares matrix factorization.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to org.apache.spark.mllib.recommendation.ALS * please refer to org.apache.spark.ml.recommendation.ALS.
*/ */
object SparkALS { object SparkALS {
...@@ -81,7 +81,7 @@ object SparkALS { ...@@ -81,7 +81,7 @@ object SparkALS {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of ALS and is given as an example! """WARN: This is a naive implementation of ALS and is given as an example!
|Please use the ALS method found in org.apache.spark.mllib.recommendation |Please use org.apache.spark.ml.recommendation.ALS
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
...@@ -31,8 +31,7 @@ import org.apache.spark._ ...@@ -31,8 +31,7 @@ import org.apache.spark._
* Logistic regression based classification. * Logistic regression based classification.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or * please refer to org.apache.spark.ml.classification.LogisticRegression.
* org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS based on your needs.
*/ */
object SparkHdfsLR { object SparkHdfsLR {
val D = 10 // Number of dimensions val D = 10 // Number of dimensions
...@@ -54,8 +53,7 @@ object SparkHdfsLR { ...@@ -54,8 +53,7 @@ object SparkHdfsLR {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of Logistic Regression and is given as an example! """WARN: This is a naive implementation of Logistic Regression and is given as an example!
|Please use either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or |Please use org.apache.spark.ml.classification.LogisticRegression
|org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
...@@ -26,7 +26,7 @@ import org.apache.spark.{SparkConf, SparkContext} ...@@ -26,7 +26,7 @@ import org.apache.spark.{SparkConf, SparkContext}
* K-means clustering. * K-means clustering.
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to org.apache.spark.mllib.clustering.KMeans * please refer to org.apache.spark.ml.clustering.KMeans.
*/ */
object SparkKMeans { object SparkKMeans {
...@@ -52,7 +52,7 @@ object SparkKMeans { ...@@ -52,7 +52,7 @@ object SparkKMeans {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of KMeans Clustering and is given as an example! """WARN: This is a naive implementation of KMeans Clustering and is given as an example!
|Please use the KMeans method found in org.apache.spark.mllib.clustering |Please use org.apache.spark.ml.clustering.KMeans
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
...@@ -31,8 +31,7 @@ import org.apache.spark._ ...@@ -31,8 +31,7 @@ import org.apache.spark._
* Usage: SparkLR [slices] * Usage: SparkLR [slices]
* *
* This is an example implementation for learning how to use Spark. For more conventional use, * This is an example implementation for learning how to use Spark. For more conventional use,
* please refer to either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or * please refer to org.apache.spark.ml.classification.LogisticRegression.
* org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS based on your needs.
*/ */
object SparkLR { object SparkLR {
val N = 10000 // Number of data points val N = 10000 // Number of data points
...@@ -55,8 +54,7 @@ object SparkLR { ...@@ -55,8 +54,7 @@ object SparkLR {
def showWarning() { def showWarning() {
System.err.println( System.err.println(
"""WARN: This is a naive implementation of Logistic Regression and is given as an example! """WARN: This is a naive implementation of Logistic Regression and is given as an example!
|Please use either org.apache.spark.mllib.classification.LogisticRegressionWithSGD or |Please use org.apache.spark.ml.classification.LogisticRegression
|org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS
|for more conventional use. |for more conventional use.
""".stripMargin) """.stripMargin)
} }
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment