Skip to content
Snippets Groups Projects
Commit cca9905b authored by KeheCAI's avatar KeheCAI Committed by Sean Owen
Browse files

update the deprecated CountMinSketchMonoid function to TopPctCMS function

http://twitter.github.io/algebird/index.html#com.twitter.algebird.legacy.CountMinSketchMonoid$
The CountMinSketchMonoid has been deprecated since 0.8.1. Newer code should use TopPctCMS.monoid().

![image](https://cloud.githubusercontent.com/assets/1327396/7269619/d8b48b92-e8d5-11e4-8902-087f630e6308.png)

Author: KeheCAI <caikehe@gmail.com>

Closes #5629 from caikehe/master and squashes the following commits:

e8aa06f [KeheCAI] update algebird-core to version 0.9.0 from 0.8.1
5653351 [KeheCAI] change scala code style
4c0dfd1 [KeheCAI] update the deprecated CountMinSketchMonoid function to TopPctCMS function
parent 59b7cfc4
No related branches found
No related tags found
No related merge requests found
...@@ -245,7 +245,7 @@ ...@@ -245,7 +245,7 @@
<dependency> <dependency>
<groupId>com.twitter</groupId> <groupId>com.twitter</groupId>
<artifactId>algebird-core_${scala.binary.version}</artifactId> <artifactId>algebird-core_${scala.binary.version}</artifactId>
<version>0.8.1</version> <version>0.9.0</version>
</dependency> </dependency>
<dependency> <dependency>
<groupId>org.scalacheck</groupId> <groupId>org.scalacheck</groupId>
......
...@@ -18,6 +18,7 @@ ...@@ -18,6 +18,7 @@
package org.apache.spark.examples.streaming package org.apache.spark.examples.streaming
import com.twitter.algebird._ import com.twitter.algebird._
import com.twitter.algebird.CMSHasherImplicits._
import org.apache.spark.SparkConf import org.apache.spark.SparkConf
import org.apache.spark.SparkContext._ import org.apache.spark.SparkContext._
...@@ -67,7 +68,8 @@ object TwitterAlgebirdCMS { ...@@ -67,7 +68,8 @@ object TwitterAlgebirdCMS {
val users = stream.map(status => status.getUser.getId) val users = stream.map(status => status.getUser.getId)
val cms = new CountMinSketchMonoid(EPS, DELTA, SEED, PERC) // val cms = new CountMinSketchMonoid(EPS, DELTA, SEED, PERC)
val cms = TopPctCMS.monoid[Long](EPS, DELTA, SEED, PERC)
var globalCMS = cms.zero var globalCMS = cms.zero
val mm = new MapMonoid[Long, Int]() val mm = new MapMonoid[Long, Int]()
var globalExact = Map[Long, Int]() var globalExact = Map[Long, Int]()
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment