-
- Downloads
[SPARK-17761][SQL] Remove MutableRow
## What changes were proposed in this pull request? In practice we cannot guarantee that an `InternalRow` is immutable. This makes the `MutableRow` almost redundant. This PR folds `MutableRow` into `InternalRow`. The code below illustrates the immutability issue with InternalRow: ```scala import org.apache.spark.sql.catalyst.InternalRow import org.apache.spark.sql.catalyst.expressions.GenericMutableRow val struct = new GenericMutableRow(1) val row = InternalRow(struct, 1) println(row) scala> [[null], 1] struct.setInt(0, 42) println(row) scala> [[42], 1] ``` This might be somewhat controversial, so feedback is appreciated. ## How was this patch tested? Existing tests. Author: Herman van Hovell <hvanhovell@databricks.com> Closes #15333 from hvanhovell/SPARK-17761.
Showing
- mllib/src/main/scala/org/apache/spark/ml/linalg/MatrixUDT.scala 2 additions, 2 deletions...src/main/scala/org/apache/spark/ml/linalg/MatrixUDT.scala
- mllib/src/main/scala/org/apache/spark/ml/linalg/VectorUDT.scala 3 additions, 3 deletions...src/main/scala/org/apache/spark/ml/linalg/VectorUDT.scala
- mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala 2 additions, 2 deletions...c/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
- mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala 3 additions, 3 deletions...rc/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 1 addition, 1 deletion.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/InternalRow.scala 22 additions, 1 deletion...ain/scala/org/apache/spark/sql/catalyst/InternalRow.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala 1 addition, 1 deletion...pache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala 2 additions, 2 deletions...cala/org/apache/spark/sql/catalyst/expressions/Cast.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/JoinedRow.scala 16 additions, 0 deletions...org/apache/spark/sql/catalyst/expressions/JoinedRow.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala 2 additions, 2 deletions...rg/apache/spark/sql/catalyst/expressions/Projection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SpecificInternalRow.scala 2 additions, 3 deletions.../spark/sql/catalyst/expressions/SpecificInternalRow.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/HyperLogLogPlusPlus.scala 3 additions, 3 deletions.../catalyst/expressions/aggregate/HyperLogLogPlusPlus.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/PivotFirst.scala 5 additions, 5 deletions...spark/sql/catalyst/expressions/aggregate/PivotFirst.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala 3 additions, 3 deletions...he/spark/sql/catalyst/expressions/aggregate/collect.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala 7 additions, 7 deletions...spark/sql/catalyst/expressions/aggregate/interfaces.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 1 addition, 2 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala 4 additions, 4 deletions...alyst/expressions/codegen/GenerateMutableProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateSafeProjection.scala 4 additions, 4 deletions...catalyst/expressions/codegen/GenerateSafeProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/package.scala 1 addition, 1 deletion...a/org/apache/spark/sql/catalyst/expressions/package.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/rows.scala 1 addition, 43 deletions...cala/org/apache/spark/sql/catalyst/expressions/rows.scala
Loading
Please register or sign in to comment