Skip to content
Snippets Groups Projects
  • Ilya Ganelin's avatar
    fd3a8a1d
    [SPARK-733] Add documentation on use of accumulators in lazy transformation · fd3a8a1d
    Ilya Ganelin authored
    I've added documentation clarifying the particular lack of clarity highlighted in the relevant JIRA. I've also added code examples for this issue to clarify the explanation.
    
    Author: Ilya Ganelin <ilya.ganelin@capitalone.com>
    
    Closes #4022 from ilganeli/SPARK-733 and squashes the following commits:
    
    587def5 [Ilya Ganelin] Updated to clarify verbage
    df3afd7 [Ilya Ganelin] Revert "Partially updated task metrics to make some vars private"
    3f6c512 [Ilya Ganelin] Revert "Completed refactoring to make vars in TaskMetrics class private"
    58034fb [Ilya Ganelin] Merge remote-tracking branch 'upstream/master' into SPARK-733
    4dc2cdb [Ilya Ganelin] Merge remote-tracking branch 'upstream/master' into SPARK-733
    3a38db1 [Ilya Ganelin] Verified documentation update by building via jekyll
    33b5a2d [Ilya Ganelin] Added code examples for java and python
    1fd59b2 [Ilya Ganelin] Updated documentation for accumulators to highlight lazy evaluation issue
    5525c20 [Ilya Ganelin] Completed refactoring to make vars in TaskMetrics class private
    c64da4f [Ilya Ganelin] Partially updated task metrics to make some vars private
    fd3a8a1d
    History
    [SPARK-733] Add documentation on use of accumulators in lazy transformation
    Ilya Ganelin authored
    I've added documentation clarifying the particular lack of clarity highlighted in the relevant JIRA. I've also added code examples for this issue to clarify the explanation.
    
    Author: Ilya Ganelin <ilya.ganelin@capitalone.com>
    
    Closes #4022 from ilganeli/SPARK-733 and squashes the following commits:
    
    587def5 [Ilya Ganelin] Updated to clarify verbage
    df3afd7 [Ilya Ganelin] Revert "Partially updated task metrics to make some vars private"
    3f6c512 [Ilya Ganelin] Revert "Completed refactoring to make vars in TaskMetrics class private"
    58034fb [Ilya Ganelin] Merge remote-tracking branch 'upstream/master' into SPARK-733
    4dc2cdb [Ilya Ganelin] Merge remote-tracking branch 'upstream/master' into SPARK-733
    3a38db1 [Ilya Ganelin] Verified documentation update by building via jekyll
    33b5a2d [Ilya Ganelin] Added code examples for java and python
    1fd59b2 [Ilya Ganelin] Updated documentation for accumulators to highlight lazy evaluation issue
    5525c20 [Ilya Ganelin] Completed refactoring to make vars in TaskMetrics class private
    c64da4f [Ilya Ganelin] Partially updated task metrics to make some vars private