-
- Downloads
[SPARK-2583] ConnectionManager error reporting
This patch modifies the ConnectionManager so that error messages are sent in reply when uncaught exceptions occur during message processing. This prevents message senders from hanging while waiting for an acknowledgment if the remote message processing failed. This is an updated version of sarutak's PR, #1490. The main change is to use Futures / Promises to signal errors. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Author: Josh Rosen <joshrosen@apache.org> Closes #1758 from JoshRosen/connection-manager-fixes and squashes the following commits: 68620cb [Josh Rosen] Fix test in BlockFetcherIteratorSuite: 83673de [Josh Rosen] Error ACKs should trigger IOExceptions, so catch only those exceptions in the test. b8bb4d4 [Josh Rosen] Fix manager.id vs managerServer.id typo that broke security tests. 659521f [Josh Rosen] Include previous exception when throwing new one a2f745c [Josh Rosen] Remove sendMessageReliablySync; callers can wait themselves. c01c450 [Josh Rosen] Return Try[Message] from sendMessageReliablySync. f1cd1bb [Josh Rosen] Clean up @sarutak's PR #1490 for [SPARK-2583]: ConnectionManager error reporting 7399c6b [Josh Rosen] Merge remote-tracking branch 'origin/pr/1490' into connection-manager-fixes ee91bb7 [Kousuke Saruta] Modified BufferMessage.scala to keep the spark code style 9dfd0d8 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2583 e7d9aa6 [Kousuke Saruta] rebase to master 326a17f [Kousuke Saruta] Add test cases to ConnectionManagerSuite.scala for SPARK-2583 2a18d6b [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2583 22d7ebd [Kousuke Saruta] Add test cases to BlockManagerSuite for SPARK-2583 e579302 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2583 281589c [Kousuke Saruta] Add a test case to BlockFetcherIteratorSuite.scala for fetching block from remote from successfully 0654128 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2583 ffaa83d [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2583 12d3de8 [Kousuke Saruta] Added BlockFetcherIteratorSuite.scala 4117b8f [Kousuke Saruta] Modified ConnectionManager to be alble to handle error during processing message 717c9c3 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2583 6635467 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2583 e2b8c4a [Kousuke Saruta] Modify to propagete error using ConnectionManager
Showing
- core/src/main/scala/org/apache/spark/network/BufferMessage.scala 4 additions, 3 deletions...c/main/scala/org/apache/spark/network/BufferMessage.scala
- core/src/main/scala/org/apache/spark/network/ConnectionManager.scala 83 additions, 60 deletions...in/scala/org/apache/spark/network/ConnectionManager.scala
- core/src/main/scala/org/apache/spark/network/Message.scala 2 additions, 0 deletionscore/src/main/scala/org/apache/spark/network/Message.scala
- core/src/main/scala/org/apache/spark/network/MessageChunkHeader.scala 5 additions, 2 deletions...n/scala/org/apache/spark/network/MessageChunkHeader.scala
- core/src/main/scala/org/apache/spark/network/SenderTest.scala 6 additions, 1 deletion.../src/main/scala/org/apache/spark/network/SenderTest.scala
- core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala 5 additions, 4 deletions...scala/org/apache/spark/storage/BlockFetcherIterator.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerWorker.scala 20 additions, 10 deletions...n/scala/org/apache/spark/storage/BlockManagerWorker.scala
- core/src/test/scala/org/apache/spark/network/ConnectionManagerSuite.scala 34 additions, 4 deletions...ala/org/apache/spark/network/ConnectionManagerSuite.scala
- core/src/test/scala/org/apache/spark/storage/BlockFetcherIteratorSuite.scala 94 additions, 4 deletions.../org/apache/spark/storage/BlockFetcherIteratorSuite.scala
- core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala 109 additions, 1 deletion...st/scala/org/apache/spark/storage/BlockManagerSuite.scala
Loading
Please register or sign in to comment