-
- Downloads
[SPARK-3709] Executors don't always report broadcast block removal properly back to the driver
The problem was that the 2nd argument in RemoveBroadcast is not tellMaster! It is "removeFromDriver". Basically when removeFromDriver is not true, we don't report broadcast block removal back to the driver, and then other executors mistakenly think that the executor would still have the block, and try to fetch from it. cc @tdas Author: Reynold Xin <rxin@apache.org> Closes #2588 from rxin/debug and squashes the following commits: 6dab2e3 [Reynold Xin] Don't log random messages. f430686 [Reynold Xin] Always report broadcast removal back to master. 2a13f70 [Reynold Xin] iii
Showing
- core/src/main/scala/org/apache/spark/network/nio/NioBlockTransferService.scala 1 addition, 1 deletion...rg/apache/spark/network/nio/NioBlockTransferService.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerSlaveActor.scala 2 additions, 2 deletions...ala/org/apache/spark/storage/BlockManagerSlaveActor.scala
Please register or sign in to comment