Skip to content
Snippets Groups Projects
Commit 1f0de3c1 authored by Sean Owen's avatar Sean Owen
Browse files

[SPARK-19991][CORE][YARN] FileSegmentManagedBuffer performance improvement

## What changes were proposed in this pull request?

Avoid `NoSuchElementException` every time `ConfigProvider.get(val, default)` falls back to default. This apparently causes non-trivial overhead in at least one path, and can easily be avoided.

See https://github.com/apache/spark/pull/17329

## How was this patch tested?

Existing tests

Author: Sean Owen <sowen@cloudera.com>

Closes #17567 from srowen/SPARK-19991.
parent 34fc48fb
No related branches found
No related tags found
No related merge requests found
......@@ -42,6 +42,12 @@ public class MapConfigProvider extends ConfigProvider {
return value;
}
@Override
public String get(String name, String defaultValue) {
String value = config.get(name);
return value == null ? defaultValue : value;
}
@Override
public Iterable<Map.Entry<String, String>> getAll() {
return config.entrySet();
......
......@@ -41,6 +41,12 @@ public class HadoopConfigProvider extends ConfigProvider {
return value;
}
@Override
public String get(String name, String defaultValue) {
String value = conf.get(name);
return value == null ? defaultValue : value;
}
@Override
public Iterable<Map.Entry<String, String>> getAll() {
return conf;
......
......@@ -60,7 +60,7 @@ object SparkTransportConf {
new TransportConf(module, new ConfigProvider {
override def get(name: String): String = conf.get(name)
override def get(name: String, defaultValue: String): String = conf.get(name, defaultValue)
override def getAll(): java.lang.Iterable[java.util.Map.Entry[String, String]] = {
conf.getAll.toMap.asJava.entrySet()
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment