-
- Downloads
[SPARK-21255][SQL][WIP] Fixed NPE when creating encoder for enum
## What changes were proposed in this pull request? Fixed NPE when creating encoder for enum. When you try to create an encoder for Enum type (or bean with enum property) via Encoders.bean(...), it fails with NullPointerException at TypeToken:495. I did a little research and it turns out, that in JavaTypeInference following code ``` def getJavaBeanReadableProperties(beanClass: Class[_]): Array[PropertyDescriptor] = { val beanInfo = Introspector.getBeanInfo(beanClass) beanInfo.getPropertyDescriptors.filterNot(_.getName == "class") .filter(_.getReadMethod != null) } ``` filters out properties named "class", because we wouldn't want to serialize that. But enum types have another property of type Class named "declaringClass", which we are trying to inspect recursively. Eventually we try to inspect ClassLoader class, which has property "defaultAssertionStatus" with no read method, which leads to NPE at TypeToken:495. I added property name "declaringClass" to filtering to resolve this. ## How was this patch tested? Unit test in JavaDatasetSuite which creates an encoder for enum Author: mike <mike0sv@gmail.com> Author: Mikhail Sveshnikov <mike0sv@gmail.com> Closes #18488 from mike0sv/enum-support.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala 40 additions, 0 deletions...ala/org/apache/spark/sql/catalyst/JavaTypeInference.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala 12 additions, 2 deletions...pache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala 2 additions, 2 deletions...ache/spark/sql/catalyst/expressions/objects/objects.scala
- sql/core/src/test/java/test/org/apache/spark/sql/JavaDatasetSuite.java 77 additions, 0 deletions...test/java/test/org/apache/spark/sql/JavaDatasetSuite.java
Loading
Please register or sign in to comment