Skip to content
Snippets Groups Projects
Commit a337c235 authored by Mahmoud Lababidi's avatar Mahmoud Lababidi Committed by Josh Rosen
Browse files

[SPARK-11158][SQL] Modified _verify_type() to be more informative on Errors by...

[SPARK-11158][SQL] Modified _verify_type() to be more informative on Errors by presenting the Object

The _verify_type() function had Errors that were raised when there were Type conversion issues but left out the Object in question. The Object is now added in the Error to reduce the strain on the user to debug through to figure out the Object that failed the Type conversion.

The use case for me was a Pandas DataFrame that contained 'nan' as values for columns of Strings.

Author: Mahmoud Lababidi <mahmoud@thehumangeo.com>
Author: Mahmoud Lababidi <lababidi@gmail.com>

Closes #9149 from lababidi/master.
parent 8d4449c7
No related branches found
No related tags found
No related merge requests found
......@@ -1127,15 +1127,15 @@ def _verify_type(obj, dataType):
return
_type = type(dataType)
assert _type in _acceptable_types, "unknown datatype: %s" % dataType
assert _type in _acceptable_types, "unknown datatype: %s for object %r" % (dataType, obj)
if _type is StructType:
if not isinstance(obj, (tuple, list)):
raise TypeError("StructType can not accept object in type %s" % type(obj))
raise TypeError("StructType can not accept object %r in type %s" % (obj, type(obj)))
else:
# subclass of them can not be fromInternald in JVM
if type(obj) not in _acceptable_types[_type]:
raise TypeError("%s can not accept object in type %s" % (dataType, type(obj)))
raise TypeError("%s can not accept object %r in type %s" % (dataType, obj, type(obj)))
if isinstance(dataType, ArrayType):
for i in obj:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment