Data Types
- Last Updated: January 22, 2018
- 1 minute read
- DataDirect Connectors
- JDBC
- Apache Spark SQL 6.0
- Documentation
The following table shows how the driver maps Apache Spark SQL data types to standard JDBC data types.
| Apache Spark SQL | JDBC |
|---|---|
| Bigint | BIGINT |
| Binary | VARBINARY or LONGVARBINARY1, 2 |
| Boolean | BOOLEAN |
| Date | DATE |
| Decimal | DECIMAL |
| Double | DOUBLE |
| Float | REAL |
| Int | INTEGER |
| Smallint | SMALLINT |
| String3, 4 | VARCHAR or LONGVARCHAR |
| Timestamp | TIMESTAMP |
| Tinyint | TINYINT |
| Varchar5 | VARCHAR |
1 If the
BinaryDescribeType property is set to
varbinary (the default), this data type maps to
VARBINARY. If set to longvarbinary,
this data type maps to LONGVARBINARY.2 Supported only for
Apache Spark SQL 2.0 and higher.
3 Maximum of 2
GB
4 If the StringDescribeType connection property is set
to
varchar (the default), the
String data type maps to VARCHAR. If StringDescribeType is set to
longvarchar, String maps to
LONGVARCHAR. StringDescribeType affects all columns reported as
String, even columns that were originally cast as Varchar. This is
important to note because the Spark Thrift server, when returning
result metadata for Varchar columns, reports column type as (12)
STRING and precision as 2147483647.5
When returning result set metadata for Varchar columns, the Spark Thrift server reports the column type as (12) STRING and the precision as 2147483647. For the latest information about this issue, refer to the Apache JIRA SPARK-5918 issue Web page.