ODBC Drivers Updates & Release Notes

[SPARKO-1847] Updated third-party libraries

The connector now uses the following third-party libraries:

• Expat 2.8.2 (previously 2.8.1)
• libcURL 8.21.0 (previously 8.20.0)

Known Issues

The following are known issues that you may encounter due to limitations in
the data source, the connector, or an application.

• [SPARKO-1404] When querying tables that contain VOID columns, the server
  returns an error.

• [SPARKO-1101] When the Auth_AccessToken line length is longer than the
  maximum limit of 1000, the connector returns an authentication error. For
  more information, see the Installation and Configuration Guide.

• [SPARKO-879] When connecting to a server that supports multiple catalogs,
  the connector no longer reports the catalog for schemas and tables as
  SPARK.

  The Spark server now reports the catalog.

• [SPARKO-670] In some cases, when retrieving timestamp data, the connector
  returns an error.

  In some cases, when connecting to certain distributions of
  Apache Spark, the connector returns the following error: "Conversion from
  number to string failed due to undersized character buffer". This issue
  affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

  As a workaround, set EnableArrow=0 in the connection string or DSN.

• [SPARKO-620] Issue with date and timestamp before the beginning of the
  Gregorian calendar when connecting to Spark 2.4.4 or later, or versions
  previous to 3.0, with Arrow result set serialization.

  When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE
  and TIMESTAMP data before October 15, 1582 may be returned incorrectly if
  the server supports serializing query results using Apache Arrow. This
  issue should not impact most distributions of Apache Spark.

  To confirm if your distribution of Spark 2.4.4 or later has been impacted
  by this issue, you can execute the following query:

  SELECT DATE '1581-10-14'
  

  If the result returned by the connector is 1581-10-24, then you are
  impacted by the issue. In this case, if your data set contains date and/or
  timestamp data earlier than October 15, 1582, you can work around this
  issue by adding EnableArrow=0 in your DSN or connection string to disable
  the Arrow result set serialization feature.

• When retrieving data from a BINARY column, a ClassCastException error
  occurs.

  In Spark 1.6.3 or earlier, the server sometimes returns a
  ClassCastException error when attempting to retrieve data from a BINARY
  column.

  This issue is fixed as of Spark 2.0.0.

  For more information, see the JIRA issue posted by Apache named "When
  column type is binary, select occurs ClassCastException in Beeline" at
  https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes

The following changes may disrupt established workflows for the connector.

2.12.0

• [SPARKO-1760] SQL transactions limited to native query mode only.

  If the user attempts to use SQL transactions when UseNativeQuery=0, the
  connector will throw an error.

Enhancements & New Features

• [SPARKO-1756] SQLProcedures and SQLProcedureColumns support.

  The connector now supports SQLProcedures and SQLProcedureColumns catalog
  functions for retrieving stored procedure metadata when connected to
  Databricks 17.0 and later.

• [SPARKO-1691] Case-insensitive metadata filters.

  The connector now supports case-insensitive metadata filters for catalog
  functions.

  To enable this feature, set CaseInsensitiveMetadataFilter to 1. For more
  information, see the Installation and Configuration Guide.

• [SPARKO-1661] Use the Arrow data type for decimals.

  The connector now enables decimalAsArrow for decimal data types. This can
  be turned off by setting the connection flag DecimalAsArrow=0.

• [SPARKO-1721] Arrow data performance improvements.

  This release improves performance when using Arrow data.

• [SPARKO-1728] Bulk fetch performance improvements.

  This release improves bulk fetch performance.

• [SPARKO-1674] TLS handshake failure logging improvements.

  Server certificates are reported in the log file when there is an error
  during SSL handshakes.

• [SPARKO-1658] Previously updated: UseNativeQuery default value

  As part of SQL stored procedure support added in connector version 2.11.0,
  the default value of UseNativeQuery has been updated to 1 (enabled).

• [SPARKO-1775] SLES 16 support

  The connector now supports SLES 16.

• [SPARKO-1798] [SPARKO-1810] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Expat 2.8.1 (previously 2.7.4)
  • libcURL 8.20.0 (previously 8.18.0)
  • OpenSSL 3.5.7 (previously 3.0.19)
  • Thrift 0.23.0 (previously 0.17.0)
  • zlib 1.3.2 (previously 1.3.1)

Resolved Issues

The following issues have been resolved in Databricks ODBC Connector
2.12.0.

• [SPARKO-1623] A translation issue for column names when a FROM clause
  subquery is referenced in a correlated WHERE clause subquery has been
  resolved.

Known Issues

The following are known issues that you may encounter due to limitations in
the data source, the connector, or an application.

• [SPARKO-1404] When querying tables that contain VOID columns, the server
  returns an error.

• [SPARKO-1101] When the Auth_AccessToken line length is longer than the
  maximum limit of 1000, the connector returns an authentication error. For
  more information, see the Installation and Configuration Guide.

• [SPARKO-879] When connecting to a server that supports multiple catalogs,
  the connector no longer reports the catalog for schemas and tables as
  SPARK.

  The Spark server now reports the catalog.

• [SPARKO-670] In some cases, when retrieving timestamp data, the connector
  returns an error.

  In some cases, when connecting to certain distributions of
  Apache Spark, the connector returns the following error: "Conversion from
  number to string failed due to undersized character buffer". This issue
  affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

  As a workaround, set EnableArrow=0 in the connection string or DSN.

• [SPARKO-620] Issue with date and timestamp before the beginning of the
  Gregorian calendar when connecting to Spark 2.4.4 or later, or versions
  previous to 3.0, with Arrow result set serialization.

  When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE
  and TIMESTAMP data before October 15, 1582 may be returned incorrectly if
  the server supports serializing query results using Apache Arrow. This
  issue should not impact most distributions of Apache Spark.

  To confirm if your distribution of Spark 2.4.4 or later has been impacted
  by this issue, you can execute the following query:

  SELECT DATE '1581-10-14'
  

  If the result returned by the connector is 1581-10-24, then you are
  impacted by the issue. In this case, if your data set contains date and/or
  timestamp data earlier than October 15, 1582, you can work around this
  issue by adding EnableArrow=0 in your DSN or connection string to disable
  the Arrow result set serialization feature.

• When retrieving data from a BINARY column, a ClassCastException error
  occurs.

  In Spark 1.6.3 or earlier, the server sometimes returns a
  ClassCastException error when attempting to retrieve data from a BINARY
  column.

  This issue is fixed as of Spark 2.0.0.

  For more information, see the JIRA issue posted by Apache named "When
  column type is binary, select occurs ClassCastException in Beeline" at
  https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes =============================================================

The following changes may disrupt established workflows for the connector.

2.12.0 -----------------------------------------------------------------------

• [SPARKO-1760] SQL transactions limited to native query mode only.

  If the user attempts to use SQL transactions when UseNativeQuery=0, the
  connector will throw an error.

Enhancements & New Features

• [SPARKO-1658] Stored Procedures

  The connector now supports SQL stored procedures when using Databricks
  Unity Catalog.

• [SPARKO-1572] Telemetry support

  The EnableTelemetry connection setting is on by default. When enabled,
  telemetry data will be sent to servers that support this feature.

• [SPARKO-1696] Bulk fetch performance improvement

  The connector has improved bulk fetch performance.

• [SPARKO-1677] Geospatial datatype metadata improvement

  The connector now returns "GEOMETRY" or "GEOGRAPHY" for SQL_DESC_TYPE_NAME
  on geospatial data types.

• [SPARKO-1735] macOS 26 support

  The connector now support macOS 26.

• [SPARKO-1571] [SPARKO-1737] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Expat 2.7.4 (previously 2.7.1)
  • OpenSSL 3.0.19 (previously 3.0.18)
  • libcURL 8.18.0 (previously 8.16.0)

Resolved Issues

The following issues have been resolved in Databricks ODBC Connector
2.11.0.

• [SPARKO-1646] The DriverConfiguration.exe dialog correctly shows the
  connector as Databricks ODBC.

Known Issues

The following are known issues that you may encounter due to limitations in
the data source, the connector, or an application.

• [SPARKO-1404] When querying tables that contain VOID columns, the server
  returns an error.

• [SPARKO-1101] When the Auth_AccessToken line length is longer than the
  maximum limit of 1000, the connector returns an authentication error. For
  more information, see the Installation and Configuration Guide.

• [SPARKO-879] When connecting to a server that supports multiple catalogs,
  the connector no longer reports the catalog for schemas and tables as
  SPARK.

  The Spark server now reports the catalog.

• [SPARKO-670] In some cases, when retrieving timestamp data, the connector
  returns an error.

  In some cases, when connecting to certain distributions of
  Apache Spark, the connector returns the following error: "Conversion from
  number to string failed due to undersized character buffer". This issue
  affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

  As a workaround, set EnableArrow=0 in the connection string or DSN.

• [SPARKO-620] Issue with date and timestamp before the beginning of the
  Gregorian calendar when connecting to Spark 2.4.4 or later, or versions
  previous to 3.0, with Arrow result set serialization.

  When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE
  and TIMESTAMP data before October 15, 1582 may be returned incorrectly if
  the server supports serializing query results using Apache Arrow. This
  issue should not impact most distributions of Apache Spark.

  To confirm if your distribution of Spark 2.4.4 or later has been impacted
  by this issue, you can execute the following query:

  SELECT DATE '1581-10-14'
  

  If the result returned by the connector is 1581-10-24, then you are
  impacted by the issue. In this case, if your data set contains date and/or
  timestamp data earlier than October 15, 1582, you can work around this
  issue by adding EnableArrow=0 in your DSN or connection string to disable
  the Arrow result set serialization feature.

• When retrieving data from a BINARY column, a ClassCastException error
  occurs.

  In Spark 1.6.3 or earlier, the server sometimes returns a
  ClassCastException error when attempting to retrieve data from a BINARY
  column.

  This issue is fixed as of Spark 2.0.0.

  For more information, see the JIRA issue posted by Apache named "When
  column type is binary, select occurs ClassCastException in Beeline" at
  https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes

The following changes may disrupt established workflows for the connector.

2.11.0

• [SPARKO-1673] Removed Windows 10 support.

  Beginning with this release, the connector no longer support Windows 10.

Enhancements & New Features

• [SPARKO-1506][SPARKO-1643] Multi-statement transaction support

  The connector now supports multi-statement transactions when using compatible Databricks warehouses/tables.

  Use either SQLSetConnectAttr(SQL_AUTOCOMMIT_OFF=0) with SQLEndTran(COMMIT) / SQLEndTran(ROLLBACK), or SQL statements (BEGIN TRANSACTION / COMMIT / ROLLBACK). Note that mixing the ODBC API calls and SQL statements to use transactions is not supported.

• [SPARKO-1562] Metric views support

  The connector now supports Databricks metric views. When the connection property EnableMetricViewMetadata is set to 1, the connector sets the session configuration spark.sql.thriftserver.metadata.metricview.enabled to true and will identify metric views as distinct from tables, and metric view measure columns as distinct from regular columns.

• [SPARKO-1619] DBR 17.3 support

  The connector now support DBR version 17.3 LTS.

• [SPARKO-1668] Debian 13 support

  The connector now supports Debian 13 and Debian 13 on ARM.

Known Issues

The following are known issues that you may encounter due to limitations in the data source, the connector, or an application.

• [SPARKO-1404] When querying tables that contain VOID columns, the server returns an error.

• [SPARKO-1101] When the Auth_AccessToken line length is longer than the maximum limit of 1000, the connector returns an authentication error. For more information, see the Installation and Configuration Guide.

• [SPARKO-879] When connecting to a server that supports multiple catalogs, the connector no longer reports the catalog for schemas and tables as SPARK.

  The Spark server now reports the catalog.

• [SPARKO-670] In some cases, when retrieving timestamp data, the connector returns an error.

  In some cases, when connecting to certain distributions of Apache Spark, the connector returns the following error: "Conversion from number to string failed due to undersized character buffer". This issue affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

  As a workaround, set EnableArrow=0 in the connection string or DSN.

• [SPARKO-620] Issue with date and timestamp before the beginning of the Gregorian calendar when connecting to Spark 2.4.4 or later, or versions previous to 3.0, with Arrow result set serialization.

  When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE and TIMESTAMP data before October 15, 1582 may be returned incorrectly if the server supports serializing query results using Apache Arrow. This issue should not impact most distributions of Apache Spark.

  To confirm if your distribution of Spark 2.4.4 or later has been impacted by this issue, you can execute the following query:

  SELECT DATE '1581-10-14'
  

  If the result returned by the connector is 1581-10-24, then you are impacted by the issue. In this case, if your data set contains date and/or timestamp data earlier than October 15, 1582, you can work around this issue by adding EnableArrow=0 in your DSN or connection string to disable the Arrow result set serialization feature.

• When retrieving data from a BINARY column, a ClassCastException error occurs.

  In Spark 1.6.3 or earlier, the server sometimes returns a ClassCastException error when attempting to retrieve data from a BINARY column.

  This issue is fixed as of Spark 2.0.0.

  For more information, see the JIRA issue posted by Apache named "When column type is binary, select occurs ClassCastException in Beeline" at https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes =============================================================

The following changes may disrupt established workflows for the connector.

2.10.0 -----------------------------------------------------------------------

• [SPARKO-1669] Removed macOS 12 and macOS 13 support

  Beginning with this release, the connector no longer supports macOS 12 or 13.

2.9.4 ------------------------------------------------------------------------

• [SPARKO-1545] Removed macOS 11 support

  Beginning with this release, the connector no longer supports macOS 11. For information about the supported versions, see the Installation and Configuration Guide.

2.9.2 ------------------------------------------------------------------------

• [SPARKO-1472] Dropped support for earlier version of LTS

  Beginning with this release, the connector no longer supports the LTS version 10.4. For a list of supported versions, see the Installation and Configuration Guide.

• [SPARKO-1426] Updated default value

  The default values of max catalog name length and max schema name length have been changed from 128 to 1024.

• [SPARKO-1389] Removed support for Ubuntu 20.04

  As early as May 2025, the connector will no longer support Ubuntu 20.04. For information about the supported versions, see the Installation and Configuration Guide.

2.9.1 ------------------------------------------------------------------------

• [SPARKO-1401] Dropped support for earlier versions of Linux distributions

  The connector no longer supports the following versions of Linux distributions:

  • CentOS 7
  • RHEL 7
  • SLES 12

  For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.0 ------------------------------------------------------------------------

• [SPARKO-1167] Updated OAuth client ID value

  For the OAuth browser based authentication, the default value of the Client ID property is now "96eecda7-19ea-49cc-abb5-240097d554f5". Previously, the default value was "5e5b7d2b-32b1-4ac0-be5d-ad3ad710fa6e".

• [SPARKO-1155] Removed libcURL folder

  Beginning with this release, on Windows, lib\LibCurl64.DllA (for the 64-bit driver) and lib\LibCurl32.DllA (for the 32-bit driver) folders have been removed.

Version History ==============================================================

2.9.4 ========================================================================

Released 2025-11-05

Enhancements & New Features

• [SPARKO-1499] File DSN in Windows support

  The connector now saves the minimal properties in the FileDSN to work around the limitation of the Windows Driver Manager. For more information, see the Installation and Configuration Guide.

• [SPARKO-1525][SPARKO-1549] Telemetry milestone support

  The connector now supports sending telemetry to databricks servers. For more information, see the Installation and Configuration Guide.

• [SPARKO-1535][SPARKO-1552][SPARKO-1553][SPARKO-1565] Linux ARM support

  The connector now supports Linux ARM architecture. For more information, see the Installation and Configuration Guide.

• [SPARKO-1545] macOS ARM support

  The connector now supports macOS ARM architecture. For more information, see the Installation and Configuration Guide.

• [SPARKO-1558] Logging Download Speeds with Cloud Fetch

  With Cloud Fetch enabled, the connector logs average download speed at INFO level, in-progress speeds at TRACE level, and slow downloads at WARNING level.

• [SPARKO-1561] Kerberos auth support

  The connector now supports Kerberos authentication for Proxy. For more information, see the Installation and Configuration Guide.

• [SPARKO-1564] SQL Scripting support

  The connector now supports SQL scripting syntax where scripts are bookended by the BEGIN and END keywords.

• [SPARKO-1599] Updated RHEL support

  The connector now supports RHEL 10 operating system. For a list of supported Linux versions, see the Installation and Configuration Guide.

• [SPARKO-1607] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.18 (previously 3.0.16)
  • libcURL 8.16.0 (previously 8.12.1)

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.4.

• [SPARKO-1516][SPARKO-1518] The connector causes queries to time out after 20 minutes.

• [SPARKO-1537] The connector uses the 'username' and 'password' properties specified in the connection string for authentication in place of -u -p.

2.9.2 ------------------------------------------------------------------------

Released 2025-05-16

Enhancements & New Features

• [SPARKO-1408] Process Name as Default UserAgentEntry

  The driver will now use the process name as the UserAgentEntry if the UserAgentEntry is not set.

• [SPARKO-1472] Upgraded LTS support

  The driver now uses LTS versions 11.3 through 15.4. For supported versions, see the Installation and Configuration Guide.

• [SPARKO-1482][05221097] Databricks domains support

  The driver now supports cloud.databricks.us and cloud.databricks.mil domains.

• [SPARKO-1325][SPARKO-1435] Timestamp_NTZ support

  The driver now recognizes timestamp_ntz columns in the following data source tables:

  • SQLGetTypeInfo (SQL_ALL_TYPES) now lists timestamp_ntz as an available type.
  • SQLGetTypeInfo (SQL_TIMESTAMP) now lists timestamp_ntz as one of the types that map to SQL timestamps.
  • SQLColumns now identifies timestamp_ntz columns correctly.
  • If UseNativeQuery is set to 0, SQLColAttribute (SQL_DESC_TYPE_NAME) identifies timestamp_ntz columns correctly.

• [SPARKO-1384][SPARKO-1405][SPARKO-1411] CRL cache support

  On Windows, the driver supports the CRL cache when UseSystemTruststore is enabled (set to 1).

• [SPARKO-1399][SPARKO-1417] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.16 (previously 3.0.15)
  • libcURL 8.12.1 (previously 8.11.0)
  • Expat 2.7.1 (previously 2.6.3)

• [SPARKO-1404] VOID type support

  The driver now supports VOID columns and lists them correctly in SQLGetColumns calls.

• [SPARKO-1419][SPARKO-1420] OAuth Token exchange support

  The driver now supports OAuth Token exchange feature for IDP different from the host. In these cases, OAuth access token (including BYOT) will be exchanged for a Databricks in-house access token. For more information, see the Installation and Configuration Guide.

• [SPARKO-1474] Upgraded Windows Server support

  The connector now supports Windows Server 2025. For a list of supported versions, see the Installation and Configuration Guide.

Resolved Issues

The following issues have been resolved in Databricks ODBC Connector 2.9.2.

• [SPARKO-1516][SPARKO-1518] A memory leak is resolved.

2.9.1 ------------------------------------------------------------------------

Released 2024-11-15

Enhancements & New Features

• [SPARKO-1383] Updated Thrift Version for HTTP User-Agent Header

  The HTTP User-Agent used by the driver has been updated from thrift/0.9.0 to thrift/0.17.0.

• [SPARKO-1005] Driver-Side Timeouts for Idle Statements

  Added support for configuring the fetch result idle timeout. For more information, see the Installation and Configuration Guide.

• [SPARKO-1266] Upgrade Arrow

  Apache Arrow has been upgraded to version 15.0.0 across all non-Windows platforms.

• [SPARKO-1351][SPARKO-1387] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.15 (previously 3.0.13)
  • libcURL 8.11.0 (previously 8.7.1)
  • ICU 74.2 (previously 71.1)
  • Arrow 15.0.0 (previously 9.0.0) for OSX & Linux (Windows was on 15.0.0)
  • lz4 1.10.0. (previously 1.9.4)
  • Expat 2.6.3 (previously 2.6.2)
  • Kerberos 1.21.3 (previously 1.21.2) for OSX & Linux

• [SPARKO-1360] Support for Token Passthrough OAuth

  The driver now supports token renewal for the Token Passthrough OAuth flow.

• [SPARKO-1369] Use 'hive_metastore' as default for Empty Initial Catalog

  If the server sends an empty string as the initial catalog, the driver will default to using the "hive_metastore" catalog.

• [SPARKO-1371] Support for DBR

  The driver now supports DBR 15.4 LTS.

• [SPARKO-1378] Buffer length based on the DM's encoding.

  The driver now returns the buffer length based on the DM's encoding.

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.1.

• [SPARKO-1149] When the driver encounters an error while sending heartbeat, it now re-attempts the heartbeat once before closing the heartbeat thread.

• [SPARKO-1376] HttpPath property missing issue when setting up DSN through Add-OdbcDsn in PowerShell has been fixed.

2.8.2 ------------------------------------------------------------------------

Released 2024-05-31

Enhancements & New Features

• [SPARKO-1204][SPARKO-1243][SPARKO-1279] Updated authentication support

  The connector now supports Browser Based (U2M) and Client Credentials (M2) authentication on GCP cloud.

  On AWS, Azure, and GCP these are the new default values:

  • Authorization endpoint: hostname + "/oidc/oauth2/v2.0/authorize"
  • Token endpoint: hostname+"/oidc/oauth2/v2.0/token"
  • Client ID (U2M): databricks-sql-odbc
  • Scope (U2M): sql offline_access
  • Scope (M2M): sql

  On Azure, if the application specifies a client ID that is different from the default value, the default scope is:

  • For U2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/user_impersonation, offline_access
  • For M2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default

  You can now configure the OAuth redirect port. To do this, set the OAuth2RedirectUrlPort property to your port. For more information, see the Installation and Configuration Guide.

• [SPARKO-1302] Updated support for Linux distributions

  The connector now supports the following operating systems:

  • Debian 11
  • Ubuntu 24.04

  For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.1 ========================================================================

Released 2024-06-14

Enhancements & New Features

• [SPARKO-1136] UC Volume ingestion

  The connector now supports UC Volume ingestion operations. To do this, set the StagingAllowedLocalPaths property to the local path. For more information, see the Installation and Configuration Guide.

• [SPARKO-1264] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Expat 2.6.2 (previously 2.6.0)
  • libcURL 8.7.1 (previously 8.6.0)

Resolved Issues

The following issues has been resolved in Databricks ODBC Connector 2.8.1.

• [SPARKO-988] When the comments' length is greater than MaxCommentLen, the connector returns an error.

• [SPARKO-1275] When using a private signed CA on Windows Truststore, the connector fails to verify the OIDC endpoint's certificate.

• [SPARKO-1267] On Windows, when using Kerbeos authentication and KrbRealm is empty, the connection fails.

• [SPARKO-1262] On Windows, when using SASL authentication and the SASL plugins folder path exceeds 128 characters, the connector returns a "Failed to initialize SASL client library: generic failure." error message.

• [SPARKO-1254] The connector returns an error for SQLCancel.

2.8.0 ------------------------------------------------------------------------

Released 2024-03-11

Enhancements & New Features

• [SPARKO-1137] Enhanced client credentials

  The connector now supports JWT assertion as the client credentials for OAuth. For more information, see the Installation and Configuration Guide.

• [SPARKO-1146] Token renew support

  For token passthrough authentication, you can now renew your token.

• [SPARKO-1147][SPARKO-1171][SPARKO-1236] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Arrow 15.0.0 (previously 9.0.0)(on Windows)
  • libcURL 8.6.0 (previously 8.4.0)
  • Zlib 1.3.1 (previously 1.2.13)

• [SPARKO-1147] Upgraded compiler support

  The connector now uses Windows Visual Studio 2022. For supported versions, see the Installation and Configuration Guide.

• [SPARKO-1147] Updated security flags

  The following security flags have been used to improve security checks in the connector:

  • /CETCOMPAT
  • /guard:cf
  • /Qspectre
  • /SDL

• [SPARKO-1165] Accept Undetermined Revocation support

  The connector can now access the SSL connection when the certificate's revocation status is undetermined. To do this, select the Accept Undetermined Revocation checkbox (set the AcceptUndeterminedRevocation property to 1). For more information, see the Installation and Configuration Guide.

• [SPARKO-1165] ProxyIgnoreList support

  The connector can now allow the hosts or domains that do not use proxy. To do this, set the ProxyIgnoreList property to a comma separated hostname. For more information, see the Installation and Configuration Guide.

• [SPARKO-1249] OAuth enhancement

  The connector can now connect to the OIDC endpoint when ProxyUID is unset.

• [SPARKO-1155] Updated libcURL location

  On Windows, the libcURL DLL and manifest files are now located under the lib folder.

Resolved Issues

The following issue has been resolved in Simba Apache Spark ODBC Connector 2.8.0.

• [SPARKO-1252] When UseNativeQuery is set to 1 for a cluster later than DBR 11, the connector returns an incorrect column number after SQLPrepare.

Enhancements & New Features

• [SPARKO-1506][SPARKO-1643] Multi-statement transaction support

  The connector now supports multi-statement transactions when using compatible Databricks warehouses/tables.

  Use either SQLSetConnectAttr(SQL_AUTOCOMMIT_OFF=0) with SQLEndTran(COMMIT) / SQLEndTran(ROLLBACK), or SQL statements (BEGIN TRANSACTION / COMMIT / ROLLBACK). Note that mixing the ODBC API calls and SQL statements to use transactions is not supported.

• [SPARKO-1562] Metric views support

  The connector now supports Databricks metric views. When the connection property EnableMetricViewMetadata is set to 1, the connector sets the session configuration spark.sql.thriftserver.metadata.metricview.enabled to true and will identify metric views as distinct from tables, and metric view measure columns as distinct from regular columns.

• [SPARKO-1619] DBR 17.3 support

  The connector now support DBR version 17.3 LTS.

• [SPARKO-1668] Debian 13 support

  The connector now supports Debian 13 and Debian 13 on ARM.

Known Issues

The following are known issues that you may encounter due to limitations in the data source, the connector, or an application.

• [SPARKO-1404] When querying tables that contain VOID columns, the server returns an error.

• [SPARKO-1101] When the Auth_AccessToken line length is longer than the maximum limit of 1000, the connector returns an authentication error. For more information, see the Installation and Configuration Guide.

• [SPARKO-879] When connecting to a server that supports multiple catalogs, the connector no longer reports the catalog for schemas and tables as SPARK.

  The Spark server now reports the catalog.

• [SPARKO-670] In some cases, when retrieving timestamp data, the connector returns an error.

  In some cases, when connecting to certain distributions of Apache Spark, the connector returns the following error: "Conversion from number to string failed due to undersized character buffer". This issue affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

  As a workaround, set EnableArrow=0 in the connection string or DSN.

• [SPARKO-620] Issue with date and timestamp before the beginning of the Gregorian calendar when connecting to Spark 2.4.4 or later, or versions previous to 3.0, with Arrow result set serialization.

  When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE and TIMESTAMP data before October 15, 1582 may be returned incorrectly if the server supports serializing query results using Apache Arrow. This issue should not impact most distributions of Apache Spark.

  To confirm if your distribution of Spark 2.4.4 or later has been impacted by this issue, you can execute the following query:

  SELECT DATE '1581-10-14'
  

  If the result returned by the connector is 1581-10-24, then you are impacted by the issue. In this case, if your data set contains date and/or timestamp data earlier than October 15, 1582, you can work around this issue by adding EnableArrow=0 in your DSN or connection string to disable the Arrow result set serialization feature.

• When retrieving data from a BINARY column, a ClassCastException error occurs.

  In Spark 1.6.3 or earlier, the server sometimes returns a ClassCastException error when attempting to retrieve data from a BINARY column.

  This issue is fixed as of Spark 2.0.0.

  For more information, see the JIRA issue posted by Apache named "When column type is binary, select occurs ClassCastException in Beeline" at https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes =============================================================

The following changes may disrupt established workflows for the connector.

2.10.0 -----------------------------------------------------------------------

• [SPARKO-1669] Removed macOS 12 and macOS 13 support

  Beginning with this release, the connector no longer supports macOS 12 or 13.

2.9.4 ------------------------------------------------------------------------

• [SPARKO-1545] Removed macOS 11 support

  Beginning with this release, the connector no longer supports macOS 11. For information about the supported versions, see the Installation and Configuration Guide.

2.9.2 ------------------------------------------------------------------------

• [SPARKO-1472] Dropped support for earlier version of LTS

  Beginning with this release, the connector no longer supports the LTS version 10.4. For a list of supported versions, see the Installation and Configuration Guide.

• [SPARKO-1426] Updated default value

  The default values of max catalog name length and max schema name length have been changed from 128 to 1024.

• [SPARKO-1389] Removed support for Ubuntu 20.04

  As early as May 2025, the connector will no longer support Ubuntu 20.04. For information about the supported versions, see the Installation and Configuration Guide.

2.9.1 ------------------------------------------------------------------------

• [SPARKO-1401] Dropped support for earlier versions of Linux distributions

  The connector no longer supports the following versions of Linux distributions:

  • CentOS 7
  • RHEL 7
  • SLES 12

  For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.0 ------------------------------------------------------------------------

• [SPARKO-1167] Updated OAuth client ID value

  For the OAuth browser based authentication, the default value of the Client ID property is now "96eecda7-19ea-49cc-abb5-240097d554f5". Previously, the default value was "5e5b7d2b-32b1-4ac0-be5d-ad3ad710fa6e".

• [SPARKO-1155] Removed libcURL folder

  Beginning with this release, on Windows, lib\LibCurl64.DllA (for the 64-bit driver) and lib\LibCurl32.DllA (for the 32-bit driver) folders have been removed.

Version History ==============================================================

2.9.4 ========================================================================

Released 2025-10-10

Enhancements & New Features

• [SPARKO-1499] File DSN in Windows support

  The connector now saves the minimal properties in the FileDSN to work around the limitation of the Windows Driver Manager. For more information, see the Installation and Configuration Guide.

• [SPARKO-1525][SPARKO-1549] Telemetry milestone support

  The connector now supports sending telemetry to databricks servers. For more information, see the Installation and Configuration Guide.

• [SPARKO-1535][SPARKO-1552][SPARKO-1553][SPARKO-1565] Linux ARM support

  The connector now supports Linux ARM architecture. For more information, see the Installation and Configuration Guide.

• [SPARKO-1545] macOS ARM support

  The connector now supports macOS ARM architecture. For more information, see the Installation and Configuration Guide.

• [SPARKO-1558] Logging Download Speeds with Cloud Fetch

  With Cloud Fetch enabled, the connector logs average download speed at INFO level, in-progress speeds at TRACE level, and slow downloads at WARNING level.

• [SPARKO-1561] Kerberos auth support

  The connector now supports Kerberos authentication for Proxy. For more information, see the Installation and Configuration Guide.

• [SPARKO-1564] SQL Scripting support

  The connector now supports SQL scripting syntax where scripts are bookended by the BEGIN and END keywords.

• [SPARKO-1599] Updated RHEL support

  The connector now supports RHEL 10 operating system. For a list of supported Linux versions, see the Installation and Configuration Guide.

• [SPARKO-1607] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.18 (previously 3.0.16)
  • libcURL 8.16.0 (previously 8.12.1)

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.4.

• [SPARKO-1516][SPARKO-1518] The connector causes queries to time out after 20 minutes.

• [SPARKO-1537] The connector uses the 'username' and 'password' properties specified in the connection string for authentication in place of -u -p.

2.9.2 ------------------------------------------------------------------------

Released 2025-04-25

Enhancements & New Features

• [SPARKO-1408] Process Name as Default UserAgentEntry

  The driver will now use the process name as the UserAgentEntry if the UserAgentEntry is not set.

• [SPARKO-1472] Upgraded LTS support

  The driver now uses LTS versions 11.3 through 15.4. For supported versions, see the Installation and Configuration Guide.

• [SPARKO-1482][05221097] Databricks domains support

  The driver now supports cloud.databricks.us and cloud.databricks.mil domains.

• [SPARKO-1325][SPARKO-1435] Timestamp_NTZ support

  The driver now recognizes timestamp_ntz columns in the following data source tables:

  • SQLGetTypeInfo (SQL_ALL_TYPES) now lists timestamp_ntz as an available type.
  • SQLGetTypeInfo (SQL_TIMESTAMP) now lists timestamp_ntz as one of the types that map to SQL timestamps.
  • SQLColumns now identifies timestamp_ntz columns correctly.
  • If UseNativeQuery is set to 0, SQLColAttribute (SQL_DESC_TYPE_NAME) identifies timestamp_ntz columns correctly.

• [SPARKO-1384][SPARKO-1405][SPARKO-1411] CRL cache support

  On Windows, the driver supports the CRL cache when UseSystemTruststore is enabled (set to 1).

• [SPARKO-1399][SPARKO-1417] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.16 (previously 3.0.15)
  • libcURL 8.12.1 (previously 8.11.0)
  • Expat 2.7.1 (previously 2.6.3)

• [SPARKO-1404] VOID type support

  The driver now supports VOID columns and lists them correctly in SQLGetColumns calls.

• [SPARKO-1419][SPARKO-1420] OAuth Token exchange support

  The driver now supports OAuth Token exchange feature for IDP different from the host. In these cases, OAuth access token (including BYOT) will be exchanged for a Databricks in-house access token. For more information, see the Installation and Configuration Guide.

• [SPARKO-1474] Upgraded Windows Server support

  The connector now supports Windows Server 2025. For a list of supported versions, see the Installation and Configuration Guide.

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.2.

• [SPARKO-1516][SPARKO-1518] A memory leak is resolved.

2.9.1 ------------------------------------------------------------------------

Released 2024-11-15

Enhancements & New Features

• [SPARKO-1383] Updated Thrift Version for HTTP User-Agent Header

  The HTTP User-Agent used by the driver has been updated from thrift/0.9.0 to thrift/0.17.0.

• [SPARKO-1005] Driver-Side Timeouts for Idle Statements

  Added support for configuring the fetch result idle timeout. For more information, see the Installation and Configuration Guide.

• [SPARKO-1266] Upgrade Arrow

  Apache Arrow has been upgraded to version 15.0.0 across all non-Windows platforms.

• [SPARKO-1351][SPARKO-1387] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.15 (previously 3.0.13)
  • libcURL 8.11.0 (previously 8.7.1)
  • ICU 74.2 (previously 71.1)
  • Arrow 15.0.0 (previously 9.0.0) for OSX & Linux (Windows was on 15.0.0)
  • lz4 1.10.0. (previously 1.9.4)
  • Expat 2.6.3 (previously 2.6.2)
  • Kerberos 1.21.3 (previously 1.21.2) for OSX & Linux

• [SPARKO-1360] Support for Token Passthrough OAuth

  The driver now supports token renewal for the Token Passthrough OAuth flow.

• [SPARKO-1369] Use 'hive_metastore' as default for Empty Initial Catalog

  If the server sends an empty string as the initial catalog, the driver will default to using the "hive_metastore" catalog.

• [SPARKO-1371] Support for DBR

  The driver now supports DBR 15.4 LTS.

• [SPARKO-1378] Buffer length based on the DM's encoding.

  The driver now returns the buffer length based on the DM's encoding.

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.1.

• [SPARKO-1149] When the driver encounters an error while sending heartbeat, it now re-attempts the heartbeat once before closing the heartbeat thread.

• [SPARKO-1376] HttpPath property missing issue when setting up DSN through Add-OdbcDsn in PowerShell has been fixed.

2.8.2 ------------------------------------------------------------------------

Released 2024-05-31

Enhancements & New Features

• [SPARKO-1204][SPARKO-1243][SPARKO-1279] Updated authentication support

  The connector now supports Browser Based (U2M) and Client Credentials (M2) authentication on GCP cloud.

  On AWS, Azure, and GCP these are the new default values:

  • Authorization endpoint: hostname + "/oidc/oauth2/v2.0/authorize"
  • Token endpoint: hostname+"/oidc/oauth2/v2.0/token"
  • Client ID (U2M): databricks-sql-odbc
  • Scope (U2M): sql offline_access
  • Scope (M2M): sql

  On Azure, if the application specifies a client ID that is different from the default value, the default scope is:

  • For U2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/user_impersonation, offline_access
  • For M2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default

  You can now configure the OAuth redirect port. To do this, set the OAuth2RedirectUrlPort property to your port. For more information, see the Installation and Configuration Guide.

• [SPARKO-1302] Updated support for Linux distributions

  The connector now supports the following operating systems:

  • Debian 11
  • Ubuntu 24.04

  For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.1 ------------------------------------------------------------------------

Released 2024-05-31

Enhancements & New Features

• [SPARKO-1136] UC Volume ingestion

  The connector now supports UC Volume ingestion operations. To do this, set the StagingAllowedLocalPaths property to the local path. For more information, see the Installation and Configuration Guide.

• [SPARKO-1264] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Expat 2.6.2 (previously 2.6.0)
  • libcURL 8.7.1 (previously 8.6.0)

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.8.1.

• [SPARKO-988] When the comments' length is greater than MaxCommentLen, the connector returns an error.

• [SPARKO-1275] When using a private signed CA on Windows Truststore, the connector fails to verify the OIDC endpoint's certificate.

• [SPARKO-1267] On Windows, when using Kerbeos authentication and KrbRealm is empty, the connection fails.

• [SPARKO-1262] On Windows, when using SASL authentication and the SASL plugins folder path exceeds 128 characters, the connector returns a "Failed to initialize SASL client library: generic failure." error message.

• [SPARKO-1254] The connector returns an error for SQLCancel.

2.8.0 ------------------------------------------------------------------------

Released 2024-03-11

Enhancements & New Features

• [SPARKO-1137] Enhanced client credentials

  The connector now supports JWT assertion as the client credentials for OAuth. For more information, see the Installation and Configuration Guide.

• [SPARKO-1146] Token renew support

  For token passthrough authentication, you can now renew your token.

• [SPARKO-1147][SPARKO-1171][SPARKO-1236] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Arrow 15.0.0 (previously 9.0.0)(on Windows)
  • libcURL 8.6.0 (previously 8.4.0)
  • Zlib 1.3.1 (previously 1.2.13)

• [SPARKO-1147] Upgraded compiler support

  The connector now uses Windows Visual Studio 2022. For supported versions, see the Installation and Configuration Guide.

• [SPARKO-1147] Updated security flags

  The following security flags have been used to improve security checks in the connector:

  • /CETCOMPAT
  • /guard:cf
  • /Qspectre
  • /SDL

• [SPARKO-1165] Accept Undetermined Revocation support

  The connector can now access the SSL connection when the certificate's revocation status is undetermined. To do this, select the Accept Undetermined Revocation checkbox (set the AcceptUndeterminedRevocation property to 1). For more information, see the Installation and Configuration Guide.

• [SPARKO-1165] ProxyIgnoreList support

  The connector can now allow the hosts or domains that do not use proxy. To do this, set the ProxyIgnoreList property to a comma separated hostname. For more information, see the Installation and Configuration Guide.

• [SPARKO-1249] OAuth enhancement

  The connector can now connect to the OIDC endpoint when ProxyUID is unset.

• [SPARKO-1155] Updated libcURL location

  On Windows, the libcURL DLL and manifest files are now located in the lib folder.

Resolved Issues

The following issue has been resolved in Simba Apache Spark ODBC Connector 2.8.0.

• [SPARKO-1252] When UseNativeQuery is set to 1 for a cluster later than DBR 11, the connector returns an incorrect column number after SQLPrepare.

Enhancements & New Features

• [SPARKO-1499] File DSN in Windows support

  The connector now saves the minimal properties in the FileDSN to work
  around the limitation of the Windows Driver Manager. For more information,
  see the Installation and Configuration Guide.

• [SPARKO-1525][SPARKO-1549] Telemetry milestone support

  The connector now supports sending telemetry to databricks servers. For
  more information, see the Installation and Configuration Guide.

• [SPARKO-1535][SPARKO-1552][SPARKO-1553][SPARKO-1565] Linux ARM support

  The connector now supports Linux ARM architecture. For more information,
  see the Installation and Configuration Guide.

• [SPARKO-1545] macOS ARM support

  The connector now supports macOS ARM architecture. For more information,
  see the Installation and Configuration Guide.

• [SPARKO-1558] Logging Download Speeds with Cloud Fetch

  With Cloud Fetch enabled, the connector logs average download speed at INFO
  level, in-progress speeds at TRACE level, and slow downloads at WARNING
  level.

• [SPARKO-1561] Kerberos auth support

  The connector now supports Kerberos authentication for Proxy. For more
  information, see the Installation and Configuration Guide.

• [SPARKO-1564] SQL Scripting support

  The connector now supports SQL scripting syntax where scripts are bookended
  by the BEGIN and END keywords.

• [SPARKO-1599] Updated RHEL support

  The connector now supports RHEL 10 operating system. For a list of
  supported Linux versions, see the Installation and Configuration Guide.

• [SPARKO-1607] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.18 (previously 3.0.16)
  • libcURL 8.16.0 (previously 8.12.1)

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector
2.9.4.

• [SPARKO-1516][SPARKO-1518] The connector causes queries to time out after
  20 minutes.

• [SPARKO-1537] The connector uses the 'username' and 'password' properties
  specified in the connection string for authentication in place of -u -p.

Known Issues

The following are known issues that you may encounter due to limitations in
the data source, the connector, or an application.

• [SPARKO-1404] When querying tables that contain VOID columns, the server
  returns an error.

• [SPARKO-1101] When the Auth_AccessToken line length is longer than the
  maximum limit of 1000, the connector returns an authentication error. For
  more information, see the Installation and Configuration Guide.

• [SPARKO-879] When connecting to a server that supports multiple catalogs,
  the connector no longer reports the catalog for schemas and tables as
  SPARK.

  The Spark server now reports the catalog.

• [SPARKO-670] In some cases, when retrieving timestamp data, the connector
  returns an error.

  In some cases, when connecting to certain distributions of
  Apache Spark, the connector returns the following error: "Conversion from
  number to string failed due to undersized character buffer". This issue
  affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

  As a workaround, set EnableArrow=0 in the connection string or DSN.

• [SPARKO-620] Issue with date and timestamp before the beginning of the
  Gregorian calendar when connecting to Spark 2.4.4 or later, or versions
  previous to 3.0, with Arrow result set serialization.

  When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE
  and TIMESTAMP data before October 15, 1582 may be returned incorrectly if
  the server supports serializing query results using Apache Arrow. This
  issue should not impact most distributions of Apache Spark.

  To confirm if your distribution of Spark 2.4.4 or later has been impacted
  by this issue, you can execute the following query:

  SELECT DATE '1581-10-14'
  

  If the result returned by the connector is 1581-10-24, then you are
  impacted by the issue. In this case, if your data set contains date and/or
  timestamp data earlier than October 15, 1582, you can work around this
  issue by adding EnableArrow=0 in your DSN or connection string to disable
  the Arrow result set serialization feature.

• When retrieving data from a BINARY column, a ClassCastException error
  occurs.

  In Spark 1.6.3 or earlier, the server sometimes returns a
  ClassCastException error when attempting to retrieve data from a BINARY
  column.

  This issue is fixed as of Spark 2.0.0.

  For more information, see the JIRA issue posted by Apache named "When
  column type is binary, select occurs ClassCastException in Beeline" at
  https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes =============================================================

The following changes may disrupt established workflows for the connector.

2.9.4 ------------------------------------------------------------------------

• [SPARKO-1545] Removed macOS 11 support

  Beginning with this release, the connector no longer supports macOS 11. For
  information about the supported versions, see the Installation and
  Configuration Guide.

Enhancements & New Features

• [SPARKO-1408] Process Name as Default UserAgentEntry

  The driver will now use the process name as the UserAgentEntry if the UserAgentEntry is not set.

• [SPARKO-1472] Upgraded LTS support

  The driver now uses LTS versions 11.3 through 15.4. For supported versions, see the Installation and Configuration Guide.

• [SPARKO-1482][05221097] Databricks domains support

  The driver now supports cloud.databricks.us and cloud.databricks.mil domains.

• [SPARKO-1325][SPARKO-1435] Timestamp_NTZ support

  The driver now recognizes timestamp_ntz columns in the following data source tables:

  • SQLGetTypeInfo (SQL_ALL_TYPES) now lists timestamp_ntz as an available type.
  • SQLGetTypeInfo (SQL_TIMESTAMP) now lists timestamp_ntz as one of the types that map to SQL timestamps.
  • SQLColumns now identifies timestamp_ntz columns correctly.
  • If UseNativeQuery is set to 0, SQLColAttribute (SQL_DESC_TYPE_NAME) identifies timestamp_ntz columns correctly.

• [SPARKO-1384][SPARKO-1405][SPARKO-1411] CRL cache support

  On Windows, the driver supports the CRL cache when UseSystemTruststore is enabled (set to 1).

• [SPARKO-1399][SPARKO-1417] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.16 (previously 3.0.15)
  • libcURL 8.12.1 (previously 8.11.0)
  • Expat 2.7.1 (previously 2.6.3)

• [SPARKO-1404] VOID type support

  The driver now supports VOID columns and lists them correctly in SQLGetColumns calls.

• [SPARKO-1419][SPARKO-1420] OAuth Token exchange support

  The driver now supports OAuth Token exchange feature for IDP different from the host. In these cases, OAuth access token (including BYOT) will be exchanged for a Databricks in-house access token. For more information, see the Installation and Configuration Guide.

• [SPARKO-1474] Upgraded Windows Server support

  The connector now supports Windows Server 2025. For a list of supported versions, see the Installation and Configuration Guide.

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.2.

• [SPARKO-1516][SPARKO-1518] A memory leak is resolved.

Known Issues

The following are known issues that you may encounter due to limitations in the data source, the connector, or an application.

• [SPARKO-1404] When querying tables that contain VOID columns, the server returns an error.

• [SPARKO-1101] When the Auth_AccessToken line length is longer than the maximum limit of 1000, the connector returns an authentication error. For more information, see the Installation and Configuration Guide.

• [SPARKO-879] When connecting to a server that supports multiple catalogs, the connector no longer reports the catalog for schemas and tables as SPARK.

  The Spark server now reports the catalog.

• [SPARKO-670] In some cases, when retrieving timestamp data, the connector returns an error.

  In some cases, when connecting to certain distributions of Apache Spark, the connector returns the following error: "Conversion from number to string failed due to undersized character buffer". This issue affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

  As a workaround, set EnableArrow=0 in the connection string or DSN.

• [SPARKO-620] Issue with date and timestamp before the beginning of the Gregorian calendar when connecting to Spark 2.4.4 or later, or versions previous to 3.0, with Arrow result set serialization.

  When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE and TIMESTAMP data before October 15, 1582 may be returned incorrectly if the server supports serializing query results using Apache Arrow. This issue should not impact most distributions of Apache Spark.

  To confirm if your distribution of Spark 2.4.4 or later has been impacted by this issue, you can execute the following query:

  SELECT DATE '1581-10-14'
  

  If the result returned by the connector is 1581-10-24, then you are impacted by the issue. In this case, if your data set contains date and/or timestamp data earlier than October 15, 1582, you can work around this issue by adding EnableArrow=0 in your DSN or connection string to disable the Arrow result set serialization feature.

• When retrieving data from a BINARY column, a ClassCastException error occurs.

  In Spark 1.6.3 or earlier, the server sometimes returns a ClassCastException error when attempting to retrieve data from a BINARY column.

  This issue is fixed as of Spark 2.0.0.

  For more information, see the JIRA issue posted by Apache named "When column type is binary, select occurs ClassCastException in Beeline" at https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes

The following changes may disrupt established workflows for the connector.

2.9.2

• [SPARKO-1472] Dropped support for earlier version of LTS

  Beginning with this release, the connector no longer supports the LTS version 10.4. For a list of supported versions, see the Installation and Configuration Guide.

• [SPARKO-1426] Updated default value

  The default values of max catalog name length and max schema name length have been changed from 128 to 1024.

• [SPARKO-1389] Removed support for Ubuntu 20.04

  As early as May 2025, the connector will no longer support Ubuntu 20.04. For information about the supported versions, see the Installation and Configuration Guide.

2.9.1

• [SPARKO-1401] Dropped support for earlier versions of Linux distributions

  The connector no longer supports the following versions of Linux distributions:

  • CentOS 7
  • RHEL 7
  • SLES 12

  For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.0

• [SPARKO-1167] Updated OAuth client ID value

  For the OAuth browser based authentication, the default value of the Client ID property is now "96eecda7-19ea-49cc-abb5-240097d554f5". Previously, the default value was "5e5b7d2b-32b1-4ac0-be5d-ad3ad710fa6e".

• [SPARKO-1155] Removed libcURL folder

  Beginning with this release, on Windows, lib\LibCurl64.DllA (for the 64-bit driver) and lib\LibCurl32.DllA (for the 32-bit driver) folders have been removed.

Version History

2.9.1

Released 2024-11-15

Enhancements & New Features

• [SPARKO-1383] Updated Thrift Version for HTTP User-Agent Header

  The HTTP User-Agent used by the driver has been updated from thrift/0.9.0 to thrift/0.17.0.

• [SPARKO-1005] Driver-Side Timeouts for Idle Statements

  Added support for configuring the fetch result idle timeout. For more information, see the Installation and Configuration Guide.

• [SPARKO-1266] Upgrade Arrow

  Apache Arrow has been upgraded to version 15.0.0 across all non-Windows platforms.

• [SPARKO-1351][SPARKO-1387] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.15 (previously 3.0.13)
  • libcURL 8.11.0 (previously 8.7.1)
  • ICU 74.2 (previously 71.1)
  • Arrow 15.0.0 (previously 9.0.0) for OSX & Linux (Windows was on 15.0.0)
  • lz4 1.10.0. (previously 1.9.4)
  • Expat 2.6.3 (previously 2.6.2)
  • Kerberos 1.21.3 (previously 1.21.2) for OSX & Linux

• [SPARKO-1360] Support for Token Passthrough OAuth

  The driver now supports token renewal for the Token Passthrough OAuth flow.

• [SPARKO-1369] Use 'hive_metastore' as default for Empty Initial Catalog

  If the server sends an empty string as the initial catalog, the driver will default to using the "hive_metastore" catalog.

• [SPARKO-1371] Support for DBR

  The driver now supports DBR 15.4 LTS.

• [SPARKO-1378] Buffer length based on the DM's encoding.

  The driver now returns the buffer length based on the DM's encoding.

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.1.

• [SPARKO-1149] When the driver encounters an error while sending heartbeat, it now re-attempts the heartbeat once before closing the heartbeat thread.

• [SPARKO-1376] HttpPath property missing issue when setting up DSN through Add-OdbcDsn in PowerShell has been fixed.

2.8.2

Released 2024-05-31

Enhancements & New Features

• [SPARKO-1204][SPARKO-1243][SPARKO-1279] Updated authentication support

  The connector now supports Browser Based (U2M) and Client Credentials (M2) authentication on GCP cloud.

  On AWS, Azure, and GCP these are the new default values:

  • Authorization endpoint: hostname + "/oidc/oauth2/v2.0/authorize"
  • Token endpoint: hostname+"/oidc/oauth2/v2.0/token"
  • Client ID (U2M): databricks-sql-odbc
  • Scope (U2M): sql offline_access
  • Scope (M2M): sql

  On Azure, if the application specifies a client ID that is different from the default value, the default scope is:

  • For U2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/user_impersonation, offline_access
  • For M2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default

  You can now configure the OAuth redirect port. To do this, set the OAuth2RedirectUrlPort property to your port. For more information, see the Installation and Configuration Guide.

• [SPARKO-1302] Updated support for Linux distributions

  The connector now supports the following operating systems:

  • Debian 11
  • Ubuntu 24.04

  For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.1

Released 2024-05-31

Enhancements & New Features

• [SPARKO-1136] UC Volume ingestion

  The connector now supports UC Volume ingestion operations. To do this, set the StagingAllowedLocalPaths property to the local path. For more information, see the Installation and Configuration Guide.

• [SPARKO-1264] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Expat 2.6.2 (previously 2.6.0)
  • libcURL 8.7.1 (previously 8.6.0)

Resolved Issues

The following issues have been resolved in Simba Apache Spark ODBC Connector 2.8.1.

• [SPARKO-988] When the comments' length is greater than MaxCommentLen, the connector returns an error.

• [SPARKO-1275] When using a private signed CA on Windows Truststore, the connector fails to verify the OIDC endpoint's certificate.

• [SPARKO-1267] On Windows, when using Kerbeos authentication and KrbRealm is empty, the connection fails.

• [SPARKO-1262] On Windows, when using SASL authentication and the SASL plugins folder path exceeds 128 characters, the connector returns a "Failed to initialize SASL client library: generic failure." error message.

• [SPARKO-1254] The connector returns an error for SQLCancel.

2.8.0

Released 2024-03-11

Enhancements & New Features

• [SPARKO-1137] Enhanced client credentials

  The connector now supports JWT assertion as the client credentials for OAuth. For more information, see the Installation and Configuration Guide.

• [SPARKO-1146] Token renew support

  For token passthrough authentication, you can now renew your token.

• [SPARKO-1147][SPARKO-1171][SPARKO-1236] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • Arrow 15.0.0 (previously 9.0.0)(on Windows)
  • libcURL 8.6.0 (previously 8.4.0)
  • Zlib 1.3.1 (previously 1.2.13)

• [SPARKO-1147] Upgraded compiler support

  The connector now uses Windows Visual Studio 2022. For supported versions, see the Installation and Configuration Guide.

• [SPARKO-1147] Updated security flags

  The following security flags have been used to improve security checks in the connector:

  • /CETCOMPAT
  • /guard:cf
  • /Qspectre
  • /SDL

• [SPARKO-1165] Accept Undetermined Revocation support

  The connector can now access the SSL connection when the certificate's revocation status is undetermined. To do this, select the Accept Undetermined Revocation checkbox (set the AcceptUndeterminedRevocation property to 1). For more information, see the Installation and Configuration Guide.

• [SPARKO-1165] ProxyIgnoreList support

  The connector can now allow the hosts or domains that do not use proxy. To do this, set the ProxyIgnoreList property to a comma separated hostname. For more information, see the Installation and Configuration Guide.

• [SPARKO-1249] OAuth enhancement

  The connector can now connect to the OIDC endpoint when ProxyUID is unset.

• [SPARKO-1155] Updated libcURL location

  On Windows, the libcURL DLL and manifest files are now located in the lib folder.

Resolved Issues

The following issue has been resolved in Simba Apache Spark ODBC Connector 2.8.0.

• [SPARKO-1252] When UseNativeQuery is set to 1 for a cluster later than DBR 11, the connector returns an incorrect column number after SQLPrepare.

Enhancements & New Features

• [SPARKO-1383] Updated Thrift Version for HTTP User-Agent Header

  The HTTP User-Agent used by the driver has been updated from thrift/0.9.0 to thrift/0.17.0.

• [SPARKO-1005] Driver-Side Timeouts for Idle Statements

  Added support for configuring the fetch result idle timeout. For more information, see the Installation and Configuration Guide.

• [SPARKO-1266] Upgrade Arrow

  Apache Arrow has been upgraded to version 15.0.0 across all non-Windows platforms.

• [SPARKO-1351][SPARKO-1387] Updated third-party libraries

  The connector now uses the following third-party libraries:

  • OpenSSL 3.0.15 (previously 3.0.13)
  • libcURL 8.10.1 (previously 8.7.1)
  • ICU 74.2 (previously 71.1)
  • Arrow 15.0.0 (previously 9.0.0) for OSX & Linux (Windows was on 15.0.0)
  • lz4 1.10.0. (previously 1.9.4)
  • Expat 2.6.3 (previously 2.6.2)
  • Kerberos 1.21.3 (previously 1.21.2)

• [SPARKO-1360] Support for Token Passthrough OAuth

  The driver now supports token renewal for the Token Passthrough OAuth flow.

• [SPARKO-1369] Use 'hive_metastore' as default for Empty Initial Catalog

  If the server sends an empty string as the initial catalog, the driver will default to using the "hive_metastore" catalog.

• [SPARKO-1371] Support for DBR

  The driver now supports DBR 15.4 LTS.

• [SPARKO-1378] Buffer length based on the DM's encoding.

  The driver now returns the buffer length based on the DM's encoding.

Resolved Issues

The following issues has been resolved in Simba Apache Spark ODBC Connector 2.9.1.

• [SPARKO-1149] When the driver encounters an error while sending heartbeat, it now re-attempts the heartbeat once before closing the heartbeat thread.

• [SPARKO-1376] HttpPath property missing issue when setting up DSN through Add-OdbcDsn in PowerShell has been fixed.

Workflow Changes =============================================================

The following changes may disrupt established workflows for the connector.

2.9.1 ---------------------------------------------------------------------

• [SPARKO-1401] Dropped support for earlier versions of Linux distributions

  The connector no longer supports the following versions of Linux distributions:

  • CentOS 7
  • RHEL 7
  • SLES 12

  For a list of supported Linux versions, see the Installation and Configuration Guide.