airflow_indexima.hive_transport

Define hive transport function utilities.

PyHive supported authentication mode:

  • NONE
  • CUSTOM
  • LDAP
  • KERBEROS
  • NOSASL

Extra configuration:

  • socket timeout_seconds
  • socket keepalive

create_transport_socket

create_transport_socket(host:str, port:Union[int, NoneType], timeout_seconds:Union[int, NoneType]=None, socket_keepalive:Union[bool, NoneType]=None) -> thrift.transport.TSocket.TSocket
Create a transport socket.

This function expose TSocket configuration option (more for clarity rather than anything else).

Parameters

  • host (str): The host to connect to.
  • port (int): The (TCP) port to connect to.
  • timeout_seconds (Optional[int]): define the socket timeout in second
  • socket_keepalive (Optional[bool]): enable TCP keepalive, default False.

Returns

(TSocket): transport socket instance.

create_hive_plain_transport

create_hive_plain_transport(socket:thrift.transport.TSocket.TSocket, username:str, password:Union[str, NoneType]=None) -> thrift_sasl.TSaslClientTransport
Create a TSaslClientTransport in 'PLAIN' authentication mode.

Parameters

  • socket (TSocket): socket to use
  • username (str): username to login
  • password (Optional[str]): optional password to login

Returns

(TSaslClientTransport): transport instance

create_hive_gssapi_transport

create_hive_gssapi_transport(socket:thrift.transport.TSocket.TSocket, service_name:str) -> thrift_sasl.TSaslClientTransport
Create a TSaslClientTransport in 'GSSAPI' authentication mode.

Parameters

  • socket (TSocket): socket to use
  • service_name (str): kerberos service name

Returns

(TSaslClientTransport): transport instance

create_hive_nosasl_transport

create_hive_nosasl_transport(socket:thrift.transport.TSocket.TSocket) -> thrift.transport.TTransport.TBufferedTransport
Create a TBufferedTransport in 'NOSASL' authentication mode.

NOSASL corresponds to hive.server2.authentication=NOSASL in hive-site.xml

Parameters

  • socket (TSocket): socket to use

Returns

(TBufferedTransport): transport instance

check_hive_connection_parameters

check_hive_connection_parameters(auth:Union[str, NoneType]=None, username:Union[str, NoneType]=None, password:Union[str, NoneType]=None, kerberos_service_name:Union[str, NoneType]=None)
Check hive connection parameters.

Parameters

  • auth (Optional[str]): authentication mode)
  • username (Optional[str]): optional username to login
  • password (Optional[str]): optional password to login
  • kerberos_service_name (Optional[str]): optional service name

Raises

  • (ValueError): if something is wrong

create_hive_transport

create_hive_transport(host:str, port:Union[int, NoneType]=None, timeout_seconds:Union[int, NoneType]=None, socket_keepalive:Union[bool, NoneType]=None, auth:Union[str, NoneType]=None, username:Union[str, NoneType]=None, password:Union[str, NoneType]=None, kerberos_service_name:Union[str, NoneType]=None) -> thrift_sasl.TSaslClientTransport
Create a TSaslClientTransport.

Implementation is heavly based on pyhive.hive.Connection constructor.

Parameters

  • host (str): The host to connect to.
  • port (int): The (TCP) port to connect to.
  • timeout_seconds (Optional[int]): define the socket timeout in second (default 60)
  • socket_keepalive (Optional[bool]): enable TCP keepalive, default off.
  • auth (Optional[str]): authentication mode (Defaul 'NONE')
  • username (Optional[str]): optional username to login
  • password (Optional[str]): optional password to login
  • kerberos_service_name (Optional[str]): optional kerberos service name

Returns

(TSaslClientTransport): transport instance

Raises

  • (ValueError): if something is wrong