airflow_indexima.hooks.indexima

Indexima hook module definition.

IndeximaHook

IndeximaHook(self, indexima_conn_id:str, connection_decorator:Union[Callable[[airflow.models.Connection], airflow.models.Connection], NoneType]=None, dry_run:Union[bool, NoneType]=False, auth:Union[str, NoneType]=None, kerberos_service_name:Union[str, NoneType]=None, timeout_seconds:Union[int, datetime.timedelta, NoneType]=None, socket_keepalive:Union[bool, NoneType]=None, *args, **kwargs)
Indexima hook implementation.

This implementation can be used as a context manager like this:

with IndeximaHook(...) as hook:
    hook.run('select ...')

This implementation can be customized with a connection_decoratorfunction which must have this profile: Callable[[Connection], Connection] (alias ConnectionDecorator)

In this handler you could retreive credentials from other backeng like aws ssm.

hive_configuration

Return hive configuration.

Returns

(Dict[str, str]): A dictionary of Hive settings (functionally same as the set command)

get_conn

IndeximaHook.get_conn(self) -> pyhive.hive.Connection
Return a hive connection.

Returns

(hive.Connection): the hive connection

get_records

IndeximaHook.get_records(self, sql:str) -> pyhive.hive.Cursor
Execute query and return curror.

(alias of run method)

run

IndeximaHook.run(self, sql:str) -> pyhive.hive.Cursor
Execute query and return curror.

check_error_of_load_query

IndeximaHook.check_error_of_load_query(self, cursor:pyhive.hive.Cursor)
Raise error if a load query fail.

Parameters

  • cursor: cursor returned by load path query.

Raises

  • (RuntimeError): if an error is found

commit

IndeximaHook.commit(self, tablename:str)
Execute a simple commit on table.

Parameters

  • tablename (str): table name to commit

rollback

IndeximaHook.rollback(self, tablename:str)
Execute a simple rollback on table.

Parameters

  • tablename (str): table name to rollback

pause

IndeximaHook.pause(self, pause_in_seconds:int)
Execute a pause statement.

Parameters

  • pause_in_seconds (int): pause delay

close

IndeximaHook.close(self)
Close current connection.