Syntax error How to use Boto3 to get the details of all connection available in AWS Glue Data catalog?

How to use Boto3 to get the details of all connection available in AWS Glue Data catalog?

Problem Statement − Use boto3 library in Python to get details of all connection present in AWS Glue Data catalog.

Example − Get the details of all connection definition.

Approach/Algorithm to solve this problem

Step 1 − Import boto3 and botocore exceptions to handle exceptions.

Step 2 − There is no parameter.

Step 3 − Create an AWS session using boto3 library. Make sure region_name is mentioned in default profile. If it is not mentioned, then explicitly pass the region_name while creating the session.

Step 4 − Create an AWS client for glue.

Step 5 − Call get_connections function.

Step 6 − It will fetch details of the connection definition from AWS Glue Data Catalog.

Step 7 − Handle the generic exception if something went wrong while checking the job.

Example

Use the following code to get definition of all connections in AWS Glue Data catalog −

import boto3
from botocore.exceptions import ClientError

def get_details_of_all_connections():
   session = boto3.session.Session()
   glue_client = session.client('glue')
   try:
      response = glue_client.get_connections()
      return response
   except ClientError as e:
      raise Exception("boto3 client error in get_details_of_all_connections: " + e.__str__())
   except Exception as e:
      raise Exception("Unexpected error in get_details_of_all_connections: " + e.__str__())
print(get_details_of_all_connections())

Output

{'ConnectionList': [
{'Name': '01_Daily', 'Description': '', 'ConnectionType': 'JDBC',
'ConnectionProperties': {'JDBC_CONNECTION_URL':
'jdbc:redshift://**********.us-east-1.redshift.amazonaws.com:5439/abc',
'JDBC_ENFORCE_SSL': 'false', 'PASSWORD': '!*******', 'USERNAME':
'********'}, 'PhysicalConnectionRequirements': {'SubnetId': 'subneta******e', 'SecurityGroupIdList': ['sg-********3'], 'AvailabilityZone':
'us-east-1a'}, 'CreationTime': datetime.datetime(2020, 12, 11, 17, 1,
51, 519000, tzinfo=tzlocal()), 'LastUpdatedTime':
datetime.datetime(2020, 12, 11, 17, 1, 51, 519000, tzinfo=tzlocal())},
{'Name': 'aurora-poc', 'ConnectionType': 'JDBC', 'ConnectionProperties':
{'JDBC_CONNECTION_URL': 'jdbc:postgresql://**********-cluster.clustercv********6p.us-east-1.rds.amazonaws.com:5432/*******,
'JDBC_ENFORCE_SSL': 'false', 'PASSWORD': '******', 'USERNAME': user'},
'PhysicalConnectionRequirements': {'SubnetId': 'subnet-35******e',
'SecurityGroupIdList': ['sg-*******d', 'sg-0***********'],
'AvailabilityZone': 'us-east-1c'}, 'CreationTime':
datetime.datetime(2020, 11, 18, 12, 38, 29, 625000, tzinfo=tzlocal()),
'LastUpdatedTime': datetime.datetime(2020, 11, 18, 12, 51, 16, 59000,
tzinfo=tzlocal())},
{'Name': 'dev-ods', 'ConnectionType': 'JDBC', 'ConnectionProperties':
{'JDBC_CONNECTION_URL': 'jdbc:postgresql://*****************.us-east1.rds.amazonaws.com:5432/store', 'JDBC_ENFORCE_SSL': 'false',
'PASSWORD': '***********', 'USERNAME': user'},
'PhysicalConnectionRequirements': {'SubnetId': 'subnet-a********',
'SecurityGroupIdList': ['sg-*********b7', 'sg-a8********e'],
'AvailabilityZone': 'us-east-1a'}, 'CreationTime':
datetime.datetime(2020, 5, 27, 3, 10, 24, 538000, tzinfo=tzlocal()),
'LastUpdatedTime': datetime.datetime(2021, 2, 26, 5, 50, 53, 991000,
tzinfo=tzlocal())},],
'ResponseMetadata': {'RequestId': '58bae80d-*******************87',
'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Sun, 28 Feb 2021
11:21:55 GMT', 'content-type': 'application/x-amz-json-1.1', 'contentlength': '11568', 'connection': 'keep-alive', 'x-amzn-requestid':
'58bae80d-************************87'}, 'RetryAttempts': 0}}
Updated on: 2021-03-23T06:30:43+05:30

379 Views

Kickstart Your Career

Get certified by completing the course

Get Started
Advertisements