• nikhilesh@helicaltech.com
  • +91-7893947676

Helical IT Solutions Pvt Ltd

One stop destination for all your BI, DW, Big Data needs

MENUMENU
  • Products
    • Helical Insight
    • Ask On Data
  • Services
    • AWS Quicksight Consulting
    • AWS Glue Consulting
    • Jaspersoft Consulting
    • Talend Consulting
    • Pentaho Consulting
    • Tableau Consulting
    • Pentaho Data Integration
    • Big Data Analytics
    • Data Lake Consulting
  • AI & Analytics
    • Generative AI Services
  • Modern Data Stack
    • DBT Services
    • Airflow Services
    • Airbyte Services
    • Databricks Services
    • Snowflake Services
  • Company
        • About Helical IT Solutions Pvt Ltd

          Helical IT Solutions Pvt Ltd, based out of Hyderabad India, is an IT company specializing in Data Warehousing, Business Intelligence and Big Data Analytics Services. Helical IT Solutions Pvt Ltd can help you in providing consultation regarding selecting of correct hardware and software based on your requirement, data warehouse modeling and implementation, big data implementation, data processing using Apache Spark or ETL tool, building data analysis in the form of reports dashboards with other features like data security, alerting and notification etc. Helical IT Solutions Pvt Ltd can use open source tool leading to low cost of ownership or your favorite proprietary tool to build the solution that you are looking for.

          • About
          • Careers
          • Contact
          • Engagement Models
          • Clients
          • Blog
  • Clients
  • Contact
        • Location

          View larger map
          • Helical IT Solutions Pvt Ltd.
            Somajiguda, Hyderabad

          • nitin@helicaltech.com
            nikhilesh@helicaltech.com

          • +91-7893947676

        • Contact Us

      • Search what you are looking for..

    Fetching AWS Glue Connection Details

    Posted on November 12, 2019 by By Sai Kavya Sathineni, in AWS | 0

    There might be requirements where you have to fetch the details from the connection defined in AWS Glue Data Catalog and use them in your Glue job for connecting to the database (to store logs into database).

    Make data easy with Helical Insight.
    Helical Insight is the world’s best open source business intelligence tool.

      Below are the steps to achieve this:

    1. Create a AWS Glue connection
      • Refer Create AWS Glue Connection
    2. Prepare a py.file (getConnection.py) to fetch the details
      • This .py file uses a function called extract_jdbc_conf(connection_name, catalog_id = None)
      • extract_jdbc_conf() returns a dictionary with keys user, password, vendor and url from connection object in the Data Catalog
      • Arguments :
        • connection_name – The name of the connection in the Data Catalog
        • catalog_id – The catalog ID (account ID) of the Data Catalog being accessed. When None, the default account ID of the caller is used.
      • Use the following code
      class fetch_connection:
      	@staticmethod
      	def getConnectionDetails(glueContext,db,catalog_id):
      		details = glueContext.extract_jdbc_conf(db, catalog_id = catalog_id)
      		source = {}
      		source['url'] = details['url']
      		source['user'] = details['user']
      		source['password'] = details['password']
      		url = details['url']
      		source['host'] = url[url.find("//")+2:-5]
      		source['port'] = url[url.rfind(":")+1:]
      		return source
      
    3. Add this .py file in a s3 bucket
    4. s3 bucket

    5. Provide the s3 Location of the .py file in the Security configuration, script libraries and job parameters in Python library path in Edit Job
    6. s3 bucket

    7. Provide the following code in the job
    8. 	import getConnection as gc
      
      CONNECTION = “<Glue Connection Name>”
      conn = gc.fetch_connection.getConnectionDetails(glueContext,CONNECTION,None)
      
      HOST_NAME = conn['host']
      USERNAME = conn['user']
      PASSWORD = conn['password']
      PORT = conn['port']
      DATABASE = “<Database name>”
      URL = conn['url']+"/"+DATABASE
      DRIVER = "org.postgresql.Driver"
      
      Note- The above code is for Postgres
      
      Use the below code for SQL Server
      URL = conn['url']
      URL = SOURCE_URL+";databaseName="+SOURCE_DB_NAME
      USERNAME = conn['user']
      PASSWORD = conn['password']
      PORT="1433"
      DRIVER = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
      
    9. Execute the job and now the job will fetch the connection details

    Make data easy with Helical Insight.
    Helical Insight is the world’s best open source business intelligence tool.

    Get your 30 Days Trail Version

    s3 bucket

    In case if you have any queries please get us at support@helicaltech.com

    Thank You
    Sai Kavya Sathineni
    BI Developer
    Helical IT Solutions Pvt Ltd

    logo

    Best Open Source Business Intelligence Software Helical Insight Here

    logo

    A Business Intelligence Framework


    logo

    Best Open Source Business Intelligence Software Helical Insight is Here

    logo

    A Business Intelligence Framework


    Accessing Data using JDBC on AWS Glue Example Tutorial AWS AWS glue aws glue connection example aws glue jdbc connection example aws glue jdbc connection properties BI Business Intelligence Fetching AWS Glue Connection Details glue connection How do I check my AWS connection How To Connect Amazon Glue to a JDBC Database How to connect AWS RDS SQL Server with AWS Glue What is Connections in AWS Glue Where does AWS Glue store metadata
    0 0 votes
    Article Rating
    Subscribe
    Connect with
    Login
    I allow to create an account
    When you login first time using a Social Login button, we collect your account public profile information shared by Social Login provider, based on your privacy settings. We also get your email address to automatically create an account for you in our website. Once your account is created, you'll be logged-in to this account.
    DisagreeAgree
    Notify of


    I allow to create an account
    When you login first time using a Social Login button, we collect your account public profile information shared by Social Login provider, based on your privacy settings. We also get your email address to automatically create an account for you in our website. Once your account is created, you'll be logged-in to this account.
    DisagreeAgree
    0 Comments
    Inline Feedbacks
    View all comments

    You might also like..

    Helical Insight

    Helical IT Solutions Launches Helical Insight 5.2.2 : Focus on Advance Embedded Analytics

    By admin

    24 Dec 2024: Helical IT Solutions is excited to unveil Helical Insight 5.2.2, the latest iteration of its cutting-edge Open Source Business Intelligence (BI) platform. This release reinforces Helical Insight's position as a cost-effective, versatile, and powerful alternative to mainstream...
    • December 26, 2024
    • 0
    Helical Insight 5.2.1

    Helical IT Solutions Launches Helical Insight 5.2.1: Elevating Open Source BI to New Heights

    By admin

    02 Sept 2024 – Helical IT Solutions is thrilled to announce the release of Helical Insight version 5.2.1, the latest upgrade to its Open Source Business Intelligence (BI) platform. This new version delivers a powerful, cost-effective BI solution that is...
    • September 2, 2024
    • 0
    Business Intelligence

    Installation of Firebird db

    By admin

    Steps to install firebird db 1. Go to google and type firebird in search box and then click on first link. License aggrement 2. Click on downloads and then install Firebird latest version(5.0.0). 3. It will navigate to the below...
    • April 30, 2024
    • 0

    Contact Form

      Let’s get started to unravel your best-fit solution.

      ABOUT HELICAL IT SOLUTIONS PVT LTD

      Helical IT Solutions Pvt Ltd specializes in Data Warehousing, Business Intelligence and Big Data Analytics. We offer consultation in selection of correct hardware and software as per requirement, implementation of data warehouse modeling, big data, data processing using Apache Spark or ETL tools and building data analysis in the form of reports and dashboards with supporting features such as data security, alerts and notification, etc.

      Learn More

      Blog

      Helical Blog

        

      Recent Posts

      • Helical IT Solutions Launches Helical Insight 5.2.2 : Focus on Advance Embedded Analytics December 26, 2024
      • Helical IT Solutions Launches Helical Insight 5.2.1: Elevating Open Source BI to New Heights September 2, 2024
      • Installation of Firebird db April 30, 2024

      © 2025 Helical IT Solutions Pvt Ltd

      0
      0Shares
      wpDiscuz
      Join Us