ETL Data Engineer

AHU Technologies Inc; Washington (District of Columbia); On-site; USD 120,000 - 180,000; Full time

Job summary

AHU Technologies Inc. seeks an ETL Data Engineer to support the Enterprise Data team in Washington, DC, onsite.

You will design and implement ETL pipelines using Azure Data Factory, Synapse, and Databricks to load and transform data for the Cloud Data Exchange. The role requires 16 years of experience, a Bachelor’s degree in IT or related field, strong SQL, data warehousing knowledge, and hands-on Azure experience.

Qualifications

• Bachelor’s degree in IT, related field, or equivalent experience.
• 16 years of experience in ETL/data engineering.
• Strong knowledge of end-to-end ETL processes and pipelines.
• Experience with Azure Data Factory, Synapse, and Databricks.
• Familiarity with data warehousing concepts and data governance.

Responsibilities

• Analyzes, designs, develops, implements, replicates, and supports complex enterprise data projects.
• Interfaces with agencies and stakeholders to define requirements and advise on environment constraints for cloud solutions.
• Develops ETL pipelines using Azure Data Factory, Synapse, and Databricks; handles data loading and transformation.
• Establishes cloud and on-prem connectivity (ADLS, ADF, Synapse, Databricks).
• Hands-on work with Azure services like Azure Data Factory, Synapse, MSSQL/SQL DB, and Python.
• Creates end-to-end data pipelines from multiple sources to landing layers or SQL tables.
• Familiar with Informatica, NiFi, Airflow, and other data integration tools.
• Works with various data formats (CSV, XML, Parquet, AVRO) across databases (Oracle, SQL Server, DB2).
• Supports data governance, security, and reliability, and monitors pipelines.

Skills

ETL development

SQL

Azure Data Factory

Azure Synapse

Databricks

Data modeling

Data governance

Informatica PowerCenter

Apache NiFi

Apache Airflow

Tableau

Power BI

MicroStrategy

Esri ArcGIS

Education

Bachelor's degree in IT or related field

Tools

Azure Data Factory

Azure Synapse

SQL Database

Databricks

Informatica PowerCenter

Apache NiFi

Apache Airflow

MicroStrategy

Tableau

Esri ArcGIS

Job description

TITLE

ETL Data Engineer

LOCATION

Washington, DC/ Onsite

Employment type

Contract w2/1099/C2C

Minimum Education

Bachelor’s degree in IT, related field, or equivalent experience.

Required Experience

16 years

Interviews

Webcam and In Person

Job Description

The Enterprise Data team at client requires an ETL data engineer to support data operations for its Cloud Data Exchange. The resource will utilize native Azure tools to perform ETL, data loading and data transformation tasks.

Employment type

Contract w2/1099/C2C

Complete Description

The ETL data engineer will support the client Enterprise Data team for data curation, processing, and transformation tasks. Specifically, the ETL data engineer will be responsible for the following tasks:

Responsibilities

• Analyzes, designs, develops, implements, replicates, and supports complex enterprise data projects.
• Interfaces with other agencies, consult with and inform user departments on system requirements, advise on environment constraints and operating difficulties for the current state and advise and resolve problems using cloud solutions and develop and replicate future enhancements to District’s data systems.
• Strong knowledge of Extraction, Transformation and Loading (ETL) processes using frameworks like Azure Data Factory, Synapse, Databricks, Informatica by gathering requirements from the stake holders or analyzing the existing code and perform enhancements or new development.
• Establishing the cloud and on-premise connectivity in different systems like ADLS, ADF, Synapse, Databricks
• Hands on experience in Azure cloud services like Azure Data Factory or Azure Synapse, MSSQL Db, Azure SQL DB, Azure Data Lake Storage Gen2, Blob Storage, Python etc.
• Worked on creating end to end pipelines to load data by reading it from multiple sources or source systems and load to landing layer or SQL tables.
• Familiarity/experience with data integration and data pipeline tools (e.g., Informatica, Synapse, Apache NiFi, Apache Airflow)
• Familiarity/experience with various data formats including database specific (Oracle, SQL Server, DB2, Quickbase), text formats(CSV, XML) and Binary(Parquet, AVRO)
• Develops, standardizes and optimizes existing data workflow/pipelines adhering to best practices.
• Adhere and contribute to enterprise data governance standards by ensuring data accuracy, consistency, security, and reliability.
• Automates, monitors, alerts, and manages data pipelines and workflows.
• Analyzes and evaluates system changes to determine feasibility, provides alternative solutions, back-up, and rollback procedures.
• Works on the development of new systems, upgrades and enhancements to existing systems and ensure systems follow approved standards and consistency after the changes.
• Develops complex programs and reports in database query language.
• Familiarity/experience with data visualization tools.

Skills

• Strong knowledge for development of Extract-Transform-Load (ETL) processes, including end-to-end pipelines with data loading from multiple sources. 15 Years Required
• Ability to gather and document requirements for data extraction, transformation and load processes. 15 Years Required
• Understanding of data warehousing, data lake, business intelligence and information management concepts and standards. 15 Years Required
• Ability to advise internal and external customers on appropriate tools and systems for complex data processing challenges. 15 Years Required
• Knowledge and use of SQL for relational databases. 11 Years Required
• Experience with various data formats including database specific (Oracle, SQL, Postgres, DB2), text formats (CSV, XML) and Binary (Parquet, AVRO). 11 Years Required
• Strong experience with Microsoft Azure tools such as Azure Data Factory, Synapse, SQL Database, Data Lake Storage Gen2, Blob Storage. 5 Years Required
• Contribute to enterprise data governance standards by ensuring accuracy, consistency, security and reliability. 7 Years Required
• Experience with data integration and data pipeline tools such as Informatica PowerCenter, Apache NiFi, Apache Airflow and FME. 5 Years Required
• Experience with visualization and reporting software such as MicroStrategy, Tableau, and Esri ArcGIS. 3 Years Highly Desired
• Strong communication skills, both oral and written. 3 Years Required
• Ability to provide excellent customer service to external partners. 3 Years Required
• Ability to work independently or as part of a larger team. 3 Years Required
• Experience performing data functions with Databricks. 3 Years Highly Desired