Senior Data Engineer

Elliott Group; Jeannette (Westmoreland County); Presencial; USD 120.000 - 160.000; Jornada completa

Descripción de la vacante

Elliott Group in Jeannette, PA seeks a Senior Data Engineer to build and maintain robust data pipelines moving data into the Enterprise Data Lakehouse. You will design complex workflows, troubleshoot transformations and collaborate with analysts and data scientists to enable new data sources.

The role requires extensive ETL experience, Databricks proficiency, and strong cloud/database knowledge in a fast-paced environment.

Formación

• Bachelors degree in CS/IS/DS required.
• Minimum 10 years hands-on ETL engineering experience.
• Strong experience with Databricks and data pipelines.

Responsabilidades

• Design and build robust ETL pipelines for the Enterprise Data Lakehouse.
• Architect and troubleshoot complex workflows and data transformations.
• Monitor data flow across databases, cloud and on-prem environments.
• Profile data, detect anomalies and improve data quality.
• Collaborate with data analysts, visualization engineers and data scientists.

Conocimientos

ETL engineering

Databricks

Cloud computing (AWS)

Data modeling

Informatica

Fivetran

Azure Data Factory

SQL/Oracle/SQL Server

Python basics

APIs & ODBC data sources

Cross-functional collaboration

Troubleshooting complex data pipelines

Educación

Bachelor's degree in Computer Science / Information Systems / Data Science

Herramientas

Databricks

Informatica

Fivetran

Azure Data Factory

Oracle

SQL Server

ODBC

Descripción del empleo

## Senior Data EngineerApply: Jeannette, PA: Full time: Posted Yesterday: R4033**Overview & Responsibilities**The Data Engineer will build robust and complicated data pipelines required for moving data from various source systems into the Enterprise Data Lakehouse. The Data Engineer will be responsible for architecting, troubleshooting and designing complex workflows and transformations. **BACKGROUND and EXPERIENCE*** Bachelor's in Computer Science, Information Systems, Data Science, Analytics.* Minimum 10 years hands on experience as an ETL engineer.* 3+ years experience on Databricks is required.* Well versed with using various database systems like Oracle, SQL Server etc.* Experience in the cloud preferably AWS, SQL, Data modeling, Informatica, Fivetran or Azure Data Factory.* Deep understanding of extracting data from ODBC sources along with usage of APIs.* Basic Python skills necessary.* Should be able to work with various applications and business units, along with the Data analysts, visualization engineer and data scientist.* Requires sophisticated skills to understand, influence, drive, and negotiate with internal customers.* Uses initiative and extensive knowledge to resolve complex issues, developing new or novel solutions. **SPECIFIC JOB RESPONSIBILITIES/COMPETENCIES*** Review source to target mapping documents.* Design and document ETL pipelines using industry standard ETL tools.* Streamline and build robust data pipelines, with clear visibility into lineage of the data.* Responsible for monitoring the flow of the data across various databases, cloud and on premise.* Implement best practices and standardize data integration across all the sources.* Profile and detect data anomalies and provide solutions for improving the quality of the data.* Build and implement Master Data Management processes, while defining algorithms and building code for cleansing the data.* Establish highly robust and efficient data flows with logging and efficient troubleshooting methods.* Be able to work in a fast pace and changing environment.* Consulted by others for their ability to identify, understand and analyze complex business concepts.* Is forward thinking in approach and considers the effect of current action on future company outcomes.* Requires the ability to work independently and to provide consultative advice on specific technical issues.* Develops new and sometimes novel resolution of complex issues and works on all application and data integration issues.* This position requires regular interaction with internal business users from all departments including Operations, Sales, Marketing, and Finance across all of the global locations of the organization. It will also work closely with the Visualization engineer, data analysts, IT and Data scientist to enable new data sources to the pipeline.