Senior Data Engineer

Elliott Company; Jeannette (Westmoreland County); On-site; USD 140,000 - 190,000; Full time

Job summary

Ebara Elliott Energy is seeking a Data Engineer in the United States to build robust data pipelines and data lakehouse workflows. You will architect, troubleshoot and design complex data transformations, collaborating with data analysts, visualization engineers, and data scientists across global locations.

The role requires strong ETL, Databricks, cloud experience (AWS/Azure), and expertise with Oracle/SQL Server databases.

Qualifications

• Bachelor's degree in CS/IS/Data Science or related field.
• Minimum 10 years hands-on ETL experience.
• 3+ years Databricks experience.
• Experience with Oracle/SQL Server databases.
• Cloud experience with AWS or Azure Data Factory.
• Deep understanding of extracting data from ODBC sources and APIs.
• Basic Python skills.
• Ability to work with multiple business units and data teams.
• Strong problem-solving and stakeholder negotiation skills.
• Independent consultant capable of resolving complex issues.

Responsibilities

• Review source-to-target mapping documents.
• Design and document ETL pipelines with industry standard tools.
• Streamline and build robust data pipelines with clear data lineage.
• Monitor data flows across databases, cloud and on-premise.
• Implement best practices and standardize data integration across sources.
• Profile data, detect anomalies and improve data quality.
• Build Master Data Management processes and cleansing code.
• Establish robust data flows with logging and troubleshooting.
• Collaborate with analysts, visualization engineers and data scientists.
• Interact with internal users across operations, sales, marketing and finance.

Skills

ETL pipelines

Databricks

Oracle Database

SQL Server

AWS

Data modeling

Informatica

Fivetran

Azure Data Factory

Python

APIs

Cross-functional collaboration

Education

Bachelor's degree in CS/IS/Data Science

Tools

Databricks

ODBC

SQL Server

Oracle DB

AWS Cloud

Informatica

Job description

Overview & Responsibilities

The Data Engineer will build robust and complicated data pipelines required for moving data from various source systems into the Enterprise Data Lakehouse. The Data Engineer will be responsible for architecting, troubleshooting and designing complex workflows and transformations.

BACKGROUND and EXPERIENCE

• Bachelor's in Computer Science, Information Systems, Data Science, Analytics.
• Minimum 10 years hands on experience as an ETL engineer.
• 3+ years experience on Databricks is required.
• Well versed with using various database systems like Oracle, SQL Server etc.
• Experience in the cloud preferably AWS, SQL, Data modeling, Informatica, Fivetran or Azure Data Factory.
• Deep understanding of extracting data from ODBC sources along with usage of APIs.
• Basic Python skills necessary.
• Should be able to work with various applications and business units, along with the Data analysts, visualization engineer and data scientist.
• Requires sophisticated skills to understand, influence, drive, and negotiate with internal customers.
• Uses initiative and extensive knowledge to resolve complex issues, developing new or novel solutions.

SPECIFIC JOB RESPONSIBILITIES/COMPETENCIES

• Review source to target mapping documents.
• Design and document ETL pipelines using industry standard ETL tools.
• Streamline and build robust data pipelines, with clear visibility into lineage of the data.
• Responsible for monitoring the flow of the data across various databases, cloud and on premise.
• Implement best practices and standardize data integration across all the sources.
• Profile and detect data anomalies and provide solutions for improving the quality of the data.
• Build and implement Master Data Management processes, while defining algorithms and building code for cleansing the data.
• Establish highly robust and efficient data flows with logging and efficient troubleshooting methods.
• Be able to work in a fast pace and changing environment.
• Consulted by others for their ability to identify, understand and analyze complex business concepts.
• Is forward thinking in approach and considers the effect of current action on future company outcomes.
• Requires the ability to work independently and to provide consultative advice on specific technical issues.
• Develops new and sometimes novel resolution of complex issues and works on all application and data integration issues.
• This position requires regular interaction with internal business users from all departments including Operations, Sales, Marketing, and Finance across all of the global locations of the organization.
• It will also work closely with the Visualization engineer, data analysts, IT and Data scientist to enable new data sources to the pipeline.

Equal Employment Opportunity

Ebara Elliott Energy is an equal employment opportunity/affirmative action employer and does not discriminate on the basis of race, colour, religion, sex, national origin, age, marital status, genetic information, disability, veteran status, or any other characteristic protected by the federal, state or local laws of the United States. Applicants and employees are protected under U.S. federal law from discrimination.

We are proud of our legacy, but our focus is on the future, putting our tradition of innovation to work on the new challenges facing global energy production and processes. Using advanced technologies, producing high-quality products, and offering reliable services are just a few of the ways that Ebara Elliott Energy is doing its part to contribute to a cleaner, more sustainable future.

As a solution provider in the energy industry, Ebara Elliott Energy designs, manufactures, and services technically advanced centrifugal and axial compressors, steam turbines, power recovery expanders, custom pumps, and cryogenic products used in the petrochemical, refining, oil & gas, and liquefied gas industries, as well as in power applications.

We employ 2,300 people in 30 locations around the world. The company is a wholly owned subsidiary of Ebara Corporation, a major industrial conglomerate headquartered in Tokyo, Japan. Ebara employs 19,000 people and has sales of US $5 billion.