Senior Data Engineer - Databricks & AI-Driven Pipelines

Federated Hermes, Inc.; Warrendale (Allegheny County); Hybrid; USD 130,000 - 180,000; Full time

Job summary

Federated Hermes, Inc. seeks an Engineer to join the Enterprise Data Team at its Warrendale, PA office. You will lead design, development, and optimization of data solutions on the Databricks platform, mentoring colleagues and establishing standards for the team’s data work.

The role requires hands-on leadership in data engineering, strong Python/Spark skills, and experience with metadata-driven ingestion, CI/CD, and cloud platforms. Hybrid in-office/remote schedule in Warrendale, PA.

Qualifications

• Minimum of 5 years of programming experience, including at least 1 year on a big data platform.
• Bachelor’s degree in CS, engineering, or related field; continuing education preferred.
• Experience with CI/CD pipelines and version control tools.

Responsibilities

• Lead design, development, and optimization of data solutions on the Databricks platform.
• Mentor junior team members and establish coding standards for the data team.
• Design and implement batch and streaming data workflows; monitor Spark jobs and cluster usage.
• Ensure data governance, security, and privacy standards are followed.

Skills

Databricks

PySpark

SQL

Data modeling

Dimensional modeling

Medallion architecture

Unity Catalog

Delta Live Tables

Delta Sharing

Python

Cloud platforms

CI/CD

Version control

BI tools

Education

Bachelor’s degree in computer science, engineering, or a related field

Tools

Databricks

Unity Catalog

Delta Live Tables

Delta Sharing

PySpark

Job description

Federated Hermes, Inc. seeks an Engineer to join the Enterprise Data Team at its Warrendale, PA office. You will lead design, development, and optimization of data solutions on the Databricks platform, mentoring colleagues and establishing standards for the team’s data work.

The role requires hands-on leadership in data engineering, strong Python/Spark skills, and experience with metadata-driven ingestion, CI/CD, and cloud platforms. Hybrid in-office/remote schedule in Warrendale, PA.