Senior Data Engineer — Spark, Databricks & Pipelines

Federated Hermes, Inc.; Warrendale (Allegheny County); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Bonus program

Paid time off

Professional development

Job summary

Federated Hermes is seeking a senior data engineer to design and maintain scalable data pipelines on Databricks, with strong PySpark/SQL and data modeling skills. The role emphasizes data quality, governance, and secure handling of batch and streaming workloads, in a hybrid in-office/remote setup at Warrendale, PA.

You will contribute to architecture, drive best practices, and collaborate across the Global Technology Organization, leveraging Azure cloud and BI tools to empower analytics across

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Minimum of 5 years programming experience including: at least 1 year working with big data platform.
• Experience in data engineering domain, Python, SQL, and cloud platforms such as Azure.
• Familiarity with data lakehouse principles, relational and Kimball data models.
• Experience with CI/CD pipelines and version control tools.
• Knowledge of data visualization tools and BI platforms preferred.
• Databricks certification or relevant cloud platforms certification preferred.

Responsibilities

• Design, build, and maintain scalable data pipelines on Databricks (Spark, Delta Lake).
• Write clean PySpark or SQL code for data transformation.
• Design robust data models for analytics and reporting.
• Ensure data quality, consistency, and governance.
• Handle batch and streaming data workflows.
• Provide architectural guidance and support in platform usage.
• Drive best practices in data engineering across the team.
• Monitor and optimize Spark job and cluster performance.
• Ensure compliance with security and data privacy standards.

Education

Bachelor’s degree in Computer Science, Engineering, or a related field

Tools

Databricks

Python

SQL

Azure

CI/CD

Version control

Data visualization tools

Job description

Federated Hermes is seeking a senior data engineer to design and maintain scalable data pipelines on Databricks, with strong PySpark/SQL and data modeling skills. The role emphasizes data quality, governance, and secure handling of batch and streaming workloads, in a hybrid in-office/remote setup at Warrendale, PA.

You will contribute to architecture, drive best practices, and collaborate across the Global Technology Organization, leveraging Azure cloud and BI tools to empower analytics across