Senior Data Engineer: Databricks & AWS Platform

pciservices; Philadelphia (Philadelphia County); On-site; USD 130,000 - 190,000; Full time

Job summary

PCI Pharma Services is seeking a Senior Data Engineer to own hands-on design, development, and optimization of our enterprise data platform built on Databricks and AWS. You will ensure scalable, secure pipelines and models powering analytics and AI/ML use cases.

Expect deep work with Delta Lake, data modeling, ETL/ELT, and cloud environments. Collaboration with the Data Platform Lead, Product, PM, Testing, Customer Success, and external partners is essential to success.

Qualifications

• Hands-on design, development, optimization, and support of an enterprise data platform on Databricks and AWS.
• Deep expertise in Delta Lake architecture, data modeling, ETL/ELT frameworks, and cloud environments (AWS).
• Focus on execution excellence, platform reliability, and technical problem resolution with cross‑functional collaboration.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks (Spark / Delta Lake).
• Build robust ETL/ELT workflows leveraging PySpark and Databricks SQL.
• Implement and support medallion architecture (Bronze, Silver, Gold).
• Design and maintain efficient data models, including 3NF, star schema, and canonical models.
• Support and maintain Data Catalog and Data Governance frameworks to ensure data quality and compliance.
• Build and maintain CI/CD pipelines to enable reliable deployment across environments.
• Collaborate with external partners for development activities and ongoing production support.

Skills

Databricks

Spark

Delta Lake

PySpark

Data modeling

ETL/ELT

AWS

Data governance

CI/CD

Databricks SQL

Tools

AWS Glue

S3

Lambda

Databricks SQL

Job description

PCI Pharma Services is seeking a Senior Data Engineer to own hands-on design, development, and optimization of our enterprise data platform built on Databricks and AWS. You will ensure scalable, secure pipelines and models powering analytics and AI/ML use cases.

Expect deep work with Delta Lake, data modeling, ETL/ELT, and cloud environments. Collaboration with the Data Platform Lead, Product, PM, Testing, Customer Success, and external partners is essential to success.