Senior Data Engineer, Databricks Platform on AWS

Pci Pharma Services; Rockford (IL); On-site; USD 120,000 - 180,000; Full time

Job summary

PCI Pharma Services is seeking a Senior Data Engineer to own the design, development, and operation of a Databricks-based data platform on AWS. The role focuses on scalable pipelines, secure data models, and robust analytics capabilities to support AI/ML use cases.

You will work with the Data Platform Lead, Product, PM, QA, Customer Success, and external partners to deliver reliable, high-performance data solutions across the organization.

Qualifications

• Senior IC responsible for design, development, optimization, and ops of the enterprise data platform on Databricks and AWS.
• Expertise in Delta Lake, data modeling, ETL/ELT frameworks, and cloud environments.
• Collaborates with Data Platform Lead, Product, Project Management, Testing, Customer Success, and external partners.
• Focus on scalability, security, reliability, and incident resolution.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks (Spark / Delta Lake).
• Build robust ETL/ELT workflows leveraging PySpark and Databricks SQL.
• Implement and support medallion architecture (Bronze, Silver, Gold).
• Design and maintain efficient data models (3NF, star schema).
• Support Data Catalog and Data Governance for data quality and compliance.
• Build and maintain CI/CD pipelines for reliable deployments.
• Collaborate with external partners for development and production support.

Skills

Databricks

PySpark

Delta Lake

Data modeling

ETL/ELT

AWS

CI/CD

Data governance

Spark

Collaboration

Tools

Databricks SQL

AWS Glue

S3

Lambda

Job description

PCI Pharma Services is seeking a Senior Data Engineer to own the design, development, and operation of a Databricks-based data platform on AWS. The role focuses on scalable pipelines, secure data models, and robust analytics capabilities to support AI/ML use cases.

You will work with the Data Platform Lead, Product, PM, QA, Customer Success, and external partners to deliver reliable, high-performance data solutions across the organization.