Data Pipeline Engineer - AWS & Databricks Delta Lake

attaintalent; États-Unis; Sur place; USD 63 000 - 92 000; Plein temps

Avantages offerts par ce poste

Medical coverage

401(k) matching

PTO & holidays

Professional development

Life and disability insurance

Résumé du poste

Attain Talent seeks a Development Engineer to support CMS modernization, building scalable data pipelines across AWS and Databricks with Delta Lake lakehouse architectures.

You will ingest, transform, and process data for analytics and reporting, collaborate with architects, QA, and business analysts, and contribute to data governance and CI/CD integration.

Competitive benefits and compensation reflect experience; base salary ranges from 62,625 to 91,850 USD depending on qualifications.

Qualifications

• Design, build, and maintain robust ETL/ELT pipelines for analytics and reporting.
• Develop data processing workflows using Python, SQL, and PySpark in Databricks/Delta Lake.
• Implement scalable lakehouse architectures for healthcare data and enterprise analytics.
• Collaborate with Data Architects, Automation Test Engineers, QA Analysts, and Business Analysts to define data transformation standards.
• Build reusable code for data ingestion, data quality validation, and exception handling.
• Optimize data processing jobs for performance, reliability, and cost in AWS and Databricks.

Responsabilités

• Ingest and transform data for analytics and reporting.
• Improve pipeline reliability and performance.
• Collaborate with cross-functional teams including Architects, QA, and Business Analysts.

Connaissances

Python

SQL

PySpark

Databricks

Delta Lake

AWS

Data pipelines

CI/CD

Data modeling

Data governance

Outils

GitHub

Databricks Repos

Delta Lake

Description du poste

Attain Talent seeks a Development Engineer to support CMS modernization, building scalable data pipelines across AWS and Databricks with Delta Lake lakehouse architectures.

You will ingest, transform, and process data for analytics and reporting, collaborate with architects, QA, and business analysts, and contribute to data governance and CI/CD integration.

Competitive benefits and compensation reflect experience; base salary ranges from 62,625 to 91,850 USD depending on qualifications.