Data Platform Engineer – AWS/Databricks Delta Lake

Attain Talent; United States; Remote; USD 63,000 - 92,000; Full time

Benefits offered by this job

Health Insurance

Dental Insurance

Vision Insurance

Telehealth

Employee Assistance Program

Paid Time Off 3.5 weeks

11 Holidays

401(k) Retirement Plan

Employer 4% 401k match

Life Insurance

Disability Insurance

Tuition Reimbursement

Job summary

Attain Talent seeks a Development Engineer to support CMS modernization, building scalable data pipelines for analytics and reporting. You will ingest, transform, and process data across AWS and Databricks, emphasizing Delta Lake lakehouse architectures.

The role requires strong Python/SQL/PySpark skills, experience with ETL/ELT, data governance, and collaborating with architects, QA, and business analysts.

Qualifications

• Experience building robust ETL/ELT pipelines for analytics and reporting.
• Proficient Python, SQL, and PySpark in Databricks environments.
• Experience with Delta Lake and data lakehouse architectures.
• Ability to design data models, schemas, and data lineage/documentation.
• Experience implementing CI/CD for data pipelines using GitHub.
• Familiarity with AWS data processing in cloud environments.
• Collaborate with Data Architects and QA Analysts to define transformation logic.
• Strong focus on data quality, governance, and security in healthcare data.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines for analytics.
• Develop data processing workflows with Python, SQL, and PySpark.
• Implement scalable lakehouse architectures with Delta Lake on Databricks.
• Collaborate with data teams to define transformation logic and standards.
• Optimize jobs for performance, reliability, and cost in AWS/Databricks.
• Contribute to data governance, metadata management, and lineage.
• Support CI/CD integration of pipelines with GitHub and Databricks.
• Investigate and resolve data issues across environments.

Skills

ETL/ELT pipelines

Python

SQL

PySpark

Data lakehouse

AWS

Data governance

Data lineage

CI/CD

Schema design

Data modeling

Tools

Databricks

Delta Lake

GitHub

Databricks Repos

Job description

Attain Talent seeks a Development Engineer to support CMS modernization, building scalable data pipelines for analytics and reporting. You will ingest, transform, and process data across AWS and Databricks, emphasizing Delta Lake lakehouse architectures.

The role requires strong Python/SQL/PySpark skills, experience with ETL/ELT, data governance, and collaborating with architects, QA, and business analysts.