Senior Data Engineer: Python, Airflow & Cloud Pipelines

Adelphi Group; Chicago (IL); Hybrid; USD 110,000 - 120,000; Full time

Benefits offered by this job

Health insurance

401(k)

Stock options

Healthcare & Dependent FSA

Vacation days

Paid parental leave

Disability benefits

Job summary

RAPP is seeking a Senior Data Engineer in Chicago to design scalable, cloud-native data pipelines and platforms. You will work with Python, Airflow, AWS Lambda, DynamoDB, and dbt to enable advanced analytics and ML use cases.

Collaboration with creative and engineering teams is essential for scalable asset workflows. The role emphasizes data governance, high availability, and CI/CD automation, with a hybrid work model allowing up to two days remote weekly.

Qualifications

• 5–8+ years of experience in data engineering, software engineering, or a related role.
• Strong expertise in Python for data engineering and automation.
• Hands-on experience with Apache Airflow for orchestration.
• Proficiency with AWS Lambda and serverless design patterns.
• Solid experience with DynamoDB (schema design, performance tuning, scaling).
• Strong knowledge of dbt for transformation and analytics modeling.
• Experience with cloud environments (AWS preferred).
• Familiarity with CI/CD workflows, Git, and DevOps practices.
• Strong problem solving and communication skills.

Responsibilities

• Data Pipeline Development: Design, build, and maintain ETL/ELT pipelines using Python and Airflow.
• Develop serverless workflows leveraging AWS Lambda for scalable data processing.
• Implement and optimize dbt models for analytics and transformations.
• Data Architecture & Storage: Design schemas and manage data in DynamoDB and other cloud-native storage solutions.
• Ensure high availability, scalability, and performance of data systems.
• Integrate structured, semi-structured, and unstructured data sources.
• Automation & Orchestration: Build workflow orchestration strategies using Airflow for scheduling and monitoring pipelines.
• Automate infrastructure deployment and CI/CD pipelines for data services.
• Quality & Governance: Implement data validation, testing, and monitoring frameworks.
• Ensure compliance with security, privacy, and governance standards.
• Collaboration & Leadershi: Partner with analytics, product, and engineering teams to deliver reliable datasets.
• Mentor junior engineers and enforce best practices in data engineering.
• Actively contribute to improving team efficiency, scalability, and standards.

Skills

Python

Apache Airflow

AWS Lambda

DynamoDB

dbt

CI/CD

Git

DevOps

Tools

Airflow

AWS Lambda

DynamoDB

dbt

AWS

Job description

RAPP is seeking a Senior Data Engineer in Chicago to design scalable, cloud-native data pipelines and platforms. You will work with Python, Airflow, AWS Lambda, DynamoDB, and dbt to enable advanced analytics and ML use cases.

Collaboration with creative and engineering teams is essential for scalable asset workflows. The role emphasizes data governance, high availability, and CI/CD automation, with a hybrid work model allowing up to two days remote weekly.