Manager – Data Engineering

Simplify Recruiting; United States; On-site; USD 150,000 - 230,000; Full time

Job summary

Simplify Recruiting in the United States seeks a Lead Data Engineer to own the data platform direction, guiding AWS-native pipelines and legacy SQL Server/SSIS ETL modernization. You will mentor a team and collaborate with analytics, DevOps, and BI stakeholders.

You will define standards, drive performance improvements, and ensure data quality, lineage, and reliability across deployments. This role blends hands-on engineering with leadership to deliver scalable data solutions.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.
• 10-12 years of data engineering/ETL with progression toward technical leadership.
• Hands-on experience with AWS data services: S3, Redshift, Glue, Athena, Lambda, and Step Functions.
• Strong SQL Server experience including T-SQL development, query optimization, and performance tuning.
• Proven ability to build and maintain SSIS ETL packages (control flow, data flow, error handling, deployment).
• Strong Python skills for scripting and Glue/PySpark-based ETL development.
• Deep expertise in dimensional data modeling (star schema) for analytics.
• Experience leading data migration programs using AWS DMS or similar tooling.
• Strong governance, data quality, lineage practices and standards enforcement.
• Excellent debugging, performance tuning, and production-support skills.
• Experience leading technical projects or data-engineering teams.
• Excellent written and verbal communication for cross-functional collaboration.

Responsibilities

• Lead ETL/ELT pipelines using AWS Glue, Lambda, and Step Functions across AWS data platforms (S3, Redshift, Athena).
• Own the migration and modernization of legacy SQL Server/SSIS workloads to AWS-native pipelines.
• Maintain and optimize existing SQL Server/SSIS ETL processes and error handling.
• Define coding standards, design patterns, and best practices for the data engineering team.
• Write and optimize complex T-SQL queries, stored procedures, and views for reporting.
• Architect and govern dimensional data models to support analytics and reporting.
• Lead data migration efforts using AWS DMS from on-premises to AWS.
• Enforce data quality, validation, monitoring, and lineage standards across pipelines.
• Drive performance improvements, reliability, and cost efficiency across pipelines.
• Mentor data engineers and conduct meaningful code reviews.
• Maintain thorough documentation of data flows and pipeline architectures.

Skills

AWS data services

SQL Server / T-SQL

Python scripting

ETL/ELT development

Dimensional modeling

AI-assisted development

Team leadership

Pipeline design

Education

Bachelor's degree in Computer Science/Engineering/IS

Tools

SSIS

AWS Glue

AWS Lambda

Step Functions

AWS DMS

Redshift

Athena

S3

Power BI

Job description

Tech Stack

AWS (S3, Redshift, Glue, Athena, Lambda, Step Functions, DMS), SQL Server (T-SQL), SSIS, Python, Data Warehousing/ETL, AI-Assisted Development

Role Overview

We are looking for a Lead Data Engineer to own the technical direction and delivery of data engineering initiatives across our health plan technology platform, spanning AWS-native data services and existing SQL Server/SSIS ETL processes. Working alongside software engineering, Data/BI & Integration, DevOps, and Edifecs EDI teams, you will set engineering standards, lead the migration and modernization of legacy workloads onto AWS-native pipelines, and mentor a team of data engineers. This role combines deep hands-on technical ownership with emerging team leadership, acting as the primary technical escalation point for data pipeline design, performance, and reliability.

Key Responsibilities

• Lead the design, build, and delivery of ETL/ELT pipelines using AWS Glue, Lambda, and Step Functions to ingest, transform, and load data across AWS-based data platforms (S3, Redshift, Athena)
• Own the technical roadmap for migrating and modernizing legacy SQL Server/SSIS ETL workloads to AWS-native data pipelines
• Maintain, enhance, and troubleshoot existing SQL Server-based ETL processes built in SSIS, including packages, control flows, data flows, and error handling
• Define coding standards, design patterns, and best practices for data engineering across the team
• Write and optimize complex T-SQL queries, stored procedures, and views supporting reporting and downstream applications
• Architect and govern data models (dimensional/star schema) supporting analytics and reporting use cases
• Lead use of AWS Database Migration Service (DMS) and related tooling to drive data migration from on-premises SQL Server to AWS
• Define and enforce data quality, validation, monitoring, and lineage standards across ETL pipelines
• Drive pipeline performance optimization, reliability improvements, and cost efficiency across both AWS-native and SQL Server/SSIS workloads
• Serve as the primary technical partner for BI, integration, and application teams (including Power BI and Edifecs/EDI teams) to ensure data availability and consistency
• Lead technical design reviews and contribute to architecture decisions at the program level
• Own production incident response and root-cause analysis for data pipeline issues; implement long-term reliability improvements
• Mentor and develop data engineers; guide technical growth and conduct meaningful code reviews
• Maintain thorough documentation of data flows, pipeline architecture, and data models to support team knowledge sharing

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field
• 10 - 12 years of experience in data engineering or ETL development, with demonstrated progression toward technical leadership
• Expert, hands-on experience with AWS data services: S3, Redshift, Glue, Athena, Lambda, and Step Functions
• Expert, hands-on experience with SQL Server, including T-SQL development, query optimization, and performance tuning
• Expert, hands-on experience building and maintaining ETL packages in SSIS (control flow, data flow, error handling, and deployment)
• Strong proficiency in Python for scripting, automation, and Glue/PySpark-based ETL development
• Deep expertise in dimensional data modeling (star schema) for analytics and reporting use cases
• Proven experience leading data migration programs using AWS DMS or equivalent tooling for on-premises to AWS transitions
• Strong command of data quality, governance, and lineage practices -- able to define and enforce standards across a team
• Strong debugging, performance-tuning, and production-support skills across complex ETL pipelines
• Experience leading technical projects or guiding a team of data engineers
• Excellent written and verbal communication skills for cross-functional and stakeholder engagement

AI Knowledge & AI Assisted Development (Required)

• Daily, practical use of AI coding/assistant tools (e.g., "Claude Code, GitHub Copilot, Cursor, or similar") to accelerate development of Glue/PySpark scripts, SSIS package logic, and T-SQL queries
• Able to critically review and validate AI-generated code, queries, and transformations for correctness, performance, and data integrity before deployment
• Defines and enforces team-level standards for responsible AI-assisted development, including validation rigor, security review of AI-generated code, and prompt hygiene
• Practical use of AI tools to assist with data profiling, anomaly detection, and technical documentation
• Deep understanding of secure and compliant AI tool usage, including never entering PHI, member data, or other sensitive information into prompts or external AI tools
• Identifies and leads adoption of AI-driven automation to improve pipeline development, testing, and migration efficiency across the team

Preferred Qualifications

• Experience in the US health insurance or payer domain: claims, eligibility, enrollment, provider, or member data, with HIPAA-aware data handling practices
• Experience with Power BI or other BI/reporting tools consuming the data pipelines you build
• Experience with additional AWS data services: EMR, Kinesis, or Redshift Spectrum
• Experience with modern orchestration tools (e.g., Apache Airflow) as an alternative or complement to SSIS
• Relevant certifications: AWS Certified Data Engineer/Analytics Specialty, Microsoft SQL Server certifications