Lead Data Engineer - AWS Pipelines & Leadership (Hybrid)

Lincoln Financial; Radnor (Delaware County); Hybrid; USD 150,000 - 200,000; Full time

Job summary

Lincoln Financial is seeking a Lead Software Engineer to provide technical leadership for the Data Strategy and Delivery team. You will design, build, and maintain scalable data pipelines and data platforms, partnering with business, IT, and analytics stakeholders to deliver secure, governed datasets for analytics.

You will lead a team of data engineers, own delivery, set the technical vision, and ensure high-quality data across the data lake and warehouse, with AWS-based tooling including

Qualifications

• Bachelor’s degree in CS or related field.
• 7+ years in AWS data engineering or similar; lead level experience.
• Proven leadership with delivery of complex data projects.
• Deep expertise in Aurora PostgreSQL and Redshift; performance tuning.
• Experience with batch and streaming pipelines and AWS services.
• Hands-on Python / PySpark and SQL for ETL.
• Familiar with data governance, security, and IAM.
• Experience with orchestration tools (Step Functions, Airflow, MWAA).

Responsibilities

• Lead and prioritize work, own delivery, set standards and a clear technical vision, review code, mentor engineers, and grow future leaders.
• Design, build, and maintain robust batch and streaming data pipelines to support business intelligence and analytics.
• Integrate structured and unstructured data from internal and external sources.
• Develop ETL workflows using AWS Glue, Lambda, Python, PySpark, and SQL-based scripts.
• Analyze database workloads and optimize queries, indexing, partitioning for Aurora PostgreSQL and Redshift.
• Build S3-based data lake and warehouse layers with Parquet and fine-grained access controls.
• Own end-to-end data flows from source to consumption; trace and resolve issues.
• Apply data modeling standards and data quality practices; create technical designs from business requirements.
• Orchestrate and schedule workflows and monitor with CloudWatch and data quality checks.
• Ensure data security, governance, and compliance across pipelines.
• Collaborate with data scientists and analysts to deliver high-quality datasets and influence AWS data infrastructure strategy.

Skills

Data engineering

AWS data services

Leadership

Python / PySpark

SQL

ETL design

DevOps

CI/CD

Automation & governance

AI tooling familiarity

Education

Bachelor's degree

Tools

Aurora PostgreSQL

Amazon Redshift

AWS Glue

AWS Lambda

S3

Athena

EC2

IAM

Job description

Lincoln Financial is seeking a Lead Software Engineer to provide technical leadership for the Data Strategy and Delivery team. You will design, build, and maintain scalable data pipelines and data platforms, partnering with business, IT, and analytics stakeholders to deliver secure, governed datasets for analytics.

You will lead a team of data engineers, own delivery, set the technical vision, and ensure high-quality data across the data lake and warehouse, with AWS-based tooling including