Lead Data Engineer

Lincoln Financial Group; Radnor (Delaware County); Hybrid; USD 120,000 - 193,000; Full time

Job summary

Lincoln Financial is seeking an experienced AWS data engineer to design, build, and optimize large-scale data pipelines for BI and analytics in a hybrid work model.

You will lead end-to-end data flows, mentor engineers, and partner with data scientists and business stakeholders to deliver high-quality datasets on AWS.

Qualifications

• Bachelor’s degree (or equivalent) in Computer Science, Information Systems, Engineering, or related field.
• 7+ years in AWS data engineering or a similar role.
• Strong expertise in Aurora PostgreSQL and Amazon Redshift, including query optimization and workload tuning.
• End-to-end data processing, from source to consumption; performance and cost optimization.
• Hands-on AWS (Glue, Lambda, S3, Athena, EC2, IAM).
• Python / PySpark and SQL development for ETL in Agile environments.
• Data governance, data quality, and data security practices.
• DevOps: Git, CI/CD, CloudFormation, Stonebranch or similar scheduler.
• Mentor engineers and lead delivery of complex projects.

Responsibilities

• Design, build, and maintain robust batch and streaming data pipelines (e.g., AWS DMS) to support BI and analytics.
• Integrate structured and unstructured data from internal and external sources.
• Develop ETL workflows using AWS Glue, Lambda, PySpark, and SQL-based scripts.
• Analyze database workloads; optimize queries, indexing, and storage on Aurora PostgreSQL and Redshift.
• Build S3-based data lake and warehouse layers with fine-grained access controls.
• Own end-to-end data flows from source to consumption; trace and resolve issues.
• Apply data modeling standards and create technical designs from business requirements.
• Orchestrate workflows with Stonebranch, Step Functions, Airflow; monitor with CloudWatch and data quality checks.
• Ensure data security, governance, and compliance; optimize data workflows.
• Collaborate with data scientists, analysts, and stakeholders to deliver datasets; shape AWS data infra strategy.
• Set team standards, review code, mentor engineers, and provide production support.

Skills

AWS data engineering

ETL development

Python / PySpark

SQL optimization

Data modeling

Data governance & security

DevOps / CI/CD

Leadership / mentoring

Education

Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field

Tools

AWS Glue

Lambda

S3

Athena

EC2

IAM

Glue Data Catalog

Job description

Be Aware of Fraudulent Recruiting Activities: Lincoln values the privacy and security of every applicant and urges all applicants to diligently protect their sensitive personal information from scams targeting job seekers. These scams can take many forms including fake employment applications, bogus interviews and falsified offer letters.

Lincoln will not ask applicants to provide their social security numbers, date of birth, bank account information or other sensitive information in job applications. Additionally, our recruiters do not communicate with applicants through free e‑mail accounts (Gmail, Yahoo, Hotmail) or conduct interviews utilizing video chat rooms. We will never ask applicants to provide payment during the hiring process or extend an offer without conducting a phone, live video or in‑person interview. Please contact Lincoln's fraud team at [e-mail] if you encounter a recruiter or see a job opportunity that seems suspicious.

Select how often (in days) to receive an alert:

Alternate Locations: Radnor, PA (Pennsylvania); Charlotte, NC (North Carolina); Fort Wayne, IN (Indiana); Greensboro, NC (North Carolina)

Work Arrangement:

Hybrid : Employee will work 3 days a week in a Lincoln office

Relocation assistance:is not available for this opportunity.

Requisition #:76901

The Role at a Glance

What you'll be doing

• Design, build, and maintain robust batch and streaming data pipelines (including change data capture, for example AWS DMS) to support business intelligence and analytics.
• Integrate structured and unstructured data from internal and external sources.
• Develop ETL workflows using AWS Glue, Lambda, custom Python, PySpark, and SQL-based scripts.
• Analyze database workloads and optimize queries, indexing, partitioning, and storage in Aurora PostgreSQL and Amazon Redshift for integrity, performance, and cost.
• Build S3-based data lake and warehouse layers (Glue Data Catalog, Athena, Parquet) with fine-grained access controls.
• Own end-to-end data flows from source to consumption; traces and resolve issues across the full chain.
• Apply data modeling standards (normal form, star, snowflake) and create technical designs from business requirements.
• Orchestrate and schedule workflows (Stonebranch, Step Functions, Airflow) and monitor them with CloudWatch and automated data quality checks.
• Ensure data security, governance, and compliance; monitor, troubleshoot, and optimize data workflows.
• Integrate with upstream and downstream applications and collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets; help shape data infrastructure strategy on AWS.
• Set team standards, review code, mentor engineers, and own production support and process improvement.

What we’re looking for

Must-haves:

• Bachelor’s degree (or equivalent) in Computer Science, Information Systems, Engineering, or a related field.
• 7+ years in AWS data engineering or a similar role.
• Strong expertise in Aurora PostgreSQL and Amazon Redshift, including database analysis, query optimization, advanced SQL, execution plans, indexing, distribution and sort keys, and workload tuning.
• Holistic view of end-to-end data processing, from source to consumption, including dependencies, bottlenecks, and downstream impact.
• Data and application integration experience with upstream source systems and downstream consumers (APIs, files, messaging, and reporting layers).
• Hands‑on AWS experience: Glue, Lambda, S3, Athena, EC2, and IAM.
• Python / PySpark and SQL development for ETL and data integration in Agile, SDLC‑based environments.
• Data modeling, data governance, data quality, and data security practices.
• DevOps and scheduling: Git, CI/CD pipelines, automated testing, CloudFormation or similar, and Stonebranch scheduling knowledge (or a similar scheduler such as Autosys).
• Hands‑on use of approved AI‑assisted development tools (GitHub Copilot or Claude Code), with validation of AI output and adherence to Lincoln data privacy and responsible AI requirements.
• Proven ability to mentor engineers and lead delivery of complex projects.

Nice-to-haves:

• AWS Certified Data Engineer – Associate, Solutions Architect, or an AWS specialty certification.
• EMR, Kinesis, Lake Formation, DynamoDB, and Apache Iceberg experience.
• Microsoft 365 Copilot and Claude Cowork experience; coaching teams on safe AI use.
• RAG data preparation, embeddings, vector stores, and AWS AI services (Bedrock, SageMaker, Amazon Q).
• Salesforce CRM and MuleSoft (API‑led integration) as upstream or downstream systems; MicroStrategy and Tableau reporting and BI background.
• Financial data experience is a plus, especially Investment products like Mutual Funds, Alternative Funds and Fund Data management.

Core Competencies:

• Technical leadership and AWS data engineering expertise.
• Collaboration and Agile delivery with business, IT, and analytics teams.
• Analytical depth, operational ownership, and a data‑centric mindset (quality, governance, security).
• AI fluency: responsible, validated use of AI tools.

Application Deadline

Applications for this position will be accepted through January 31, 2027 subject to earlier closure due to applicant volume.

What’s it like to work here?

At Lincoln Financial, we love what we do. We make meaningful contributions each and every day to empower our customers to take charge of their lives. Working alongside dedicated and talented colleagues, we build fulfilling careers and stronger communities through a company that values our unique perspectives, insights and contributions and invests in programs that empower each of us to take charge of our own future.

What’s in it for you:

Clearly defined career tracks and job levels, along with associated behaviors for each of Lincoln's core values andleadership attributes

Leadership development and virtual training opportunities

Free financial counseling, health coaching and employee assistance program

Tuition assistance program

Work arrangements that work for you

Effective productivity/technology tools and training

The pay range for this position is$120,375 - $192,600with anticipated pay for new hires between the minimum and midpoint of the range and could vary above and below the listed range as permitted by applicable law. Pay is based on non‑discriminatory factors including but not limited to work experience, education, location, licensure requirements, proficiency and qualifications required for the role. The base pay is just one component of Lincoln’s total rewards package for employees. In addition, the role may be eligible for the Annual Incentive Program, which is discretionary and based on the performance of the company, business unit and individual. Other rewards may include long‑term incentives, sales incentives and Lincoln’s standard benefits package.

About The Company

Lincoln Financial (NYSE: LNC) helps people to confidently plan for their version of a successful future. We focus on identifying a clear path to financial security, with products including annuities, life insurance, group protection, and retirement plan services.

With our 120‑year track record of expertise and integrity, millions of customers trust our solutions and service to help put their goals in reach.

Lincoln Financial Distributors, a broker‑dealer, is the wholesale distribution organization of Lincoln Financial. Lincoln Financial is the marketing name for Lincoln Financial Corporation and its affiliates including The Lincoln National Life Insurance Company, Fort Wayne, IN, and Lincoln Life & Annuity Company of New York, Syracuse, NY. Lincoln Financial affiliates, their distributors, and their respective employees, representatives and/or insurance agents do not provide tax, accounting or legal advice.

Lincoln is committed to creating an inclusive environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

Additional Information

This position may be subject to Lincoln’s Political Contribution Policy.

Any unsolicited resumes or candidate profiles submitted through our web site or to personal e‑mail accounts of employees of Lincoln Financial are considered property of Lincoln Financial and are not subject to payment of agency fees.

Lincoln Financial ("Lincoln" or "