Senior Data Engineer

Delta Air Lines, Inc.; Atlanta (GA); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Competitive salary

Profit sharing

401(k) with generous company contrib.

Paid time off

Flight privileges

Career development

Job summary

Delta Air Lines, Inc. invites applications for a Senior Data Engineer to advance the data foundation powering IT service assurance and operational intelligence.

You will lead design and delivery of secure, scalable data solutions across ITSM, CMDB, observability, applications, and infrastructure. In this role, you will build reusable data products, drive automation, and guide modernization efforts while collaborating with technology, operations, analytics, security, and architecture teams to

Qualifications

• 7+ years of IT experience, including 4+ years designing, developing, and supporting enterprise data pipelines and data platforms.

Responsibilities

• Lead complex data initiatives from concept to production.
• Design secure, scalable AWS data platforms and lakehouse patterns.
• Provide senior technical leadership for ingestion, transformation, modeling, metadata, quality, lineage, and performance.
• Build and optimize batch, streaming, and near-real-time data integrations across diverse sources.
• Develop Python and SQL pipelines for data orchestration, analytics, and APIs.
• Establish automated data quality controls and monitoring across the data lifecycle.
• Advance AI-ready architectures for SageMaker, Bedrock, LLMs, and vector databases.
• Drive CI/CD, IaC, automated testing, and resilient deployment practices.

Skills

Python

SQL

AWS

ETL/ELT

Data lakehouse

CI/CD

Data modeling

Data governance

Python orchestration

Communication

Tools

Amazon S3

AWS Glue

Amazon Athena

Amazon Redshift

AWS Lambda

Amazon EventBridge

Lake Formation

IAM

SageMaker

Bedrock

Job description

How you'll help us Keep Climbing (overview & key responsibilities)

At Delta, we connect people and cultures, power extraordinary experiences, and shape the future of travel. Here, you'll work on complex challenges on a global scale, using augmented intelligence to build emerging technologies, amplify human potential and continuously grow the skills that keep you at the forefront of what's next. It is an exciting time to be part of Information Technology as Delta continues to modernize how operational data is accessed, trusted, and used to improve reliability, efficiency, and decision-making. Our team is focused on creating the data foundation, access patterns, and intelligence capabilities needed to connect operational signals across ITSM, telemetry, application, and infrastructure platforms

The Senior Data Engineer role supports IT Service Assurance by leading the design and delivery of secure, scalable, and AI-ready data solutions that strengthen operational intelligence, service reliability, automation, and decision‑making. Operating with a high degree of autonomy, this position translates IT operational priorities into trusted data products, reusable integration patterns, and modern engineering capabilities that connect signals across ITSM, CMDB, observability, application, and infrastructure platforms. This role partners across technology, operations, analytics, security, and architecture teams to improve data quality, enable self‑service access, advance automation, and create the foundation for reporting, predictive insights, semantic search, and intelligent enterprise solutions

• Lead independently across complex data initiatives, proactively identifying improvement opportunities, shaping solutions, and driving delivery from concept through production.

• Design and modernize secure, scalable AWS data platforms, pipelines, Lakehouse patterns, and reusable data products aligned to enterprise architecture and governance standards.

• Provide senior technical leadership for ingestion, transformation, modeling, metadata, quality, lineage, performance optimization, and production reliability.

• Build and optimize batch, incremental, event‑driven, and near‑real‑time integrations across APIs, databases, files, ServiceNow, CMDB, observability, telemetry, and enterprise platforms.

• Develop efficient, maintainable Python and SQL for data pipelines, orchestration, transformations, analytics layers, APIs, automation, and reusable engineering frameworks.

• Establish trusted data products through automated controls for completeness, accuracy, freshness, reconciliation, monitoring, alerting, and exception handling.

• Advance AI‑ready architectures that prepare structured and unstructured data for Amazon SageMaker, Amazon Bedrock, LLMs, RAG, vector databases, semantic search, agents, and MCP‑enabled solutions.

• Design retrieval, embedding, indexing, enrichment, and context‑management patterns that improve data usability, discoverability, and responsible AI outcomes.

• Drive operational excellence through CI/CD, Infrastructure as Code, automated testing, source control, code reviews, runbooks, production monitoring, and resilient deployment practices.

• Apply security, privacy, identity, access control, encryption, retention, data classification, and responsible AI requirements across the data lifecycle.

• Influence stakeholders, mentor engineers, promote reusable standards, evaluate emerging technologies, and guide modernization efforts that improve scalability, automation, cost, and business value.

What you need to succeed (minimum qualifications)

• Is at least 18 years of age and has authorization to work in the United States.

• Possesses a high school diploma, GED, or high school equivalency.

• Embraces diverse people, thinking, and styles.

• Consistently prioritizes safety and security of self, others, and personal data.

• Strong analytical and interpersonal skills, with the ability to develop effective relationships and influence without direct authority.

• Superior written and verbal communication skills, as well as executive presence to be an effective representative of the business across the organization.

• 7+ years of IT experience, including 4+ years of hands‑on experience designing, developing, and supporting enterprise data pipelines, data platforms, lakehouses, or analytical data solutions.

• Demonstrated experience designing and implementing cloud‑based data lake, data warehouse, or lakehouse architectures using AWS services.

• Hands‑on experience with Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift, AWS Lambda, Amazon EventBridge, AWS Lake Formation, AWS IAM, and related data services.

• Advanced proficiency in Python and SQL for ingestion, transformation, automation, analysis, data services, and performance optimization.

• Experience designing scalable ETL and ELT pipelines integrating APIs, databases, structured files, semi‑structured content, and cloud services.

• Strong experience with dimensional, relational, normalized, denormalized, and analytics‑oriented data modeling, including layered raw, curated, published, and analytics architectures.

• Strong understanding of incremental ingestion, change‑data capture, watermarking, deduplication, record versioning, historical tracking, late‑arriving data, schema evolution, and reconciliation

• Experience implementing metadata, lineage, cataloging, source‑to‑target mapping, data‑quality testing, freshness monitoring, logging, alerting, and exception handling.

• Working knowledge of Amazon SageMaker, Amazon Bedrock, foundation models, LLM application architectures, RAG, embeddings, vector search, semantic retrieval, and AI‑ready data preparation.

• Experience preparing structured and unstructured data through parsing, chunking, enrichment, embedding generation, indexing, access filtering, and context management.

• Strong understanding of data governance, data classification, privacy, encryption, retention, identity, authorization, and secure handling of personal and sensitive information.

• Experience with Amazon Redshift development concepts, including SQL optimization, workload performance, distribution and sort strategies, materialized views, stored procedures, access controls, and columnar formats such as Parquet.

• Experience with Git, automated testing, CI/CD, Infrastructure as Code such as AWS CDK, or CloudFormation.

• Ability to communicate complex technical concepts, data limitations, architecture decisions, AI data requirements, and implementation plans to technical and business audiences.

• Demonstrated ability to lead technical design, mentor engineers, conduct code and architecture reviews, establish reusable standards, and manage work across concurrent Agile initiatives.

• Consistent commitment to safety, security, responsible AI, operational reliability, and protection of personal and sensitive data.

What will give you a competitive edge (preferred qualifications)

• AWS Certified Data Engineer, AWS Certified Solutions Architect, AWS Certified Machine Learning Engineer, or another relevant AWS certification.

• Hands‑on experience with Amazon SageMaker, Amazon Bedrock, Bedrock Knowledge Bases, Bedrock Agents or AgentCore, Amazon OpenSearch Service, foundation models, embedding models, and production generative AI architectures.

• Experience implementing enterprise RAG solutions, semantic search, hybrid retrieval, reranking, retrieval evaluation, grounding, citations, and AI‑powered knowledge repositories.

• Experience designing or integrating vector databases and vector‑capable platforms such as Amazon OpenSearch Service, PostgreSQL with pgvector, or comparable technologies.

• Experience with prompt engineering, model evaluation, guardrails, hallucination reduction, responsible AI controls, observability, and secure AI deployment patterns.

• Deep knowledge of ServiceNow data, APIs, table structures, reference fields, ITSM processes, configuration management data, and CMDB relationships.

• Experience integrating observability and operational platforms such as Dynatrace, PagerDuty, Amazon CloudWatch, ServiceNow, Sumo Logic, or similar enterprise services

• Experience implementing OAuth 2.0, OpenID Connect, identity federation, role‑based access controls, and document‑or row‑level authorization for data and AI retrieval

• Experience with automated data testing and quality frameworks such as AWS Glue Data Quality, Great Expectations, dbt tests, or comparable technologies

• Demonstrated history of leading data modernization, cloud migration, operational intelligence, platform engineering, generative AI, or AI‑enablement initiatives

Benefits and Perks to Help You Keep Climbing

Ourculture is rooted in a shared dedication to living our values – Care,Integrity, Resilience, Servant Leadership, and Teamwork – every day, ineverything we do. At Delta, our people are our success. At the heart of what weoffer is our focus on Sharing Success with Delta employees. Exploring a careerat Delta gives you a chance to see the world while earning great compensationand benefits to help you keep climbing along the way:

• Competitivesalary, industry‑leading profit sharing program, and performanceincentives

• 401(k) withgenerous company contributions up to 9%

• Paid time offincluding vacation, holidays, paid personal time, maternity and parentalleave

• Comprehensivehealth benefits including medical, dental, vision, short/long termdisability and life benefits

• Family careassistance through fertility support, surrogacy and adoption assistance,lactation support, subsidized back‑up care, and programs that help withloved ones in all stages

• HolisticWellbeing programs to support physical, emotional, social, and financialhealth, including access to an employee assistance program offeringsupport for you and anyone in your household, free financial coaching, andextensive resources supporting mental health

• Domestic andInternational space‑available flight privileges for employees and eligiblefamily members

• Careerdevelopment programs to achieve your long‑term career goals

• World‑widepartnerships to engage in community service and innovative goals createdto focus on sustainability and reducing our carbon footprint

• BusinessResource Groups created to connect employees with common interests topromote inclusion, provide perspective and help implement strategies

• Recognitionrewards and awards through the platform Unstoppable Together

• Access to over500 discounts, specialty savings and voluntary benefits through Deltaperkssuch as car and hotel rentals and auto, home, and pet insurance, legalservices, and childcare

Delta Air Lines, Inc. is an Equal Employment Opportunity / affirmIve Action employer and provides reasonable accommodation in its application process for qualified individuals with disabilities and disabled veterans. If you are a qualified individual, you may request a reasonable accommodation if you are unable or limited in your ability to access job openings through this site, apply for jobs through Delta’s online system, or at any point in the selection process. To request a reasonable accommodation, please click here