AI-Driven Data Engineering Lead

Navy Federal Credit Union; Vienna (VA); On-site; USD 130,000 - 170,000; Full time

Job summary

Navy Federal Credit Union seeks a Senior Data Engineer to lead AI-enabled data engineering initiatives across the enterprise. You will design scalable data pipelines, autonomous or human-guided AI agents, and reusable automation frameworks to accelerate NFCU’s move to AI-powered data platforms.

The role requires hands-on expertise with Spark, Python, SQL, Databricks, Microsoft Fabric, and Azure AI services, plus leadership to mentor engineers and ensure secure, governed, and observable delivery

Qualifications

• Bachelor’s degree in Information Systems, Computer Science, Engineering, Data Engineering, or a related field.
• Advanced hands‑on experience in data engineering using Apache Spark, Python, SQL, Databricks, Microsoft Fabric, and modern cloud-native data and analytics technologies.
• Experience designing scalable batch, near real‑time, and real‑time data pipelines within Lakehouse/Medallion patterns.

Responsibilities

• Lead design, development, and deployment of AI-enabled data engineering accelerators and frameworks.
• Apply Generative AI, Agentic AI, RAG, vector search, and orchestration to automate data pipelines.
• Design Lakehouse and Medallion architectures using Databricks, Fabric, Spark, Python, SQL.
• Establish AI-first engineering standards, governance, and reusable patterns.
• Develop production-ready pipelines with reliability, observability, and data quality focus.
• Collaborate with architects, DevSecOps, product owners, governance, and stakeholders.
• Evaluate new AI/data tech and recommend adoption aligned with NFCU standards.
• Lead Agile delivery, CI/CD, testing, deployment, and production readiness improvements.
• Mentor engineers and promote secure, governed, measurable AI-enabled delivery.
• Ensure data system security, integrity, traceability, and regulatory alignment.

Skills

Apache Spark

Python

SQL

Databricks

Microsoft Fabric

Azure AI Services

LangChain

LangGraph

Vector search

Agent orchestration

Education

Bachelor’s degree in Information Systems, Computer Science, Engineering, Data Engineering, or a related field

Tools

Databricks Mosaic AI

Databricks

Microsoft Foundry

Job description

Navy Federal Credit Union seeks a Senior Data Engineer to lead AI-enabled data engineering initiatives across the enterprise. You will design scalable data pipelines, autonomous or human-guided AI agents, and reusable automation frameworks to accelerate NFCU’s move to AI-powered data platforms.

The role requires hands-on expertise with Spark, Python, SQL, Databricks, Microsoft Fabric, and Azure AI services, plus leadership to mentor engineers and ensure secure, governed, and observable delivery