Lead Backend Engineer - Spark, AI-Driven Data Pipelines

Capital One; Richmond (VA); On-site; USD 179,000 - 225,000; Full time

Benefits offered by this job

Health benefits

Job summary

Capital One is seeking a Lead Software Engineer for Back End in Enterprise Platforms Technology to architect and scale core data infrastructure with Java, Spark, and AI-native workflows. You will guide a team of back-end and data engineers, optimize cloud architectures on AWS, and design APIs that interface with AI/ML models.

Requirements include a Bachelor's degree, 4+ years in software engineering, 1+ year cloud experience, and 7+ years preferred with SQL/NoSQL expertise.

Qualifications

• Bachelor's degree required.
• 4+ years of professional software engineering experience.
• 1+ year of cloud computing experience (AWS/Azure/Google Cloud).
• 7+ years of engineering experience preferred.
• 4+ years of SQL and NoSQL experience.
• 2+ years of Spark performance tuning.

Responsibilities

• Architect and build scalable data pipelines and microservices using Java, Spark, Scala, Spring Boot.
• Scale systems to ingest and serve billions of data points with low latency and high availability.
• Integrate AI-native development tools and autonomous agents into development workflows.
• Design backend APIs interfacing with AI/ML models and LLM endpoints for data enrichment.
• Provide technical leadership, code reviews, and reliability practices.
• Leverage AWS services to build cost-effective cloud architectures and tune Spark jobs.
• Ensure data quality, pipeline integrity, and zero-downtime deployments.

Education

Bachelor's Degree

Master's Degree

Tools

Java

Apache Spark

Scala

Spring Boot

AWS

EMR

Glue

Lambda

Step Functions

S3

RDS

DynamoDB

Kinesis

SQL

NoSQL

Open Source frameworks

Claude Code

GitHub Copilot

Cursor

OpenAI Canvas

Job description

Capital One is seeking a Lead Software Engineer for Back End in Enterprise Platforms Technology to architect and scale core data infrastructure with Java, Spark, and AI-native workflows. You will guide a team of back-end and data engineers, optimize cloud architectures on AWS, and design APIs that interface with AI/ML models.

Requirements include a Bachelor's degree, 4+ years in software engineering, 1+ year cloud experience, and 7+ years preferred with SQL/NoSQL expertise.