Java Data Engineer with AI

Surge Technology Solutions Inc; Chicago (IL); On-site; USD 120,000 - 160,000; Full time

Job summary

Surge Technology Solutions Inc. is seeking an experienced Java Data Engineer with AI expertise to design, build, and maintain scalable data pipelines and curated datasets.

The role emphasizes Java, Spring Boot, Python, SQL, and AWS cloud‑native data services, with Spark, Kafka, and DynamoDB to support high‑volume processing. The position involves developing cloud‑native data solutions, implementing data quality and observability, and leveraging AI‑assisted development tools to improve

Qualifications

• Strong Java and Spring Boot hands‑on experience.
• Proficient in Python and SQL.
• Hands‑on with DynamoDB, EMR, EC2; scalable data pipelines and transformations.
• Experience with deployment, troubleshooting, and automated testing.
• Knowledge of data quality, monitoring, and observability; Agile/Scrum environment.
• Experience using Claude or other AI‑assisted development tools.

Responsibilities

• Design, develop, and maintain scalable data pipelines and curated datasets.
• Build data‑intensive applications using Java, Spring Boot, Python, and SQL.
• Develop cloud‑native data solutions using AWS services.
• Work with DynamoDB, EMR, EC2, Spark, and Kafka for high‑volume processing.
• Develop batch and real‑time data pipelines for analytics and operations.
• Implement data integration, transformations, validation, and processing frameworks.
• Deploy, troubleshoot, optimize performance, and provide production support.
• Establish data quality, monitoring, and observability; automate testing.

Skills

Java

Spring Boot

Python

SQL

AWS

Spark

Kafka

DynamoDB

Data Pipelines

Data Quality

Observability

Automated Testing

Claude

Spec-Driven

BMAD

Capital One Experience

Tools

EMR

EC2

Job description

Location: McLean, VA or Chicago, IL /Onsite

Exp : 10 + Yrs

Job Summary

We are seeking an experienced Java Data Engineer with AI expertise to design, build, and maintain scalable data pipelines, data transformations, and curated datasets. The ideal candidate will have strong hands‑on experience with Java, Spring Boot, Python, SQL, AWS cloud-native data services, Spark, Kafka, and DynamoDB.

This role will work across modern cloud and big data platforms to deliver reliable, high-quality, and scalable data solutions supporting both analytical and operational use cases. Experience leveraging AI-assisted development tools and methodologies is highly desirable.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines, transformations, and curated datasets.
• Build data-intensive applications and services using Java, Spring Boot, Python, and SQL.
• Develop cloud-native data solutions using AWS services and infrastructure.
• Work with DynamoDB, EMR, EC2, Spark, and Kafka to build and support high-volume data processing solutions.
• Develop batch and real-time data pipelines supporting analytical and operational workloads.
• Implement data integration, transformation, validation, and processing frameworks.
• Perform application and data pipeline deployment, troubleshooting, performance tuning, and production support.
• Implement automated testing and ensure data pipelines meet reliability and quality standards.
• Establish and maintain data quality, monitoring, observability, and operational controls.
• Collaborate with application developers, data engineers, architects, DevOps, and business stakeholders.
• Apply AI-assisted development practices to improve engineering productivity, code quality, testing, and solution delivery.
• Participate in Agile development activities, including design, development, code reviews, testing, deployment, and production support.

Required Skills

• Strong hands‑on experience with Java and Spring Boot.
• Proficiency in Python and SQL.
• Hands‑on experience with:
• Amazon DynamoDB
• Amazon EMR
• Amazon EC2
• Experience designing and implementing scalable data pipelines and data transformations.
• Experience with deployment, troubleshooting, and automated testing.
• Strong understanding of data quality, monitoring, and observability.
• Experience working in Agile/Scrum development environments.
• Strong analytical and problem‑solving skills.

AI / Modern Development Experience

• Experience using Claude or other AI‑assisted development tools.
• Familiarity with Spec-Driven Development (Spec-Driven) approaches.
• Experience with BMAD (Breakthrough Method for Agile AI-Driven Development) or similar AI‑assisted software development methodologies.
• Ability to leverage AI tools for requirements analysis, code generation, testing, documentation, and developer productivity.

Preferred Qualifications

• Prior experience with Capital one is a plus.
• Experience working with large‑scale distributed data platforms.
• Experience with real‑time/streaming data architectures.
• Experience with cloud‑native application and data platform modernization.
• Financial services or banking domain experience is a plus.
• Strong communication and collaboration skills.

Core Technologies

Java | Spring Boot | Python | SQL | AWS | DynamoDB | EMR | EC2 | Spark | Kafka | Data Pipelines | Data Quality | Observability | Automated Testing | Claude | Spec-Driven Development | BMAD