Spark or Java Developer

Absolute Business Solutions Corp.; St. Louis (MO); On-site; USD 100,000 - 130,000; Full time

Benefits offered by this job

401k Fully Vested with Match

Tuition Assistance Program

Annual Health and Wellness Allowance

Annual Funds for Education and Training

Volunteer Time Off

Charitable Match

Referral Program

Living Our Values awards

Job summary

Absolute Business Solutions Corp. is looking for a talented Apache Spark Developer to enhance NGA’s Data Modernization Services. The position focuses on building efficient data processing pipelines within a secure environment while ensuring optimal performance and reliability.

The ideal candidate will have expertise in Apache Spark and Python, alongside a minimum of 5 years in a relevant role, qualifying for TS/SCI clearance. A competitive benefits package is offered.

Qualifications

• Bachelor's degree plus 5 years' experience in data engineering or Spark development.
• Strong hands-on experience with Apache Spark and Python.
• Experience with Parquet and/or Delta Lake, and distributed data systems.

Responsibilities

• Design, develop, and maintain Apache Spark pipelines using PySpark and/or Scala.
• Optimize Spark jobs, including memory tuning and storage efficiency.
• Integrate Spark pipelines with downstream systems for analytics and AI/ML.

Skills

Apache Spark

Python (PySpark)

Data processing at scale

Performance tuning

Bash / scripting

Education

Bachelor’s degree in a relevant field

Tools

Docker

Kubernetes

CI/CD (Jenkins, Git)

Job description

Apache Spark Developer

We are actively hiring a Secret or Top Secret/TS/SCI-cleared Apache Spark Developer to support NGA’s Data Modernization Services (DMS) mission by building and optimizing large-scale data processing pipelines. This role focuses on developing high-performance Spark applications within a containerized, Kubernetes-based environment, supporting mission analytics, data exploitation, and AI/ML integration. The ideal candidate thrives in distributed data environments, understands performance tuning deeply, and can operate effectively in secure, air-gapped systems. This role is on-site/flexible hours in Herndon, VA; Springfield, VA; St. Louis, MO; or Aurora, CO.

Clearance required: minimum Secret, TS/SCI eligibility, preference for CI polygraph.

Core Technology Stack

• Data / Processing
  • Apache Spark (PySpark, Scala)
  • Structured Streaming
• Infrastructure
  • Docker
  • S3 / object storage
  • Git, Jenkins (CI/CD)
• Languages
  • Python (PySpark)
  • Scala (preferred)
  • Bash / scripting

Key Responsibilities

• Design, develop, and maintain Apache Spark pipelines (batch and streaming) using PySpark and/or Scala
• Process and transform large-scale datasets using modern data lake architectures (Delta Lake, Parquet)
• Optimize Spark jobs for performance, including:
  • Memory tuning
  • File sizing and storage efficiency
• Implement Structured Streaming pipelines for near real-time data processing
• Develop and deploy Spark applications within containerized environments (Docker)
• Execute workloads in Kubernetes clusters, supporting scalable and distributed processing
• Integrate Spark pipelines with downstream systems, including:
  • Analytics platforms (SQL, notebooks)
  • AI/ML workflows and feature engineering pipelines
• Support data ingestion and storage in object-based systems (e.g., S3-compatible storage)
• Troubleshoot data pipeline failures and ensure reliability in mission-critical environments
• Operate within secure, air-gapped environments, including:
  • Managing dependencies without internet access
  • Working within controlled network and security constraints

Required Qualifications

• TS/SCI (eligibility) with ability/willingness to obtain/maintain counterintelligence polygraph
• Bachelor’s degree plus 5 years’ experience in data engineering or Spark development (will entertain additional years’ experience in lieu of degree)
• Strong hands-on experience with:
  • Apache Spark (mandatory)
  • Python (PySpark)
  • Data processing at scale
• Experience working with:
  • Parquet and/or Delta Lake
  • Distributed data systems
• Familiarity with:
  • Docker / containerization
  • Kubernetes (basic to intermediate experience)
  • Object storage systems (e.g., S3 or equivalent)
  • Strong troubleshooting and performance tuning skills
  • Proficiency in Bash or scripting

Preferred Qualifications

• Experience with Scala for Spark development
• Experience with Structured Streaming in production environments
• Familiarity with Iceberg or lakehouse architectures
• Experience with CI/CD pipelines (Jenkins, Git)
• Exposure to Terraform or Infrastructure as Code
• Experience supporting AI/ML data pipelines
• Prior experience supporting NGA, IC, or DoD programs

Benefits

• Retirement Planning – 401k Fully Vested with Match
• Tuition Assistance Program – Annual contributions to help you pay down your loans
• Annual Health and Wellness Allowance – buy an Apple Watch, a treadmill, or hit the gym on us
• Career Development – Annual Funds to spend on Education and Training
• Volunteer Time Off – Annually, all employees can spend 8 hours directly supporting a charity of choice
• Charitable Match – ABSC matches an employee’s donation to a qualifying charity
• Referral Program – We pay for internal and external referrals!
• LOV Awards – Earn bonus awards throughout the year from our Living Our Values awards program

Equal Opportunity Employer, including veterans and individuals with disabilities.