Spark or Java Developer

Absolute Business Solutions Corp (ABSC); Aurora (CO); On-site; USD 150,000 - 275,000; Full time

Benefits offered by this job

Generous PTO

401k with match

Tuition Assistance

Health & Wellness allowance

Education and training funds

Volunteer Time Off

Referral Program

Living Our Values awards

Job summary

Absolute Business Solutions Corp (ABSC) is seeking a Secret or TS/SCI-cleared Apache Spark Developer to build and optimize large-scale data pipelines in a secure, air-gapped environment.

You will design and implement Spark (PySpark/Scala) pipelines, transform data with Delta Lake/Parquet, and deploy in containerized Kubernetes environments. Collaboration across analytics, AI/ML, and engineering teams is required.

Qualifications

• TS/SCI eligibility with ability to obtain/maintain counterintelligence polygraph
• Bachelor's degree plus 5 years data engineering or Spark development (or equivalent experience)
• Hands-on experience with Apache Spark, PySpark, and data processing at scale
• Experience with Parquet and Delta Lake and distributed data systems
• Familiarity with Docker/containerization and Kubernetes
• Experience with object storage (S3 or equivalent) and cloud environments (AWS/GCP/Azure)
• Strong troubleshooting and performance tuning skills
• Proficiency in Bash scripting

Responsibilities

• Design, develop, and maintain Apache Spark pipelines (batch and streaming) using PySpark and/or Scala
• Process and transform large-scale datasets using Delta Lake/Parquet in modern data lake architectures
• Optimize Spark jobs for performance including partitioning, shuffle, memory tuning, and storage efficiency
• Implement Structured Streaming pipelines for near real-time data processing
• Develop and deploy Spark applications within Docker containers and Kubernetes clusters
• Execute workloads in distributed Kubernetes environments to enable scalable processing
• Integrate Spark pipelines with analytics platforms, AI/ML workflows, and downstream systems
• Support data ingestion and storage in object-based systems and operate in air-gapped, secure environments

Skills

Apache Spark

PySpark

Scala

Kubernetes

Docker

S3

AWS

GCP

Azure

Bash

Education

Bachelor's degree + 5 years data engineering/Spark

Tools

Delta Lake

Parquet

Git

Jenkins

Job description

Absolute Business Solutions Corp (ABSC) is not just another tech company. We're a community of innovators, engineers, analysts and business professionals working together with our customers to tackle the most complex challenges. For more than 20 years we've supported critical DoD, IC and Federal Civilian missions and global, multi-national corporations. We specialize in supporting our clients in the Intelligence, Technology, Defense, AI/ML, and Data Science fields. As we continue to grow at a rapid pace, we are seeking some amazing new professionals to join our team.

We are actively hiring a Secret or Top Secret/ TS/SCI-cleared Apache Spark Developer to support NGAs Data Modernization Services (DMS) mission by building and optimizing large-scale data processing pipelines. This role focuses on developing high-performance Spark applications within a containerized, Kubernetes-based environment, supporting mission analytics, data exploitation, and AI/ML integration. The ideal candidate thrives in distributed data environments, understands performance tuning deeply, and can operate effectively in secure, air-gapped systems.

This role is on-site/flexible hours in Herndon, VA; Springfield, VA; St. Louis, MO; or Aurora, CO.

Clearance Required for this role: TS/SCI with willingness/ability to obtain/maintain Counterintelligence Polygraph

Core Technology Stack

Data / Processing

• Apache Spark (PySpark, Scala)
• Delta Lake, Parquet
• Structured Streaming

Infrastructure

• Kubernetes (execution environment)
• Docker

Storage / Cloud (Abstracted)

• S3 / object storage
• AWS / GCP / Azure (environment-dependent)

DevOps (Exposure Level)

• Git, Jenkins (CI/CD)

Languages

• Python (PySpark)
• Scala (preferred)
• Bash / scripting

Key Responsibilities

• Design, develop, and maintain Apache Spark pipelines (batch and streaming) using PySpark and/or Scala
• Process and transform large-scale datasets using modern data lake architectures (Delta Lake, Parquet)
• Optimize Spark jobs for performance, including:
  • Partitioning strategies
  • Shuffle optimization
  • Memory tuning
  • File sizing and storage efficiency
• Implement Structured Streaming pipelines for near real-time data processing
• Develop and deploy Spark applications within containerized environments (Docker)
• Execute workloads in Kubernetes clusters, supporting scalable and distributed processing
• Integrate Spark pipelines with downstream systems, including:
  • Analytics platforms (SQL, notebooks)
  • AI/ML workflows and feature engineering pipelines
• Support data ingestion and storage in object-based systems (e.g., S3-compatible storage)
• Troubleshoot data pipeline failures and ensure reliability in mission-critical environments
• Operate within secure, air-gapped environments, including:
  • Managing dependencies without internet access
  • Working within controlled network and security constraints

Required Qualifications

• TS/SCI (eligibility) with ability/willingness to obtain/maintain counterintelligence polygraph
• Bachelors degree plus 5 years experience in data engineering or Spark development (will entertain additional years experience in lieu of degree)
• Strong hands-on experience with:
  • Apache Spark or Enterprise Java Developer
  • Python (PySpark)
  • Data processing at scale
• Experience working with:
  • Parquet and/or Delta Lake
  • Distributed data systems
• Familiarity with:
  • Docker / containerization
  • Kubernetes (basic to intermediate experience)
• Experience with object storage systems (e.g., S3 or equivalent)
• Strong troubleshooting and performance tuning skills
• Proficiency in Bash or scripting

Preferred Qualifications

• Experience with Scala for Spark development
• Experience with Structured Streaming in production environments
• Familiarity with Iceberg or lakehouse architectures
• Experience with CI/CD pipelines (Jenkins, Git)
• Exposure to Terraform or Infrastructure as Code
• Experience supporting AI/ML data pipelines
• Prior experience supporting NGA, IC, or DoD programs

Pay range dependent on geolocation for candidate.

An approximate salary range for this position is $150,000-$275000. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.

Who we are:

ABSC is a technology and services company that combines the agility of a small business with proven processes refined over more than two decades in business. We specialize in supporting public sector clients in the Intelligence, Defense, Health, and Safety areas. Our team stands ready to deliver the next generation of programs, personnel, and solutions to help advance our federal government customers driving innovation, agility, and security across all mission areas.

Some of our benefits include:

• Generous PTO plus 11 Federal Holidays
• Retirement Planning 401k Fully Vested with Match
• Tuition Assistance Program Annual contributions to help you pay down your loans
• Annual Health and Wellness Allowance buy an Apple Watch, a treadmill, or hit the gym on us
• Career Development Annual Funds to spend on Education and Training
• Volunteer Time Off Annually, all employees can spend 8 hours directly supporting a charity of choice
• Charitable Match ABSC matches an employees donation to a qualifying charity"Li>7 /＼>Assume normal text
• Referral Program We pay for internal and external referrals!
• LOV Awards Earn bonus awards throughout the year from our Living Our Values awards program

*ABSC is a proud V3, Virginia Values Vets, member which recognizes our commitment to hiring Veterans. If you are a veteran, please be sure to include that in your application. Thank you! *

Equal Opportunity Employer, including veterans and individuals with disabilities.