Senior Data Engineer - TS/SCI Clearance, Spark & Airflow

Shield Consulting Solutions, Inc.; Maryland; On-site; USD 230,000 - 240,000; Full time

Benefits offered by this job

25 days PTO

11 paid holidays

100% employer-paid healthcare for you+

8% 401(k) employer match

Job summary

Shield Consulting Solutions, Inc. is seeking a Software Engineer with extensive expertise in dataflow design and Spark-based distributed processing. The role focuses on designing, implementing, and optimizing data ingress/egress pathways to ensure efficient, scalable analytics workloads.

On-site in Maryland, active TS/SCI w/ Polygraph clearance is required. The candidate should have strong Linux CLI, Python, Java, and Apache Airflow experience, plus familiarity with SQL databases and Git, aiming

Qualifications

• 7 years of experience as a software engineer.
• Bachelor’s degree in a technical discipline; 2 extra years of software engineering experience may substitute for a degree.
• Active TS/SCI w/ Polygraph clearance required.

Responsibilities

• Design, implement, and optimize data ingress/egress pathways for analytics workloads.
• Develop and maintain Apache Spark-based distributed processing pipelines to support scalable data processing.

Skills

Linux CLI

Bash scripting

Python

Java

Apache Airflow

Apache Spark

SQL (MySQL/MariaDB/PostgreSQL)

Jupyter Notebook

pandas/NumPy

Parquet/JSON/CSV/XML

Data quality & validation

Git

Slurm

JIRA/Confluence

Education

Bachelor's degree in a technical discipline

Tools

Slurm

JIRA

Confluence

Job description

Shield Consulting Solutions, Inc. is seeking a Software Engineer with extensive expertise in dataflow design and Spark-based distributed processing. The role focuses on designing, implementing, and optimizing data ingress/egress pathways to ensure efficient, scalable analytics workloads.

On-site in Maryland, active TS/SCI w/ Polygraph clearance is required. The candidate should have strong Linux CLI, Python, Java, and Apache Airflow experience, plus familiarity with SQL databases and Git, aiming