Senior Dataflow Engineer - Spark, Airflow, Big Data

Erias Ventures, LLC.

Maryland

On-site

USD 229,000 - 251,000

Full time

17 hours ago
Be an early applicant
Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Benefits offered by this job

Above Market Pay
401k Immediate Vesting
Spot Bonuses
Professional Development Bonuses
Medical Coverage
Vision & Dental
Disability Insurance
Wellness Reimbursement
Paid Time Off
Birthday Off
Internet Reimbursement
Equipment Allowance
Company Events

Job summary

Erias Ventures, LLC is seeking a Software Engineer with expertise in dataflow design and Spark-based processing to design, implement, and optimize analytics pipelines. The role demands a Top-Secret/SCI clearance with polygraph and a strong background in Python and Java development.

You will architect and optimize data ingress/egress pathways, leverage Airflow for complex workflows, and work with SQL-based data stores and large datasets to deliver scalable solutions for mission-focused analytics.

Qualifications

  • Extensive experience with Linux CLI and scripting for automation.
  • Python and Java software development experience.
  • Experience with Airflow for orchestration of complex workflows.
  • Big data processing with Apache Spark.
  • Proficient SQL querying with MySQL, MariaDB, PostgreSQL.
  • Data wrangling with Pandas/NumPy and diverse data formats.
  • Git source control and Jupyter Notebook usage.
  • Awareness of data quality concepts and anomaly detection.

Responsibilities

  • Design, implement, and optimize data ingress/egress pathways for analytics workloads.
  • Develop and maintain Spark-based distributed processing pipelines.
  • Create and monitor workflows with Apache Airflow.
  • Collaborate across disciplines to deliver scalable data solutions.
  • Ensure secure, efficient data processing in support of mission needs.

Skills

Linux CLI
Bash scripting
Python
Java
Apache Airflow
Apache Spark
SQL Databases
Jupyter Notebook
Pandas/NumPy
Parquet/JSON/CSV/XML
Data quality
Git

Education

Master's degree in Computer Science or related
Bachelor's degree in Computer Science or related
9 years SWE experience (alternative)

Tools

Slurm
JIRA
Confluence

Job description

Erias Ventures, LLC is seeking a Software Engineer with expertise in dataflow design and Spark-based processing to design, implement, and optimize analytics pipelines. The role demands a Top-Secret/SCI clearance with polygraph and a strong background in Python and Java development.

You will architect and optimize data ingress/egress pathways, leverage Airflow for complex workflows, and work with SQL-based data stores and large datasets to deliver scalable solutions for mission-focused analytics.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Dataflow Engineer — Linux, Python, Airflow, Spark
Senior Dataflow Engineer — Linux, Python, Airflow, Spark

Captivation • Maryland

On-site
USD 130,000 - 270,000
Annual salary $130k–$270k
20% 401k contribution
HSA contribution
+2
Senior Data Engineer - TS/SCI Clearance, Spark & Airflow
Senior Data Engineer - TS/SCI Clearance, Spark & Airflow

Shield Consulting Solutions, Inc. • Maryland

On-site
USD 230,000 - 240,000
25 days PTO
11 paid holidays
100% employer-paid healthcare for you+
+1
Senior Software Engineer: Airflow, Linux, Python
Senior Software Engineer: Airflow, Linux, Python

Erias Ventures, LLC • Maryland

On-site
USD 213,000 - 233,000
Above Market Salary
Roth/Traditional 401k with immediate v
Spot Bonuses
+5
Senior Data Platform Engineer (TS/SCI) | Airflow & Spark
Senior Data Platform Engineer (TS/SCI) | Airflow & Spark

Shield Consulting Solutions, Inc. • Maryland

Hybrid
USD 205,000 - 215,000
25 days PTO and 11 paid holidays
100% employer-paid healthcare for you
8% 401(k) employer match
Cloud Analytics Engineer — Spark, SQL & SciPy
Cloud Analytics Engineer — Spark, SQL & SciPy

Erias Ventures • Maryland

On-site
USD 225,000 - 250,000
Above Market Pay
401k with Immediate Vesting
Spot Bonuses
+1
Senior Data Processing Engineer – Spark & Airflow (TS/SCI)
Senior Data Processing Engineer – Spark & Airflow (TS/SCI)

Nyla Technology Solutions • Maryland

On-site
USD 199,000 - 235,000
Nyla FLEX program
Paid leave options
Flexible schedule options
+5
Senior Data Engineer – Remote (AWS, Spark, Airflow)
Senior Data Engineer – Remote (AWS, Spark, Airflow)

Digital Envoy, Inc. • Northern (KY)

Hybrid
USD 140,000 - 180,000
Competitive Salary & Bonus program
Medical, Dental and Vision
Paid Holidays & Unlimited PTO
+1
Senior HPC Data Engineer - Spark, Airflow & Linux
Senior HPC Data Engineer - Spark, Airflow & Linux

Erias Ventures, LLC • Annapolis (MD)

On-site
USD 229,000 - 251,000
Above Market Hourly Pay
11% 401k with Immediate Vesting
Professional Development Bonuses
+3
Senior HPC Software Engineer Apache Linux Python (Remote)
Senior HPC Software Engineer Apache Linux Python (Remote)

Erias Ventures, LLC • Annapolis (MD)

Hybrid
USD 213,000 - 233,000
Above Market Pay
401k with vesting
Spot Bonuses
+10
Data Engineer - Spark, Airflow & ETL Pipelines
Data Engineer - Spark, Airflow & ETL Pipelines

TP-Link • Irvine (CA)

On-site
USD 100,000 - 120,000
Free snacks and drinks
Fully paid medical, dental, and vision
401K contribution
+3