Senior Data Processing Engineer – Spark & Airflow (TS/SCI)

Nyla Technology Solutions; Maryland; On-site; USD 199,000 - 235,000; Full time

Benefits offered by this job

Nyla FLEX program

Paid leave options

Flexible schedule options

Health insurance 100% paid

401k matching

Professional development funds

Student loan repayment

Volunteer hours

Job summary

Nyla Technology Solutions is seeking a Software Engineer in Maryland with expert dataflow design and Spark-based processing to build scalable data pipelines for enterprise analytics.

You will design, implement, and optimize end-to-end data ingress/egress, ensuring data quality and system integrity while collaborating with security and data engineering teams. TS/SCI poly clearance is required for this role.

Qualifications

• Experience with distributed Big Data processing engines, especially Apache Spark.
• Hands-on software development with Python and Java.
• Practical experience with Apache Airflow for workflow management.
• Data manipulation with pandas/NumPy and working with Parquet/JSON/CSV/XML formats.
• Familiarity with MySQL/MariaDB/PostgreSQL for large dataset queries.
• Knowledge of data quality, automated validation, and anomaly detection.
• Linux CLI, utilities, and Bash scripting for automation.
• Git source control workflows.

Responsibilities

• Design, implement, and optimize end-to-end data ingress and egress pipelines.
• Ensure scalable, reliable processing of enterprise analytics workloads.
• Maintain data quality and system integrity across data flows.
• Collaborate with data engineering and security teams to meet requirements.

Skills

Spark

Python

Java

Airflow

Data Wrangling

SQL

Linux

Git

Education

Bachelor's degree in a related field

Master's degree in a related field

High School Diploma / GED with experience

Tools

Apache Airflow

Git

Pandas

NumPy

Jupyter

Parquet

JSON

CSV

XML

MySQL

MariaDB

PostgreSQL

Job description

Nyla Technology Solutions is seeking a Software Engineer in Maryland with expert dataflow design and Spark-based processing to build scalable data pipelines for enterprise analytics.

You will design, implement, and optimize end-to-end data ingress/egress, ensuring data quality and system integrity while collaborating with security and data engineering teams. TS/SCI poly clearance is required for this role.