Senior Data Engineer — Scalable Analytics & Spark

engineeringjobs.net, Inc.; Maryland; On-site; USD 130,000 - 180,000; Full time

Benefits offered by this job

Paid time off

Holidays

Healthcare for employees and dependets

401(k) match

Immediate 401(k) vesting

Job summary

engineeringjobs.net, Inc. seeks a software engineer to design, implement, and optimize data ingress/egress for analytics workloads.

You will apply dataflow design and data transport mechanisms with Apache Spark to support scalable, reliable data processing across the organization. The role requires seven years of software engineering experience, a technical bachelor's degree (or 2 extra years for substitution), and an active TS/SCI clearance with polygraph, plus strong Python/Java, Linux,

Qualifications

• Requires seven years of software engineering experience.
• Bachelor's degree in a technical discipline with 2 years substitution allowed.
• Active TS/SCI clearance with polygraph required.
• Recent Python and Java development experience required.
• Experience with Linux, Airflow, Spark, SQL databases, data formats, data quality, and Git.

Responsibilities

• Design, implement, and optimize data ingress and egress pathways for analytics workloads.
• Apply dataflow design and data transport mechanisms to support organizational data processing.
• Utilize Apache Spark distributed processing to enable scalable analytics workloads.

Skills

Linux

Bash

Python

Java

Apache Airflow

Apache Spark

SQL

MySQL

MariaDB

PostgreSQL

Jupyter Notebook

Pandas

NumPy

Data Wrangling

Git

Dataflow Design

Education

Bachelor's degree in a technical discipline

2 years substitution for degree

Tools

Airflow

Spark

Jupyter

Pandas

NumPy

Git

Dataflow Design

Job description

engineeringjobs.net, Inc. seeks a software engineer to design, implement, and optimize data ingress/egress for analytics workloads.

You will apply dataflow design and data transport mechanisms with Apache Spark to support scalable, reliable data processing across the organization. The role requires seven years of software engineering experience, a technical bachelor's degree (or 2 extra years for substitution), and an active TS/SCI clearance with polygraph, plus strong Python/Java, Linux,