Senior Big Data Engineer — Scale Data Pipelines on AWS

BitSight; Raleigh (NC); Hybrid; USD 119,000 - 150,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Paid parental leave

Flexible time off

401(k) plan with company match

Life and disability insurance

Tuition reimbursement

Job summary

Bitsight is seeking a Senior Software Engineer (Big Data) to design, build, and optimize data pipelines that ingest, process, and analyze large-scale security data. You will work with Java, Python, Hadoop, Spark, and data lakehouse technologies to deliver scalable, reliable systems.

As part of a collaborative team across multiple locations, you will contribute to AWS-based infrastructure, CI/CD practices, and data-driven features that surface actionable insights for customers.

Qualifications

• Experience building and operating large-scale data pipelines.
• Strong proficiency in Java with solid CS fundamentals.
• Experience in Python and big data frameworks like Hadoop/Spark.
• Familiarity with data lakehouse architectures and search tech (Elasticsearch/OpenSearch).
• CI/CD and Kubernetes experience; ability to collaborate with cross-functional teams.

Skills

Big Data

Java

Python

Spark

Hadoop

Data Lakehouse

Elasticsearch/OpenSearch

CI/CD

Kubernetes

AI Tools

Tools

AWS

GitHub Copilot

Cursor

Job description

Bitsight is seeking a Senior Software Engineer (Big Data) to design, build, and optimize data pipelines that ingest, process, and analyze large-scale security data. You will work with Java, Python, Hadoop, Spark, and data lakehouse technologies to deliver scalable, reliable systems.

As part of a collaborative team across multiple locations, you will contribute to AWS-based infrastructure, CI/CD practices, and data-driven features that surface actionable insights for customers.