Remote Senior Data Engineer — ETL, Spark & Kafka

MBO Partners; United States; Remote; USD 90,000 - 130,000; Part time

Benefits offered by this job

Eligibility for Paid Sick Leave (PSL)

Flexible working hours

Job summary

A leading independent jobs platform is seeking an experienced Data Engineer to work remotely. You will design and implement data pipelines while optimizing data flows using technologies like Spark and Kafka. Candidates should have a minimum of 5 years in data engineering, preferably with experience in production environments. This role requires a strong understanding of data formats, streaming, and query optimization, with a focus on driving innovation in the civil market.

Qualifications

• 5+ years of experience designing, building, and maintaining data pipelines in production environments.
• 3+ years of experience with Spark (Databricks preferred) and distributed systems.
• Ability to obtain and maintain a Public Trust clearance.

Responsibilities

• Work with large, complex datasets and build robust ETL/ELT pipelines.
• Optimize data flows using technologies like Databricks, Spark, and Kafka.

Skills

Designing data pipelines

Building with Spark

Processing data with Kafka

Proficiency in Python

Implementing data quality best practices

Managing structured/unstructured data

Tools

Databricks

Spark

Kubernetes

AWS

Job description

A leading independent jobs platform is seeking an experienced Data Engineer to work remotely. You will design and implement data pipelines while optimizing data flows using technologies like Spark and Kafka. Candidates should have a minimum of 5 years in data engineering, preferably with experience in production environments. This role requires a strong understanding of data formats, streaming, and query optimization, with a focus on driving innovation in the civil market.