Sr. Data Engineer

MBO Partners; United States; Remote; USD 90,000 - 130,000; Part time

Benefits offered by this job

Eligibility for Paid Sick Leave (PSL)

Flexible working hours

Job summary

A leading independent jobs platform is seeking an experienced Data Engineer to work remotely. You will design and implement data pipelines while optimizing data flows using technologies like Spark and Kafka. Candidates should have a minimum of 5 years in data engineering, preferably with experience in production environments. This role requires a strong understanding of data formats, streaming, and query optimization, with a focus on driving innovation in the civil market.

Qualifications

• 5+ years of experience designing, building, and maintaining data pipelines in production environments.
• 3+ years of experience with Spark (Databricks preferred) and distributed systems.
• Ability to obtain and maintain a Public Trust clearance.

Responsibilities

• Work with large, complex datasets and build robust ETL/ELT pipelines.
• Optimize data flows using technologies like Databricks, Spark, and Kafka.

Skills

Designing data pipelines

Building with Spark

Processing data with Kafka

Proficiency in Python

Implementing data quality best practices

Managing structured/unstructured data

Tools

Databricks

Spark

Kubernetes

AWS

Job description

MBO Partners is a deep jobs platform that connects and enables independent professionals and microbusiness owners to do business safely and effectively with enterprise organizations. Its unmatched experience and industry leadership enable it to operate on the forefront of the independent economy and consistently advance the next way of working.

Location: Remote

Anticipated Hours: 40 per week

Duration: As soon as cleared through September 2026

Security Clearance: Must be able to obtain and maintain a Public Trust

Job Description:

We’re looking for an experienced data engineer who thrives on solving applied problems with code, modern tooling, and scalable infrastructure. You’ll work with large, complex datasets, build robust ETL/ELT pipelines, and optimize data flows using technologies like Databricks, Spark, and Kafka.

In this role, you’ll grow your technical expertise, apply best practices and use tools like Spark, Kafka, EKS, and others. With your drive to establish processes and lead technological innovation, you’ll make a lasting impact on the civil market.

Join us. The world can’t wait.

You Have:

• 5+ years of experience designing, building, and maintaining data pipelines in production environments
• 3+ years of experience developing with Spark (Databricks preferred), PySpark, or similar distributed systems
• 3+ years of experience with data lakehouse or warehouse platforms, schema design, and query optimization
• 3+ years of experience processing data using streaming (Kafka, Kinesis, etc.) and batch methods
• Proficiency in Python and SQL (Scala or Java a plus)
• Experience implementing best practices for data quality, testing, and observability
• Strong understanding of structured/unstructured data formats (Parquet, Avro, JSON, Delta)
• Knowledge of data, information, and message exchange structures and standards
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements

Nice If You Have:

• 3+ years of experience in data analytics
• Experience designing data flows that leverage a medallion architecture
• Experience with containerization and orchestration (Docker, Kubernetes, EKS)
• Experience with Kafka
• Experience with Nifi
• Experience using AWS
• Knowledge of microservices and integrating with data services
• Knowledge of database schema design

Eligibility Requirements:

• Legal authorization to work in the U.S. is required.
• As a contractor, including remote contractors, you may be required to complete a background check.

A contractor will be paid for the time you work; this does not include paid time off (PTO) or holidays. If you participate in our Payroll Services (W2) engagement, you may be eligible for Paid Sick Leave (PSL), depending on your work location and state-specific regulations.

Seniority level

• Mid‑Senior level

Employment type

• Contract

Job function

• Management, Consulting, and Information Technology

Industries

• Business Consulting and Services, Professional Services, and Government Administration