Senior Data Engineer, Emerging Products - Lakehouse

NinjaOne; Connecticut; Hybrid; USD 110,000 - 200,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) plan

Unlimited PTO

Hybrid remote availability

Career growth opportunities

Job summary

NinjaOne is seeking a Senior Data Engineer to help build the foundation of a modern data platform. You'll design Medallion-layer lakehouse architectures and develop robust streaming and batch pipelines using Kafka, Spark, and Airflow, ensuring data quality and reliability across the organization.

You will work with Databricks and various table formats (Iceberg/Delta/Hudi) while collaborating with data scientists, analysts, and product teams to deliver scalable, data-driven solutions.

Qualifications

• 10+ years of experience in data engineering with a strong focus on data platform and distributed systems.
• Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or equivalent work experience preferred.
• Hands-on experience building and managing lakehouse architectures, including Medallion (Bronze/Silver/Gold) layering patterns.
• Deep expertise in Apache Kafka, Apache Spark, and Apache Airflow for both streaming and batch pipeline development.
• Proficiency in open table formats - Iceberg, Delta Lake, or Hudi - with the ability to evaluate trade-offs and select the right format for the workload.
• Experience with Databricks for large-scale data processing and analytics workloads.
• Strong proficiency in Python and SQL.
• Experience that will make you a standout candidate: Hands-on experience with Trino or Starburst for distributed query execution; Familiarity with hybrid streaming/batch architectures and event-driven data systems; Experience designing Gold-layer datasets optimized for BI and self-serve analytics.

Responsibilities

• Lakehouse Architecture: Design and implement Medallion-layer data architectures (Bronze, Silver, Gold), building structured ingestion pipelines and curated data layers that serve downstream analytics and data science workloads.
• Pipeline Development: Build and maintain scalable streaming and batch data pipelines using Kafka, Spark, and Airflow, supporting reliable and low-latency data movement across the platform.
• Table Format Management: Own and manage open table formats including Iceberg, Delta, and Hudi - selecting the right format for the workload and maintaining performance and reliability at scale.
• Query Engine Optimization: Work with Trino/Starburst and Databricks-based environments to optimize large-scale analytics queries and support platform consumers.
• Platform Reliability: Monitor pipeline health, troubleshoot data quality and performance issues, and continuously improve the reliability and efficiency of the data platform.
• Collaboration: Partner closely with data scientists, analysts, and product teams to understand data requirements and deliver platform solutions that enable data-driven decision-making.
• Other duties as needed.

Skills

Lakehouse

Kafka

Spark

Airflow

Databricks

Python

SQL

Medallion patterns

Iceberg

Delta Lake

Hudi

Trino/Starburst

Hybrid streaming/batch

Distributed systems

Education

Bachelor's degree in CS/CE/IT or equivalent

Tools

Iceberg

Delta Lake

Hudi

Kafka

Spark

Airflow

Databricks

Job description

NinjaOne is seeking a Senior Data Engineer to help build the foundation of a modern data platform. You'll design Medallion-layer lakehouse architectures and develop robust streaming and batch pipelines using Kafka, Spark, and Airflow, ensuring data quality and reliability across the organization.

You will work with Databricks and various table formats (Iceberg/Delta/Hudi) while collaborating with data scientists, analysts, and product teams to deliver scalable, data-driven solutions.