Principal Data Engineer: Real-Time Data Platform Leader

Medical Guardian; Pennsylvania; On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health Plan

Paid Time Off

Short Term Disability & Life Insurance

401k with Company Match

Job summary

Medical Guardian is seeking a Principal Data Engineer in Pennsylvania to lead the data engineering foundation, focusing on real-time streaming, IoT telemetry, and data services. You will design, build, and operate production pipelines, mentor engineers, and drive architecture decisions across Databricks, Spark, SQL, Python, and Azure services.

This hands-on leadership role emphasizes delivering reliable data capabilities rapidly.

Qualifications

• 10+ years of professional experience in data/ software engineering, data platform engineering, or related fields.
• 7+ years building production data pipelines, data platforms, or data services.
• 5+ years with modern cloud data platforms including Databricks, Spark, Delta Lake, SQL, Python/PySpark.
• 3+ years leading and mentoring data engineers or related teams.
• Strong experience with Azure cloud services for data engineering, streaming, and operations.
• Experience designing real-time streaming and event-driven data pipelines in production.

Responsibilities

• Design, build, optimize, and operate production-grade batch and streaming data pipelines on Azure and Databricks.
• Develop ETL/ELT workflows to ingest, transform, validate, and serve large volumes of data.
• Build data products, data services, APIs, and microservices supporting analytics and ML teams.
• Lead real-time streaming ingestion for IoT telemetry and operational data feeds.
• Define reliability, latency, observability, and supportability for streaming systems.
• Mentor engineers, lead design reviews, and enforce engineering standards.

Skills

Leadership

Stakeholder management

Data modeling

Data quality

Databricks

Python/PySpark

SQL

Streaming

ETL/ELT

CI/CD

Data pipelines

Tools

Databricks

Azure

Spark

Delta Lake

Python/PySpark

SQL

CI/CD

Job description

Medical Guardian is seeking a Principal Data Engineer in Pennsylvania to lead the data engineering foundation, focusing on real-time streaming, IoT telemetry, and data services. You will design, build, and operate production pipelines, mentor engineers, and drive architecture decisions across Databricks, Spark, SQL, Python, and Azure services.

This hands-on leadership role emphasizes delivering reliable data capabilities rapidly.