Principal Data Platform Leader - Real-Time IoT & Analytics

Medical Guardian LLC; Philadelphia (Philadelphia County); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health Care Plan (Medical, Dental &amp

Paid Time Off (Vacation, Sick Time Off

Company Paid Short Term Disability and

Retirement Plan (401k) with Company

Job summary

Medical Guardian is seeking a Principal Data Engineer to lead and contribute hands-on to building real-time data pipelines and platform services. You will own architecture, development, and production readiness for data streams, APIs, and analytics data products, while mentoring engineers and shaping priorities with stakeholders.

The role emphasizes Databricks, Spark, Python/PySpark, and Azure services, with a focus on reliability, scalability, and measurable business impact in a fast-paced

Qualifications

• 10+ years in data engineering, software engineering, or data platforms.
• 7+ years building production data pipelines and data services.
• 5+ years cloud data platforms (Databricks, Spark, Delta Lake, SQL, Python/PySpark) and orchestration.
• 3+ years leading data engineers or technical teams.

Responsibilities

• Hands-on design, build, and operation of production-grade data pipelines on Azure and Databricks, including real-time IoT telemetry.
• Develop ETL/ELT workflows for large structured, semi-structured, and streaming data.
• Create data products, APIs, and microservices for analytics and ML/AI teams.
• Lead streaming ingestion patterns for real-time data and define reliability, latency, and observability goals.
• Provide technical leadership through reviews, standards, and governance for data platforms.

Skills

Databricks

Spark

Delta Lake

Python/PySpark

Azure

CI/CD

Data pipelines

Real-time streaming

Leadership

Stakeholder management

Tools

Azure Event Hubs

Azure Stream Analytics

Azure Service Bus

Azure Data Factory

Azure Functions

ADLS

Azure Cosmos DB

Databricks

Spark

CI/CD tooling

Job description

Medical Guardian is seeking a Principal Data Engineer to lead and contribute hands-on to building real-time data pipelines and platform services. You will own architecture, development, and production readiness for data streams, APIs, and analytics data products, while mentoring engineers and shaping priorities with stakeholders.

The role emphasizes Databricks, Spark, Python/PySpark, and Azure services, with a focus on reliability, scalability, and measurable business impact in a fast-paced