Principal Data Engineer (Snowflake & Azure Lakehouse | Technical Lead)

TECHNOLOGY PARTNERS INC; St. Louis (MO); On-site; USD 103,000 - 172,000; Full time

Job summary

Technology Partners is seeking a Principal Data Engineer to lead a modern Snowflake + Azure lakehouse data modernization for large-scale claims data. You will own architectural outcomes, drive design decisions, and mentor a team of engineers in a fast-paced, client-facing environment.

Responsibilities include delivering end-to-end data architectures, implementing Medallion patterns, real-time ingestion, governance, and observable pipelines with focus on reliability and business impact.

Qualifications

• 5+ years in data engineering with at least 2+ years in a technical lead or principal role.
• Deep hands-on Snowflake architecture and optimization experience.
• Strong SQL skills with complex transformations and performance tuning.
• Experience with dbt Core for modeling and testing.
• Proven ability to operate in client-facing consulting environments.

Responsibilities

• Lead design and implementation of end-to-end data architecture across Snowflake and Azure environments.
• Build and deliver multiple data-architecture patterns (Snowflake-native, hybrid Azure + Snowflake, event-driven ingestion).
• Implement Medallion Architecture with full lineage, governance, and promotion rules.
• Design real-time ingestion pipelines using Snowpipe and Event Grid.
• Develop scalable pipelines for high-volume ingestion using Snowpark (Python/Scala).
• Mentor engineering teams and manage technical risks and timelines.

Skills

Snowflake

Snowpark

Advanced SQL

dbt Core

Azure Cloud

Data Governance

Leadership

Client-facing

ETL Pipelines

Python/Scala

Education

Advanced degree in Computer Science or Data Engineering

Tools

Azure Data Factory

Snowpipe

Event Grid

ADLS Gen2

Notebook Workflows

Streamlit

Job description

Technology Partners is currently seeking a talentedPrincipal Data Engineer (Snowflake & Azure Lakehouse | Technical Lead).

Do you have experience leading large-scale Snowflake data architectures, building production-grade pipelines, and driving client-facing data engineering engagements?

Let us help you make your next big career move a reality!

What You Will Be Doing:

You will serve as the hands-on technical lead for a high-stakes, enterprise data modernization program focused on transforming large-scale claims data ingestion into a modern Snowflake + Azure lakehouse architecture. This is a delivery-critical role where you own the technical outcome, lead architecture decisions, build core pipelines, and act as the primary engineering voice with senior client stakeholders. You will operate in a fast-paced, high-visibility environment, balancing execution, leadership, and client engagement.

Key Responsibilities:

• Lead the design and implementation ofend-to-end data architectureacross Snowflake and Azure environments
• Build and deliver multipledata architecture patterns(Snowflake-native, hybrid Azure + Snowflake, event-driven ingestion, and notebook‑based workflows)
• ImplementMedallion Architecture (Raw ? Silver ? Gold)with full lineage, governance, and promotion rules aligned with business definitions
• Design and implementreal‑tim ingestion pipelinesusing Snowpipe, Event Grid, and secure file ingestion (e.g., PGP-encrypted sources)
• Develop scalable pipelines for high-volume, multi-source data ingestion, transformation, and validation usingSnowpark (Python/Scala)
• Build and optimizedbt Core modelsincluding transformations, testing, and data quality enforcement
• Configure and manageAzure data services(ADLS Gen2, Azure Data Factory, Blob Storage, Private Endpoints, Managed Identity)
• Develop and maintainoperational dashboardsfor monitoring pipeline health, SLA adherence, and data quality (targeting 99.5% uptime)
• Leadclient-facing technical discussions, demos, and working sessionswith executive stakeholders
• Translate complex technical decisions into clear business outcomes and data value narratives
• Scope work, estimate delivery timelines, and manage technical risks and change requests
• Lead and mentor supporting engineering resources while ensuring high-quality deliverables
• Drive AI/ML integration initiatives includingSnowflake Cortex, Streamlit, and Azure AI services
• Ensure all architecture, pipelines, and transformations are well-documented and production-ready

Required Skills & Experience:

• 5+ years of experience indata engineering, with at least 2+ years in atechnical lead or principal role
• Deep, hands‑on expertise in:
  • Snowflake architecture and optimization
  • Snowpark (Python/Scala)for pipeline development
  • Advanced SQL(complex transformations, performance tuning, schema design)
  • dbt Core(modeling, testing, incremental processing, source management)
• Strong experience withAzure cloud ecosystem, including:
  • ADLS Gen2, Azure Data Factory, Azure Functions
  • Event Grid, Blob Storage, Private Endpoints, Managed Identity
• Proven experience buildingevent-driven and streaming data pipelines(e.g., Snowpipe)
• Experience implementingMedallion/Lakehouse architecture patternsin production environments
• Experience withdata governance tools(e.g., Microsoft Purview or equivalent)
• Proven ability to operate inclient-facing consulting environments, managing stakeholders and delivering under pressure
• Strong leadership, communication, and problem‑solving skills
• Ability to balance speed of delivery with system reliability and scalability

Desired Skills & Experience:

• Experience withAI/ML integration on data platforms(Snowflake Cortex, Streamlit, Azure AI Foundry, or similar)
• Exposure toPGP/GPG encryption workflowsin automated pipelines
• Familiarity withIceberg table formatsor modern table storage frameworks
• Experience buildinglightweight data applications or UI layers(e.g., Streamlit)
• Domain knowledge ininsurance, claims, or TPA data ecosystems
• Experience scaling data platforms across multiple business domains
• Advanced degree in Computer Science, Data Engineering, or related quantitative field

Pay:

$87.50 - $125.00 /hr.

We are interested in every qualified candidate who is eligible to work in the United States. However, we are not able to provide sponsorship at this time or accept candidates who would require a corp-to-corp agreement.

If this position sounds like you,WE SHOULD TALK!

Your better future is ready, and we want to put the right tools in your hands to get you there. Let's go!

Keywords:

Principal Data Engineer, Snowflake, Snowpark, Azure, ADLS Gen2, Data Lakehouse, Medallion Architecture, dbt Core, SQL, Python

Looking for more opportunities with Technology Partners? Check out technologypartners.net/jobs!

All offers of employment at Technology Partners are contingent upon clear results of a thorough background check and drug screening that meet corresponding laws and regulations at the city, state and federal level.

Pay ranges are influenced by candidate qualifications, experience, and role specifics, with the actual rate determined considering skills, market conditions, and are subject to change by the employer; pay negotiations follow all state and federal legal guidelines.