Senior Data Engineer

Foundation Partners Group; United States; On-site; USD 130,000 - 190,000; Full time

Job summary

Foundation Partners Group is seeking a Senior Data Engineer to design, build, and operate modern data platforms at scale. You'll work across Azure, Microsoft Fabric, and AWS, building reliable pipelines, performant data models, and self-service analytics that drive real business decisions.

A core part of this role is containerized workload execution: designing and deploying pipeline jobs using Azure Container Apps Jobs for scheduled and event-driven processing.

Qualifications

• 5-8 years of hands-on data engineering in production environments.
• Proven track record delivering data platforms on Azure and/or AWS.
• Experience migrating legacy on-premises infrastructure to cloud-native solutions.
• Hands-on experience with Azure Container Apps Jobs or a comparable containerized job execution platform.

Responsibilities

• Develop Python-based ETL/ELT pipelines ingesting data from APIs, flat files, databases, and SaaS platforms.
• Build and deploy containerized jobs using Azure Container Apps Jobs, including scheduling, scaling rules, secrets management (Key Vault), and Container Registry integration.
• Maintain data movement between on-premises SQL Server and cloud targets.
• Design idempotent, fault-tolerant pipeline patterns with robust logging, alerting, and retry logic.
• Partner with analytics teams to deliver clean, well-documented data models for Power BI and similar BI tools.

Skills

Python

SQL

Microsoft Fabric

Azure

AWS

Data Modeling

Education

Bachelor's degree in Computer Science, Information Systems, Data Science, or related field

Tools

Azure Data Factory

Azure Container Apps Jobs

Fabric Data Pipelines

Azure SQL

AWS S3

Redshift

Container Registry

Key Vault

Job description

We're looking for a Senior Data Engineer to design, build, and operate modern data platforms at scale. You'll work across Azure, Microsoft Fabric, and AWS - building reliable pipelines, performant data models, and self-service analytics that drive real business decisions. A core part of this role is containerized workload execution: designing and deploying pipeline jobs using Azure Container Apps Jobs for scheduled and event-driven processing. This is a hands-on role for an engineer who thrives in ambiguity, values clean architecture, and moves comfortably between strategy and implementation.

About the Role

• Design and build scalable data pipelines using Python and cloud-native orchestration tools (Azure Data Factory, Azure Container Apps Jobs, Fabric Data Pipelines)
• Architect solutions across Microsoft Fabric Warehouses, Azure SQL, and AWS (S3, Redshift), selecting the right tool for the workload
• Manage and optimize large-scale data warehouse environments for performance, cost, and maintainability

Responsibilities

• Develop Python-based ETL/ELT pipelines ingesting data from APIs, flat files, databases, and SaaS platforms
• Build and deploy containerized jobs using Azure Container Apps Jobs, including scheduling, scaling rules, secrets management (Key Vault), and Container Registry integration
• Maintain data movement between on-premises SQL Server and cloud targets
• Design idempotent, fault-tolerant pipeline patterns with robust logging, alerting, and retry logic
• Partner with analytics teams to deliver clean, well-documented data models for Power BI and similar BI tools

Qualifications

• 5-8 years of hands-on data engineering in production environments
• Proven track record delivering data platforms on Azure and/or AWS
• Experience migrating legacy on-premises infrastructure to cloud-native solutions
• Hands-on experience with Azure Container Apps Jobs or a comparable containerized job execution platform

Required Skills

• Python — production-grade pipelines, transformations, and automation
• SQL — advanced T-SQL and ANSI SQL; SQL Server, Azure SQL, Redshift, and Fabric query engines
• Microsoft Fabric — Warehouses, Lakehouses, Data Pipelines, OneLake
• AWS — S3 for data lake storage, Redshift for cloud warehousing
• Data Modeling — dimensional modeling, star/snowflake schema, ER modeling for OLTP and OLAP

Preferred Skills

• Microsoft Fabric in production (Fabric Warehouses, OneLake integration)
• dbt or similar transformation frameworks
• Cloud certifications: DP-203 (Azure Data Engineer), AWS Certified Data Analytics, or equivalent
• Bachelor's degree in Computer Science, Information Systems, Data Science, or a related field — or equivalent professional experience