Remote Senior Data Engineer - AI Data Pipelines & Analytics

Allied Benefit Systems; Chicago (IL); Remote; USD 140,000 - 190,000; Full time

Job summary

Allied Benefit Systems is seeking a Senior Data Engineer to design, build, and own the data infrastructure powering analytics, ML, and AI applications across the enterprise. You will be the technical owner of data ingestion, storage, transformation, and reporting infrastructure for the AI team.

The role involves designing scalable data pipelines, API integrations, and production ETL/ELT processes, with a focus on Azure-based platforms, data modeling, and governance to support trusted enterprise

Qualifications

• 5+ years of experience in data engineering, software engineering, or related disciplines.
• Strong Python development skills.
• Advanced SQL expertise.
• Experience developing API integrations and data ingestion solutions.
• Experience designing and operating production ETL/ELT pipelines.
• Experience with cloud-based data platforms (Azure preferred).
• Strong understanding of data modeling and warehousing concepts.
• Experience with healthcare data (claims, provider, or member).
• Familiarity with ML workflows and MLOps concepts.
• Experience with monitoring, observability, and data quality frameworks.
• Experience with Git, CI/CD, IaC, and modern software delivery practices.

Responsibilities

• Design and implement scalable data pipelines and ETL/ELT processes.
• Develop integrations with internal and external data sources, APIs, databases, and enterprise systems.
• Build and maintain data ingestion frameworks for structured and unstructured data.
• Own data movement and orchestration across Azure, Fabric, Snowflake, and other data platforms.
• Develop and maintain data models supporting analytics, reporting, and ML workloads.
• Manage AI-team data storage patterns including data lake, warehouse, and reporting architectures.
• Establish dataset versioning, lineage, and reproducibility practices.
• Create and enforce data quality standards and monitoring procedures.
• Design scalable approaches for growing data volumes and sources.
• Own infrastructure supporting Power BI and reporting platforms for internal reporting or derivative data products.
• Create semantic models, curated datasets, and reporting-ready data products.
• Partner with analytics stakeholders to ensure high-quality, trusted reporting assets.
• Collaborate with ML Engineers and Data Scientists to provide high-quality datasets.
• Collaborate with business/IT teams to understand evolving data requirements.
• Assist in data governance, security, and compliance in healthcare environments.
• Advise on 3rd party data source negotiations.
• Other duties as assigned.

Skills

Python

SQL

ETL/ELT

Azure

Data modeling

Healthcare data

Education

Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent

Tools

Azure Data Lake

Power BI

Snowflake

Databricks

Microsoft Fabric

Git

CI/CD

Job description

Allied Benefit Systems is seeking a Senior Data Engineer to design, build, and own the data infrastructure powering analytics, ML, and AI applications across the enterprise. You will be the technical owner of data ingestion, storage, transformation, and reporting infrastructure for the AI team.

The role involves designing scalable data pipelines, API integrations, and production ETL/ELT processes, with a focus on Azure-based platforms, data modeling, and governance to support trusted enterprise