Senior Data Engineer

Allied Benefit Systems; Chicago (IL); Remote; USD 140,000 - 190,000; Full time

Job summary

Allied Benefit Systems is seeking a Senior Data Engineer to design, build, and own the data infrastructure powering analytics, ML, and AI applications across the enterprise. You will be the technical owner of data ingestion, storage, transformation, and reporting infrastructure for the AI team.

The role involves designing scalable data pipelines, API integrations, and production ETL/ELT processes, with a focus on Azure-based platforms, data modeling, and governance to support trusted enterprise

Qualifications

• 5+ years of experience in data engineering, software engineering, or related disciplines.
• Strong Python development skills.
• Advanced SQL expertise.
• Experience developing API integrations and data ingestion solutions.
• Experience designing and operating production ETL/ELT pipelines.
• Experience with cloud-based data platforms (Azure preferred).
• Strong understanding of data modeling and warehousing concepts.
• Experience with healthcare data (claims, provider, or member).
• Familiarity with ML workflows and MLOps concepts.
• Experience with monitoring, observability, and data quality frameworks.
• Experience with Git, CI/CD, IaC, and modern software delivery practices.

Responsibilities

• Design and implement scalable data pipelines and ETL/ELT processes.
• Develop integrations with internal and external data sources, APIs, databases, and enterprise systems.
• Build and maintain data ingestion frameworks for structured and unstructured data.
• Own data movement and orchestration across Azure, Fabric, Snowflake, and other data platforms.
• Develop and maintain data models supporting analytics, reporting, and ML workloads.
• Manage AI-team data storage patterns including data lake, warehouse, and reporting architectures.
• Establish dataset versioning, lineage, and reproducibility practices.
• Create and enforce data quality standards and monitoring procedures.
• Design scalable approaches for growing data volumes and sources.
• Own infrastructure supporting Power BI and reporting platforms for internal reporting or derivative data products.
• Create semantic models, curated datasets, and reporting-ready data products.
• Partner with analytics stakeholders to ensure high-quality, trusted reporting assets.
• Collaborate with ML Engineers and Data Scientists to provide high-quality datasets.
• Collaborate with business/IT teams to understand evolving data requirements.
• Assist in data governance, security, and compliance in healthcare environments.
• Advise on 3rd party data source negotiations.
• Other duties as assigned.

Skills

Python

SQL

ETL/ELT

Azure

Data modeling

Healthcare data

Education

Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent

Tools

Azure Data Lake

Power BI

Snowflake

Databricks

Microsoft Fabric

Git

CI/CD

Job description

Position Summary

The Senior Data Engineer will design, build, and own the data infrastructure that powers analytics, machine learning, and AI applications across the enterprise. The role will serve as the technical owner of data ingestion, storage, transformation, and reporting infrastructure for the AI team.

Essential Functions

• Design and implement scalable data pipelines and ETL/ELT processes.
• Develop integrations with internal and external data sources, APIs, databases, and enterprise systems.
• Build and maintain data ingestion frameworks for structured and unstructured data.
• Own data movement and orchestration across platforms such as Azure Storage, Microsoft Fabric, Snowflake, or future enterprise data solutions.
• Develop and maintain data models supporting analytics, reporting, and machine learning workloads.
• Manage AI-team data storage patterns, including data lake, warehouse, and reporting architectures.
• Establish and maintain dataset versioning, lineage, and reproducibility practices.
• Create and enforce data quality standards and monitoring procedures.
• Design scalable approaches for handling growing data volumes and data sources.
• Own the technical infrastructure supporting Power BI and future reporting platforms that the AI team uses for internal reporting or to produce derivative data products for the enterprise.
• Create semantic models, curated datasets, and reporting-ready data products.
• Partner with analytics stakeholders to ensure high-quality, trusted reporting assets.
• Partner with Machine Learning Engineers and Data Scientists to provide high-quality datasets for model development.
• Collaborate with business and IT teams to understand evolving data requirements.
• Assist in establishing data governance, security, and compliance practices appropriate for healthcare data environments.
• Advise business teams during 3rd party data source negotiations.
• Other duties as assigned.

EDUCATION

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent combination of education and work experience required.

Experience And Skills

• 5+ years of experience in data engineering, software engineering, or related disciplines.
• Strong Python development skills.
• Advanced SQL expertise.
• Experience developing API integrations and data ingestion solutions.
• Experience designing and operating production ETL/ELT pipelines.
• Experience with cloud-based data platforms (Azure preferred).
• Strong understanding of data modeling and warehousing concepts.
• Experience with Microsoft Fabric, Azure Data Lake, Power BI, Snowflake, Databricks, or similar technologies.
• Experience working with healthcare, claims, provider, or member data.
• Familiarity with machine learning workflows and MLOps concepts.
• Experience implementing monitoring, observability, and data quality frameworks.
• Experience with Git, CI/CD, Infrastructure-as-Code, and modern software delivery practices
• Experience with coding assistants, e.g., Claude Code, for feature-oriented development

Position Competencies

• Accountability
• Analytical Problem Solving
• Collaboration
• Quality and Risk Management
• Systems Thinking
• Technical/Functional Expertise

Physical Demands

• This is a standard desk role requiring extended sitting and computer work

Work Environment

• Remote

Here at Allied, we believe that great talent can thrive from anywhere. Our remote friendly culture offers flexibility and the comfort of working from home, while also ensuring you are set up for success. To support a smooth and efficient remote work experience, the internet connection must be obtained through a cable broadband or fiber optic internet service provider with speeds of at least 100Mbps download/25Mbps upload. Reliable internet service is essential for staying connected and productive.

The company has reviewed this job description to ensure that essential functions and basic duties have been included. It is not intended to be construed as an exhaustive list of all functions, responsibilities, skills, and abilities. Additional functions and requirements may be assigned by supervisors as deemed appropriate.

Compensation is not limited to base salary. Allied values our Total Rewards, and offers a competitive Benefit Package including, but not limited to, Medical, Dental, Vision, Life and Disability Insurance, Generous Paid Time Off, Tuition Reimbursement, EAP, and a Technology Stipend.

Allied reserves the right to amend, change, alter, and revise, pay ranges and benefits offerings at any time. All applicants acknowledge that by applying to the position you understand that the specific pay range is contingent upon meeting the qualification and requirements of the role, and for the successful completion of the interview selection and process. It is at the Company's discretion to determine what pay is provided to a candidate within the range associated with the role.