Senior AI Engineer — Enterprise Data Pipelines

Calance; United States; On-site; USD 140,000 - 180,000; Full time

Job summary

Calance is seeking a Senior AI Engineer to design and maintain production-grade decisioning and data processing systems in a rules-driven enterprise environment.

The role focuses on Python-based decision pipelines, business rule implementation, and CI/CD delivery, with emphasis on reliable, scalable production systems. Expect hands-on work with Databricks, PySpark, and Pandas, plus Azure AI Foundry and OpenAI integrations as needed.

Qualifications

• Strong, hands-on experience with Python for production systems.
• Hands-on experience with Databricks for enterprise data engineering and production pipelines.
• Hands-on experience with Azure AI Foundry for developing and deploying enterprise AI solutions.
• Advanced proficiency with PySpark and Pandas.
• Proven experience implementing business rules or rule-driven systems.
• Experience with OpenAI or other LLMs in enterprise apps.
• Familiarity with Git-based delivery models and CI/CD pipelines.

Responsibilities

• Design, build, and support scalable Python-based processing and decisioning pipelines using Databricks, PySpark, and Pandas.
• Develop and maintain enterprise data processing workflows within the Databricks platform.
• Translate complex business rules and domain logic into clean, testable, and maintainable decisioning systems.
• Build modular, reusable components for orchestrated, production workflows.
• Develop and integrate AI-powered solutions using Azure AI Foundry, including OpenAI and other LLM services when required.
• Integrate internal and external APIs with robust error handling and retry strategies.
• Implement data validation, structured logging, and monitoring for production observability.
• Participate in Git-based development with branching, PRs, code reviews, and conflict resolution.
• Build and maintain CI/CD pipelines in Azure DevOps with release workflows and deployment monitoring.
• Support and troubleshoot live production systems.

Skills

Python

Databricks

PySpark

Pandas

Azure AI Foundry

OpenAI models

Git

Azure DevOps

CI/CD

Production pipelines

Tools

Databricks

Azure DevOps

Airflow

ADF

Job description

Calance is seeking a Senior AI Engineer to design and maintain production-grade decisioning and data processing systems in a rules-driven enterprise environment.

The role focuses on Python-based decision pipelines, business rule implementation, and CI/CD delivery, with emphasis on reliable, scalable production systems. Expect hands-on work with Databricks, PySpark, and Pandas, plus Azure AI Foundry and OpenAI integrations as needed.