Data & AI / LLM Engineer

SANS Consulting Services, Inc.; United States; On-site; USD 150,000 - 230,000; Full time

Job summary

SANS Consulting Services, Inc. seeks a Senior Data & AI / LLM Engineer to design, build, and operationalize data, ML, and Generative AI solutions in the insurance domain.

This is a hands-on engineering role spanning the full lifecycle from problem understanding to production deployment on Azure. The role emphasizes Databricks, Spark, Delta Lake, Python, and Git-based DevOps, with strong collaboration across data scientists, SMEs, and engineering teams to productionize models and enterprise AI

Qualifications

• 15+ years of experience in data/AI engineering within insurance or a related domain.
• Proficient in building scalable data pipelines and ML/LLM solutions.

Responsibilities

• Design, build, test, and maintain scalable data pipelines and data products using Databricks, Spark/PySpark, Delta Lake, SQL, and Python.
• Perform exploratory and advanced data analysis to identify patterns, anomalies, trends, relationships, and opportunities within complex insurance datasets.
• Build reliable ingestion, transformation, integration, and data-quality processes across structured and unstructured data.
• Work with large-scale datasets across cloud data platforms, APIs, files, databases, and enterprise applications.
• Apply sound practices around data quality, lineage, observability, metadata, security, and governance.
• Optimize pipelines and workloads for reliability, performance, scalability, and cost.
• Develop, evaluate, deploy, and monitor machine learning models using Python and appropriate ML frameworks.
• Understand core ML techniques including supervised and unsupervised learning, classification, regression, clustering, feature engineering, model evaluation, and model monitoring.
• Translate business problems into analytical or machine-learning approaches and determine when traditional analytics, ML, or Generative AI is most appropriate.
• Work with data scientists, business SMEs, and engineering teams to productionize models and analytical solutions.
• Design and implement enterprise LLM applications, including Retrieval-Augmented Generation (RAG), document processing, summarization, extraction, classification, conversational assistants, and agentic workflows.
• Develop solutions using LLM APIs and frameworks, with experience in prompt engineering, embeddings, vector search, chunking, grounding, tool/function calling, and structured outputs.
• Understand evaluation techniques for LLM solutions, including accuracy, relevance, hallucination risk, latency, cost, security, and responsible AI considerations.
• Integrate structured enterprise data and unstructured content into secure and governed AI solutions.
• Build reusable AI components and APIs that can be incorporated into broader enterprise applications.

Skills

Python

SQL

Databricks

LLMs/Generative AI

Azure

Education

Bachelor's or Master's degree in Computer Science, Data Science, AI, Engineering, or related field

Tools

Azure Databricks

Azure AI services

MLflow/MLOps

Vector databases

Job description

We are seeking an 15 years+ experienced Senior Data & AI / LLM Engineer to design, build, and operationalize modern data, machine learning, and Generative AI solutions within the insurance domain.

This is a hands-on engineering role suited to someone who can work across the full lifecycle, from understanding insurance business problems and analyzing data, through building scalable data pipelines and ML/LLM solutions, to deploying production-quality code using Azure and DevOps practices.

The successful candidate will also be expected to embrace modern AI-assisted software engineering, using tools such as GitHub Copilot and Claude to accelerate development, refactoring, testing, documentation, and maintenance while maintaining appropriate engineering, security, and code-quality standards.

Key Responsibilities

• Design, build, test, and maintain scalable data pipelines and data products using Databricks, Apache Spark/PySpark, Delta Lake, SQL, and Python.
• Perform exploratory and advanced data analysis to identify patterns, anomalies, trends, relationships, and opportunities within complex insurance datasets.
• Build reliable ingestion, transformation, integration, and data-quality processes across structured and unstructured data.
• Work with large-scale datasets across cloud data platforms, APIs, files, databases, and enterprise applications.
• Apply sound practices around data quality, lineage, observability, metadata, security, and governance.
• Optimize pipelines and workloads for reliability, performance, scalability, and cost.
• Develop, evaluate, deploy, and monitor machine learning models using Python and appropriate ML frameworks.
• Understand core ML techniques including supervised and unsupervised learning, classification, regression, clustering, feature engineering, model evaluation, and model monitoring.
• Translate business problems into analytical or machine-learning approaches and determine when traditional analytics, ML, or Generative AI is most appropriate.
• Work with data scientists, business SMEs, and engineering teams to productionize models and analytical solutions.
• Design and implement enterprise LLM applications, including Retrieval-Augmented Generation (RAG), document processing, summarization, extraction, classification, conversational assistants, and agentic workflows.
• Develop solutions using LLM APIs and frameworks, with experience in areas such as prompt engineering, embeddings, vector search, chunking, grounding, tool/function calling, and structured outputs.
• Understand evaluation techniques for LLM solutions, including accuracy, relevance, hallucination risk, latency, cost, security, and responsible AI considerations.
• Integrate structured enterprise data and unstructured content into secure and governed AI solutions.
• Build reusable AI components and APIs that can be incorporated into broader enterprise applications.

AI-Assisted Software Engineering

• Effectively use modern AI development tools such as GitHub Copilot, Claude, and similar coding assistants to accelerate engineering delivery.
• Use AI tools to help create, refactor, debug, test, document, review, and maintain code.
• Apply engineering judgment when using AI-generated code, including validating functionality, security, performance, maintainability, and compliance with coding standards.
• Demonstrate an AI-first engineering mindset, identifying opportunities to automate repetitive development, analysis, testing, and documentation activities rather than simply using AI as an autocomplete capability.
• Build and deploy solutions using the Microsoft Azure technology stack, with relevant experience across services such as Azure Databricks, Azure Storage/Data Lake, Microsoft Entra ID, Key Vault, Azure Functions, APIs, Azure AI services, and related cloud services.
• Work within modern CI/CD and DevOps practices using tools such as Azure DevOps and/or GitHub.
• Use Git-based source control, branching, pull requests, automated testing, build/release pipelines, infrastructure configuration, and environment management.
• Design solutions with appropriate consideration for security, identity, secrets management, logging, monitoring, resilience, scalability, and production support.
• Partner with architecture, cybersecurity, data governance, infrastructure, and application teams to ensure solutions meet enterprise technology standards.

Insurance Domain Knowledge

• Policy, customer, broker, submission, quote, bind, premium, exposure, risk, claims, loss, and underwriting data.
• Commercial and/or specialty insurance processes.
• Underwriting and pricing workflows.
• Claims analytics and automation.
• Customer and broker analytics.
• Submission ingestion, document extraction, triage, and risk assessment.
• Insurance data quality, lineage, privacy, regulatory, and governance considerations.

The engineer should be able to engage with underwriters, actuaries, claims professionals, operations teams, data analysts, and technology teams and translate business needs into appropriate data and AI solutions.

Required Skills & Experience

• Strong professional experience in data engineering, analytics, machine learning, AI, or software engineering.
• Advanced programming skills in Python and SQL.
• Strong experience with Databricks, Spark/PySpark, and Delta Lake.
• Experience designing and supporting production-scale ETL/ELT and data pipelines.
• Practical experience performing data analysis on large and complex datasets.
• Working knowledge of statistical analysis and machine-learning techniques.
• Hands-on experience developing solutions using LLMs and Generative AI.
• Experience with RAG, embeddings, vector search, prompt engineering, evaluation, or related LLM patterns.
• Experience with the Microsoft Azure cloud ecosystem.
• Strong understanding of Git, CI/CD, DevOps, testing, code reviews, and production deployment practices.
• Experience using AI coding assistants such as GitHub Copilot, Claude, or comparable tools as part of day‑to‑day software engineering.
• Ability to work independently from an ambiguous business problem through analysis, technical design, implementation, testing, deployment, and production support.

Preferred Qualifications

• Prior experience in P&C, commercial, specialty, or reinsurance environments.
• Experience with Azure Databricks and Azure AI services.
• Experience operationalizing ML models using MLflow/MLOps practices.
• Experience with vector databases/vector search and enterprise RAG architectures.
• Experience working with insurance documents and unstructured datasets.
• Experience building APIs and microservices supporting data and AI applications.
• Familiarity with data governance, data cataloging, lineage, model governance, and Responsible AI practices.
• Experience with Infrastructure as Code and automated cloud deployments.

Bachelor's or Master's degree in Computer Science, Data Science, Artificial Intelligence, Engineering, Mathematics, or a related discipline, or equivalent professional experience