Senior Consultant, AI/ML Engineer

Hollstadt Consulting; Minnesota; On-site; USD 140,000 - 210,000; Full time

Job summary

Hollstadt Consulting in Minnesota is seeking a Senior ML/AI Engineer to design, train, and deploy production ML models and GenAI applications. You will help build and extend the shared infrastructure and tools used across teams.

You will lead the integration of model outputs into decision systems, create explainable and trustworthy guidance, and work across data science, MLOps, and product teams to push prototypes into scalable services.

Qualifications

• 5+ years building and shipping ML/AI systems in production.
• Strong data science fundamentals and solid experimental methodology.
• Hands-on GenAI/LLM engineering experience.
• Excellent Python and AWS experience.

Responsibilities

• Build ML models and evaluate using relevant metrics.
• Integrate model output with business rules and LLM reasoning.
• Ship GenAI applications with RAG pipelines and agents.
• Engineer the AI platform with reusable libraries and gateways.
• Lead technically, review designs, mentor engineers.

Skills

Python

Data science fundamentals

ML model training/evaluation

LLM/GenAI engineering

AWS

DevOps/CI/CD

Tools

Docker

SageMaker

Bedrock

Lambda

S3

OpenTelemetry

Terraform

Git

Job description

The AI CoE builds AI products and the shared platform that powers them. We're looking for a Senior ML / AI Engineer who is equally comfortable building models and building the platform around them: someone

who can train and evaluate an ML model, ship a production LLM/GenAI application, and extend shared AI infrastructure and tooling that the wider team depends on.

This is a senior builder role at the intersection of applied data science, GenAI application engineering, and AI platform engineering. Our work spans predictive modeling, LLM-based systems, and the platform underneath them; you'll take problems from data and prototype through to governed, monitored production services, and set the technical patterns other engineers build on. Projects vary over time — we value engineers who can move across the stack rather than stay in one lane

Performance expectations:

• Build ML models — frame problems, engineer features, train and evaluate models (ranking, scoring, survival/time-to-event, classification, forecasting), and reason rigorously about metrics (AUC, C-index, calibration), validation strategy, subgroup performance, and failure modes.
• Integrate models into decision systems — combine model output with business/domain rules and LLM reasoning to produce explainable, trustworthy recommendations.
• Ship GenAI applications — design and deploy LLM-powered features: RAG pipelines, agents, structured extraction, summarization, decision-reasoning trails, and evaluation harnesses using Claude/Bedrock and other models.
• Engineer the AI platform — extend the shared AI gateway (unified multi-model access, API keys, per-team budgets, failover, observability) and reusable libraries/SDKs that other teams build on.
• Own the RAG/data layer — embeddings, vector stores, retrieval quality, chunking, and grounding strategies; measure and improve retrieval and answer quality.
• Build evaluation & quality tooling — offline/online eval, LLM-as-judge, regression suites, statistical validation, and guardrails so model and prompt changes ship safely.
• Productionize — wrap models and pipelines as tested, observable services (Python, containers, AWS Lambda/SageMaker/EKS), with monitoring for quality, cost, latency, and drift.
• Lead technically — set patterns and standards, review designs and code, mentor engineers, and partner with data scientists, MLOps, clinical/domain experts, and product owners to move prototypes to production.

Required Qualifications

• 5+ years building and shipping ML / AI systems in production (not just notebooks/POCs), including technical leadership of non-trivial projects.
• Strong data science / ML fundamentals — feature engineering, model training and evaluation, metrics (AUC, C-index, calibration, precision/recall), gradient-boosted trees (XGBoost), and sound experimental methodology (validation strategy, subgroup analysis).
• Experience building ranking, scoring, or survival/time-to-event models, and integrating model output into a larger decision system.
• Hands-on GenAI / LLM engineering — RAG, prompt engineering, function/tool calling, embeddings and vector search, and LLM evaluation.
• Excellent Python — production-grade, tested, well-structured code; comfortable building APIs/services and shared libraries.
• AWS experience — Bedrock and/or SageMaker, Lambda, S3, plus containers (Docker) and Git-based workflows.
• Solid software engineering practice: version control, testing, code review, CI/CD; ability to reason about cost, latency, and reliability of AI systems in production.

Preferred Qualifications

• AI platform engineering — building shared gateways/proxies, model routing, multi-tenancy, quota/budget enforcement, or internal AI SDKs.
• Experience with agent frameworks, real-time/voice AI, or streaming inference.
• Vector databases (Qdrant, OpenSearch, pgvector) and retrieval-quality tuning at scale.
• IaC (Terraform), observability (OpenTelemetry/CloudWatch), and FinOps for AI workloads.
• Healthcare / clinical ML — survival analysis, outcome prediction, or working with clinical/scientific datasets alongside domain experts.
• Serving models as endpoints (SageMaker), cross-account inference, and train/serve parity.
• Experience in a regulated / PHI-handling environment (HIPAA) — data governance, PII handling, auditability.