Senior ML Engineer with Python (IR-536)

Intellectsoft; United States; Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Udemy courses

Team-building events

Flexible hours

Work from anywhere

Job summary

Intellectsoft is seeking an experienced ML Engineer to build scalable ML systems, backend APIs with FastAPI, and MLOps pipelines for production LLMs.

You will integrate Azure OpenAI, LangChain, LangGraph, and vector DBs, and implement robust, scalable architectures with DI, testing, and observability in a fast-paced team.

Candidates should have 7+ years Python, 2+ years ML, and strong problem-solving in regulated domains.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or related field.
• 7+ years of Python coding experience.
• 2+ years of hands-on ML and production LLM systems.
• 3+ years building backend APIs with FastAPI, async patterns, rate limiting, SQLAlchemy.

Responsibilities

• Build and refine ML engineering platforms and components.
• Develop scalable backend systems, APIs, and microservices using FastAPI.
• Implement MLOps: KPI measurement, drift detection, and feedback loops.
• Deploy and operationalize ML and Deep Learning models with a focus on LLMs and GenAI.
• Integrate Azure OpenAI and other LLM providers with retry logic and error handling.
• Maintain up-to-date knowledge of state-of-the-art technologies in LLMs and transformer architectures.
• Scale ML algorithms on massive data under strict SLAs.
• Build and orchestrate model pipelines with feature engineering, inference, and retraining.
• Write production-ready, testable code with strong OO and asyncio patterns.
• Implement DI and layered architecture; build LLM observability (Langfuse).
• Develop prompt management systems with versioning and fallbacks.
• Implement Celery workflows for asynchronous tasks and complex pipelines.
• Build multi-tenant architectures with data isolation.
• Cost optimization for LLM usage; integrate third-party APIs.

Skills

Python

ML/LLM

FastAPI

LangChain

LangGraph

Async patterns

SQLAlchemy

DI/ABCs

Vector databases

Hugging Face

MLOps

OCR/Document understanding

DevOps basics

Education

Bachelor's or Master's in CS

Tools

FastAPI

SQL

Pinecone/Weaviate/Chroma

SQLAlchemy

LangChain

LangGraph

MLflow

Langfuse

Hugging Face

LlamaIndex

Job description

Our customer's product is an AI-powered platform that helps businesses make better decisions and work more efficiently. It uses advanced analytics and machine learning to analyze large amounts of data and provide useful insights and predictions. The platform is widely used in various industries, including healthcare, to optimize processes, improve customer experiences, and support innovation. It integrates easily with existing systems, making it easier for teams to make quick, data-driven decisions to deliver cutting-edge solutions.

Requirements

• Bachelor’s or Master’s degree in Computer Science or a related field.
• Strong Python coding skills - 7+ years.
• 2+ years of hands-on experience with machine learning and production LLM systems.
• Experience building backend APIs with FastAPI, async patterns, rate limiting, and SQLAlchemy - 3+ years.
• Experience designing maintainable and extensible systems using dependency injection, interfaces, and abstract base classes.
• Experience with vector databases such as Pinecone, Weaviate, or Chroma, as well as hybrid search.
• Strong understanding of RAG architectures, including retrieval, reranking, context assembly, and response generation.
• Hands-on experience with LangChain and LangGraph for building and orchestrating LLM workflows.
• Advanced Python skills, including async/await, type hints, Pydantic, and SOLID principles.
• MLOps experience with MLflow, model versioning, and A/B testing; experience with Langfuse is a plus.
• Experience in NLP and computer vision, including document understanding, OCR, and GPT-4 Vision.
• Experience building feature pipelines, real-time and batch inference systems, and model serving.
• Hands-on experience with Hugging Face is required; experience with LlamaIndex is a plus.
• Familiarity with database technologies such as SQL.
• Good problem-solving skills and the ability to work in a fast-paced, team-oriented environment.

Nice to have skills:

• Understanding of DevOps, CI / CD including: Docker containerization, Azure DevOps pipelines or GitHub Actions, Kubernetes (nice to have)
• Data security including: Multi-tenant data isolation, Secure key management (Azure Key Vault), Audit trail implementation
• Experience in designing on cloud platform including: Azure (strongly preferred): Azure OpenAI, Blob Storage, Key Vault, Container Registry, AWS or GCP
• Experience in data engineering in Big Data systems including: Large-scale data processing, ETL/ELT pipelines.
• Rate limiting and quota management for high-throughput API usage.
• Cost management and optimization for LLM usage at scale.
• Document processing expertise (PDF extraction, OCR tooling).
• Production incident management and on-call experience.
• Testing strategies for non-deterministic LLM outputs (e.g., golden datasets, fuzzy matching).
• Domain knowledge in regulated industries (e.g., healthcare/pharma workflows, regulatory compliance) is a plus.

Responsibilities:

• Build, refine, and use ML Engineering platforms and components;
• develop and implement scalable backend systems, APIs, and microservices using FastAPI.
• Implement MLOps including model KPI measurement, tracking, model drift detection, and model feedback loops.
• Deploy and operationalize ML and Deep Learning models, with a strong focus on LLMs and Generative AI.
• Integrate Azure OpenAI (GPT-4, GPT-4 Vision) and other LLM providers with proper retry logic and error handling.
• Maintain up-to-date knowledge of state-of-the‑art technologies such as LLMs, GenAI, and transformer architectures.
• Scale machine learning algorithms to work on massive data sets under strict SLAs.
• Build and orchestrate model pipelines including feature engineering, inferencing, and continuous model training.
• Write backend application code in Python and SQL using strong object-oriented principles and asynchronous programming (asyncio, async/await).
• Implement dependency injection patterns and layered architecture (Service, Foundation, Orchestration, DAL).
• Build LLM observability (e.g., Langfuse) to track prompts, tokens, costs, and latency.
• Develop prompt management systems with versioning and fallback mechanisms.
• Implement Celery (or similar) workflows for asynchronous task processing and complex pipelines.
• Build multi-tenant architectures with client data isolation.
• Implement cost optimization strategies for LLM usage (prompt caching, batch processing, token optimization).
• Integrate third‑party APIs and services (e.g., document/OCR services, cloud storage, enterprise systems).
• Collaborate with client-facing teams to understand business context and contribute to technical requirement gathering.
• Write production-ready code that is testable, maintainable, and accounts for edge cases and errors.
• Ensure high quality of deliverables by following architecture/design guidelines, coding best practices, and periodic design/code reviews.
• Write unit tests and higher-level tests to handle expected edge cases and errors gracefully.
• Troubleshoot backend application code using structured logging and distributed tracing.
• Use bug tracking, code review, version control, and other tools to organize and deliver work.
• Participate in scrum calls and agile ceremonies, communicating progress, issues, and dependencies.
• Document application changes and updates, including API documentation via OpenAPI/Swagger.
• Research and evaluate emerging architecture patterns and technologies through rapid learning, proofs-of-concept, and prototypes.

Benefits

Awesome projects with an impact Udemy courses of your choice Team-buildings, events, marathons & charity activities to connect and recharge Workshops, trainings, expert knowledge-sharing that keep you growing Clear career path Absence days for work-life balance Flexible hours & work setup - work from anywhere and organize your day your way