Senior Data Engineer — AI-Driven Data Pipelines & ML

Vatica Health, Inc.; United States; Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Competitive salary

Bonus potential

401k plan

Growth opportunities

Healthcare benefits

Flexible PTO

Job summary

Vatica Health, Inc. is seeking a Senior Data Engineer to design, build, and operate data pipelines and APIs powering data-driven, AI-enabled products.

You will own data movement from ingestion to consumption, ensuring reliability, security, and availability for teams and products dependent on it. You will partner with product, analytics, data science, and ML teams to translate business needs into scalable production solutions.

Qualifications

• 8–10 years of professional experience in data engineering or related field.
• Strong Python and SQL skills, with experience delivering production data pipelines at scale.
• Hands-on experience with at least one workflow orchestration platform and big data processing frameworks.
• Experience developing and deploying production REST APIs, including integration, security, and automated testing.

Responsibilities

• Design, build, and maintain batch and streaming pipelines for structured and unstructured data from databases, APIs, event streams, and files.
• Develop ETL/ELT processes and optimize performance, cost, scalability, and reliability.
• Build and operate cloud solutions using AWS services such as EC2, ECS, Lambda, API Gateway, and SQS.
• Develop and manage data workflows, including scheduling, dependencies, retries, alerts, and backfills.
• Establish monitoring, logging, and observability to identify issues and support reliable production operations.
• Design, develop, test, and maintain secure REST APIs and Python-based data services using frameworks such as FastAPI or Flask.
• Implement API authentication, authorization, validation, versioning, error handling, rate limiting, and documentation.
• Build data foundations for machine learning and generative AI, including feature pipelines, training and inference datasets, embeddings, and retrieval-based solutions.
• Integrate AI tools and large language model (LLM) APIs into workflows for document processing, enrichment, classification, and summarization.
• Implement data quality checks, validation, lineage, and documentation, applying security, privacy, and compliance standards.
• Lead architecture and code reviews, mentor engineers, and communicate technical recommendations to business and technical stakeholders.

Skills

Python

SQL

Airflow

REST APIs

Cloud platforms

Data modeling

Git CI/CD

Docker

Terraform

AI/ML workflows

Education

Bachelor's or Master's in CS/Engineering

Tools

AWS

FastAPI

Job description

Vatica Health, Inc. is seeking a Senior Data Engineer to design, build, and operate data pipelines and APIs powering data-driven, AI-enabled products.

You will own data movement from ingestion to consumption, ensuring reliability, security, and availability for teams and products dependent on it. You will partner with product, analytics, data science, and ML teams to translate business needs into scalable production solutions.