Lead Data & ML Ops Architect — Remote

ICA.ai; Arlington (VA); Remote; USD 160,000 - 240,000; Full time

Benefits offered by this job

Health Insurance

Dental Insurance

Vision insurance

401(k) plan with company match

Paid Time Off

Education and Professional Development

Remote work from anywhere within the U

Job summary

ICA, Inc. is seeking a Principal Data & ML Ops Architect to lead the architecture of large-scale data and document-intelligence platforms. You will design ingestion, parsing, and retrieval pipelines to support analytics, AI/ML, and production deployments while enforcing governance and security patterns.

The role requires hands-on involvement with Python, SQL, schemas, APIs, and production data pipelines, with collaboration across Data Engineering, Data Science, DevOps/MLOps, and client teams.

Qualifications

• Deep experience architecting and owning production data platforms or distributed data systems.
• Strong Python, SQL, data modeling, ETL/ELT, workflow orchestration, and cloud-data-platform experience.
• Significant experience with unstructured data or document-processing systems at scale.
• Strong understanding of metadata, schema evolution, lineage, provenance, data quality, observability, retries, replay, backfills, and recovery.
• Experience designing systems with sensitive-data controls, governance, access management, and audit requirements.
• Strong understanding of how modern AI/ML and retrieval systems depend on production data architecture.
• Demonstrated technical judgment, ownership, problem solving, and ability to challenge fragile or overly complex designs.
• Strong technical communication and decision-making skills.
• Must be authorized to work in the United States and have lived in the US for 3 or more consecutive years.
• Must be able and willing to obtain a Public Trust Clearance

Responsibilities

• Architect large-scale data and document-processing platforms, from ingestion through downstream consumption.
• Design robust patterns for document parsing, extraction, normalization, validation, storage, search, human review, and reprocessing.
• Establish standards for schemas, metadata, lineage, provenance, data quality, reproducibility, and auditability.
• Design reliable ingestion and transformation pipelines using APIs, files, batch processing, and streaming where appropriate.
• Create platform patterns that enable Data Science and AI prototypes to become secure, maintainable production solutions.
• Architect data and retrieval foundations supporting RAG, analytics, AI workflows, evaluation, and feedback loops.
• Make architecture decisions involving scalability, reliability, security, privacy, cost, and operational complexity.
• Lead technical design reviews, investigate production failures, mentor engineers, and collaborate across teams.
• Remain technically hands-on to review and develop Python, SQL, schemas, APIs, and production data pipelines.

Skills

Python

SQL

Data Modeling

ETL/ELT

Workflow Orchestration

Cloud Platform

Unstructured Data

Metadata

Data Governance

Observability

Public Trust Clearance

Technical Leadership

Tools

Python

Job description

ICA, Inc. is seeking a Principal Data & ML Ops Architect to lead the architecture of large-scale data and document-intelligence platforms. You will design ingestion, parsing, and retrieval pipelines to support analytics, AI/ML, and production deployments while enforcing governance and security patterns.

The role requires hands-on involvement with Python, SQL, schemas, APIs, and production data pipelines, with collaboration across Data Engineering, Data Science, DevOps/MLOps, and client teams.