Senior ML Data Infrastructure Engineer

Cognizant; United States; On-site; USD 140,000 - 190,000; Full time

Job summary

Cognizant is seeking an experienced software engineer to build an ML data infrastructure platform. You will design scalable data processing pipelines, feature stores, and dataset versioning to power model development and evaluation.

In this role you will collaborate with data scientists, data engineers, and security teams to ensure data quality, reproducibility, and compliant data access patterns across our data lakes and warehouses on GCP.

Qualifications

• 7+ years of software engineering experience, with 3+ years in data infrastructure.
• Strong expertise in GCP data and ML infrastructure (BigQuery, Dataflow, Cloud Storage, Cloud Composer, Dataproc).
• Proficiency in Python and SQL.
• Experience with data quality and testing frameworks.
• Experience with data governance, security, and IAM best practices.

Responsibilities

• Design and implement scalable data processing pipelines for ML training and validation
• Build and maintain feature stores with support for both batch and real-time features
• Develop data quality monitoring, validation, and testing frameworks
• Create systems for dataset versioning, lineage tracking, and reproducibility
• Implement automated data documentation and discovery tools
• Design efficient data storage and access patterns for ML workloads
• Partner with data scientists to optimize data preparation workflows

Skills

Python

SQL

Data pipelines

Data governance

Cloud data platforms

Spark / Beam / Flink

Machine learning data infrastructure

Tools

BigQuery

Dataflow

Cloud Storage

Cloud Composer

Dataproc

Spark

Beam

Flink

Feast

Tecton

Airflow

Dagster

Kafka

Kinesis

Job description

**No Visa transfer / c2c/ sponsorship available for this position**

Role Overview:

We’re seeking an experienced engineer to build our ML data infrastructure platform. You’ll create the systems and tools that enable efficient data preparation, feature engineering, and dataset management for machine learning. This role focuses on the data foundation that powers our ML capabilities.

Key Responsibilities:

• Design and implement scalable data processing pipelines for ML training and validation
• Build and maintain feature stores with support for both batch and real-time features
• Develop data quality monitoring, validation, and testing frameworks
• Create systems for dataset versioning, lineage tracking, and reproducibility
• Implement automated data documentation and discovery tools
• Design efficient data storage and access patterns for ML workloads
• Partner with data scientists to optimize data preparation workflows

Technical Requirements:

• 7+ years of software engineering experience, with 3+ years in data infrastructure
• Strong expertise in GCP's data and ML infrastructure:
• BigQuery for data warehousing
• Dataflow for data processing
• Cloud Storage for data lakes
• Cloud Composer (managed Airflow)
• Dataproc for Spark workloads
• Deep expertise in data processing frameworks (Spark, Beam, Flink)
• Experience with feature stores (Feast, Tecton) and data versioning tools
• Proficiency in Python and SQL
• Experience with data quality and testing frameworks
Knowledge of data pipeline orchestration (Airflow, Dagster)

Nice to Have:

• Experience with streaming systems (Kafka, Kinesis)
• Experience with GCP-specific security and IAM best practices
• Knowledge of Cloud Logging and Cloud Monitoring for data pipelines
• Familiarity with Cloud Build and Cloud Deploy for CI/CD
• Experience with streaming systems (Pub/Sub, Dataflow)
• Knowledge of ML metadata management systems
• Familiarity with data governance and security requirements
• Experience with dbt or similar data transformation tools