ML Data Platform Engineer

Basis Research Institute; New York (NY); On-site; Full time

Job summary

A nonprofit AI research organization is seeking a Data Engineer to design and build data pipelines for training and evaluation while ensuring high data quality standards. The ideal candidate will have strong skills in SQL and Python, experience with distributed computing frameworks, and familiarity with cloud data platforms. This full-time position is based in New York City with a competitive salary. The role involves collaboration with research and engineering teams to optimize data infrastructure and maintain data governance.

Qualifications

• Demonstrated achievements in building data pipelines for ML/AI systems.
• Experience with cloud data platforms for batch and real-time processing.
• Knowledge of data governance and compliance policies.

Responsibilities

• Design and build data pipelines for training and evaluation.
• Implement data quality frameworks including validation and monitoring.
• Collaborate with teams to understand data needs and provide solutions.

Skills

SQL

Python for data processing

Distributed computing frameworks (Spark, Dask)

Workflow orchestration tools (Airflow, Dagster, Prefect)

Feature engineering

Data quality and governance

Data modeling principles

Tools

Snowflake

BigQuery

Kafka

Flink

Job description

A nonprofit AI research organization is seeking a Data Engineer to design and build data pipelines for training and evaluation while ensuring high data quality standards. The ideal candidate will have strong skills in SQL and Python, experience with distributed computing frameworks, and familiarity with cloud data platforms. This full-time position is based in New York City with a competitive salary. The role involves collaboration with research and engineering teams to optimize data infrastructure and maintain data governance.