Senior ML Data Engineer — Real-Time Features & Pipelines

Talantir; United States; Remote; USD 55,000 - 83,000; Full time

Benefits offered by this job

Health, Dental, Vision Insurance

Job summary

Talantir in the United States is seeking a data engineer specialized in ML feature engineering and pipeline development. You will build and maintain feature pipelines in Databricks, integrate multiple data sources, and support ML model training and deployment with MLflow.

The role requires strong Python and SQL skills, hands-on Spark experience, and the ability to design real-time and batch data flows within medallion-inspired architectures.

Qualifications

• 7 years in data engineering and at least 4 years focusing on ML feature engineering ETL pipeline development and data preparation for ML
• Proven experience managing pipelines on Databricks using Apache Spark with a strong understanding of the medallion architecture
• Familiarity with ML lifecycle management with MLflow experience and advanced skills in SparkPy, PySpark for big data processing
• Proficient in Python for data manipulation and SQL for query optimization
• Experience building pipelines for real-time and batch model serving in production environments and knowledge of CI/CD practices for ETLELT pipeline development
• Expertise in metadata and master data management within technical data catalogues
• Understanding of data security and compliance especially with sensitive data like PII

Responsibilities

• Feature engineering and data integration to support ML models
• Develop data pipelines from diverse sources and create user behavior features
• Design ETL/ELT pipelines following medallion architecture
• Support ML training, calibration, and deployment with MLflow
• Build low-latency pipelines for real-time and batch inference
• Apply CI/CD for ETL/ELT deployment
• Ensure data governance and regulatory compliance
• Collaborate with ML scientists and engineers to align data work with business goals

Skills

Data engineering

ML feature engineering

ETL pipeline development

Python

SQL

Apache Spark

Databricks

Tools

Apache Spark

Databricks

SparkSQL

Job description

Talantir in the United States is seeking a data engineer specialized in ML feature engineering and pipeline development. You will build and maintain feature pipelines in Databricks, integrate multiple data sources, and support ML model training and deployment with MLflow.

The role requires strong Python and SQL skills, hands-on Spark experience, and the ability to design real-time and batch data flows within medallion-inspired architectures.