AI Data Engineer — Databricks & ML Pipelines (Hybrid)

Lorven Technologies Inc.; Princeton (NJ); Hybrid; USD 130,000 - 200,000; Full time

Job summary

Lorven Technologies Inc. is seeking a senior data engineer to design and implement data pipelines for AI model metadata, training data lineage, and model performance tracking.

You will build data infrastructure on Databricks and Spark for large-scale processing and develop feature engineering pipelines for AI model training and inference. You will also implement model versioning, experiment tracking, and registry integration, while creating automated workflows for AI agent discovery and

Qualifications

• 5-7 years in data engineering with 2+ years in AI/ML workloads.
• Strong Python and ML frameworks experience; Databricks and Spark for ML workflows.

Responsibilities

• Design and implement data pipelines for AI model metadata and training data lineage.
• Build data infrastructure on Databricks and Spark for large-scale processing.
• Develop ML feature engineering pipelines and data preprocessing workflows.
• Implement model versioning, experiment tracking, and model registry integration.
• Create automated workflows for AI agent discovery, classification, and inventory management.
• Design knowledge graphs for AI model relationships and data lineage.
• Build real-time data pipelines for model monitoring and drift detection.
• Ensure data governance, security, and AI ethics in data handling.
• Collaborate with data scientists to optimize feature stores.
• Create documentation for AI data architectures and integration patterns.

Skills

Python

ML frameworks

Databricks

Apache Spark

SQL

Data engineering

Collaboration

Education

Bachelor's or Master's in CS/DS/ML

Tools

Databricks

Apache Spark

Job description

Lorven Technologies Inc. is seeking a senior data engineer to design and implement data pipelines for AI model metadata, training data lineage, and model performance tracking.

You will build data infrastructure on Databricks and Spark for large-scale processing and develop feature engineering pipelines for AI model training and inference. You will also implement model versioning, experiment tracking, and registry integration, while creating automated workflows for AI agent discovery and