Senior AI Data Engineer Hybrid (Databricks & ML Lifecycle)

TECHNEPTUNE CONSULTING INC

Princeton (NJ)

Hybrid

USD 150,000 - 190,000

Full time

6 days ago

Be an early applicant

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Benefits offered by this job

Databricks expertise

Job summary

TECHNEPTUNE CONSULTING INC is seeking a Senior AI Data Engineer in a hybrid role located in Princeton, NJ, with NYC collaboration. You will design and optimize AI data pipelines, model metadata tracking, and data lineage to support governance and scalable AI initiatives.

The ideal candidate has 5–7 years in data engineering, at least 2 years in AI/ML workloads, and strong Python plus Databricks/Spark experience.

Qualifications

Bachelor's or Master's in CS/DS/ML or related field.
5-7 years of hands-on experience in data engineering, with at least 2 years focused on AI/ML workloads.
Proficiency in Python and ML frameworks (PyTorch, TensorFlow, scikit-learn).
Strong experience with Databricks, Apache Spark, and distributed computing for ML workflows.
Deep understanding of the machine learning lifecycle, including model training, deployment, and monitoring processes.
Experience with feature engineering, data preprocessing techniques, and ML data pipelines.
Knowledge of vector databases, embeddings, and similarity search for AI applications.
Proficiency in SQL for structured and unstructured data management.
Understanding of data governance, model governance, and AI ethics principles.
Strong analytical and problem-solving capabilities with attention to data quality.
Excellent collaboration skills for working with data scientists, ML engineers, and architects.

Responsibilities

Design and implement data pipelines for AI model metadata and training data lineage.
Build data infrastructure on Databricks leveraging Spark for large-scale distributed dataset processing.
Develop MCP servers and enable AI data distribution via MCP.
Develop feature engineering pipelines and data preprocessing workflows for AI model training and inference.
Implement model versioning, experiment tracking, and model registry integration using MLflow or similar tools.
Create automated workflows for AI agent discovery, classification, and inventory management across the enterprise.
Design and maintain knowledge graph structures for representing AI model relationships, dependencies, and data lineage.
Build real-time data pipelines for AI model monitoring, drift detection, and performance tracking.
Develop data quality frameworks specific to AI training datasets and validation data.
Collaborate with data scientists to optimize data access patterns and feature store implementations.
Implement security and compliance controls for sensitive AI training data and model artifacts.
Create comprehensive documentation for AI data architectures, schemas, and integration patterns.