Senior AI Data Engineer: AI Pipelines & MLOps Lead

Gandiva Insights LLC; New Jersey; Hybrid; USD 140,000 - 190,000; Full time

Job summary

Gandiva Insights LLC is seeking a Senior AI Data Engineer to design and optimize AI data pipelines across model metadata, training data lineage, and performance metrics. You will build scalable Databricks-based infrastructure and implement end-to-end ML workflows including versioning and experiment tracking.

The role requires 10+ years of experience, strong Python and ML framework skills, and a focus on data governance, security, and collaboration with data scientists.

Qualifications

• Bachelor's or Master's degree in CS/DS/ML or related field.
• ],
• job_responsibilities_description
• CoT_job_summary_short
• job_summary_short
• contract_type
• location_type
• remote_scope
• perks
• stated_min
• stated_max
• stated_absolute
• salary_estimation_cot
• estimated_low
• estimated_medium
• estimated_high
• currency
• frequency
• bonus
• tax
• application_email
• job_end_date
• contact_person
• hiring_department_name

Responsibilities

• Design and implement data pipelines for AI model metadata, training data lineage, and model performance metrics.
• Build Databricks/Spark-based data infrastructure for large-scale ML workflows.
• Develop MCP servers and enable AI data distribution via MCP.
• Create feature engineering pipelines and data preprocessing for training and inference.
• Implement model versioning, experiment tracking, and registry integration with MLflow.
• Create automated workflows for AI agent discovery and inventory management.
• Design knowledge graph structures for AI model relationships and data lineage.
• Build real-time pipelines for model monitoring and drift detection.
• Develop data quality frameworks for AI training and validation data.
• Collaborate with data scientists to optimize data access and feature stores.
• Implement security and compliance controls for AI data and artifacts.

Job description

Gandiva Insights LLC is seeking a Senior AI Data Engineer to design and optimize AI data pipelines across model metadata, training data lineage, and performance metrics. You will build scalable Databricks-based infrastructure and implement end-to-end ML workflows including versioning and experiment tracking.

The role requires 10+ years of experience, strong Python and ML framework skills, and a focus on data governance, security, and collaboration with data scientists.