AI Engineer

Talentd; Pune District; Presencial; INR 1.200.000 - 2.400.000; Jornada completa

Descripción de la vacante

TalentD is seeking an AI/ML Engineer to design, develop, and deploy ML models and AI workflows within the Databricks environment. You will build scalable Delta Lake pipelines using PySpark and Python to process large datasets for AI applications.

You will implement MLOps using MLflow, optimize PySpark jobs and Databricks clusters for performance and cost, and collaborate with Data Scientists, Data Engineers, and DevOps to integrate AI models into core business applications while ensuring

Formación

• Experience designing and deploying ML models and AI workflows.
• Proficiency in building scalable data pipelines with PySpark and Python.
• Hands-on experience with model lifecycle management tools (MLflow).
• Knowledge of data governance and security concepts (Unity Catalog).

Responsabilidades

• Design, develop, and deploy ML models and AI workflows within Databricks.
• Build scalable Delta Lake pipelines to ingest, clean, and preprocess large-scale datasets.
• Implement MLOps practices for tracking, versioning, deployment, and monitoring of models.
• Optimize PySpark jobs and Databricks clusters for performance and cost efficiency.
• Collaborate with Data Scientists, Data Engineers, and DevOps to integrate AI assets into core apps and APIs.
• Establish governance and security through Unity Catalog and lineage tracking.

Conocimientos

AI/ML expertise

Collaborative work

Herramientas

Databricks

Delta Lake

PySpark

Python

MLflow

Unity Catalog

Descripción del empleo

Job Description

This role covers the following areas:

• AI/ML Development: Design, develop, and deploy machine learning models, LLMs, and AI workflows within the Databricks environment.
• Data Pipeline Engineering: Build scalable, efficient Delta Lake pipelines using PySpark and Python to ingest, clean, and preprocess large-scale datasets for AI applications.
• MLOps Implementation: Set up and manage the ML lifecycle using MLflow for model tracking, versioning, deployment, and monitoring.
• Optimization and Performance: Optimize PySpark jobs, Databricks clusters, and Python code for maximum performance and cost efficiency.
• Collaboration and Integration: Partner with Data Scientists, Data Engineers, and DevOps teams to integrate AI models into core business applications and APIs.
• Governance and Security: Implement Unity Catalog for data governance, access control, and lineage tracking across AI assets.