Databricks AI Engineer

Perficient; United States; On-site; USD 150,000 - 230,000; Full time

Job summary

Perficient seeks a Databricks AI Engineer to design and deploy scalable AI solutions on the Databricks Lakehouse Platform. You will build data pipelines with Spark, Delta Lake, and Unity Catalog, and develop end-to-end ML workflows using MLflow and Databricks ML.

The role emphasizes Generative AI, LLMs, RAG-based apps, and strong MLOps practices across CI/CD, governance, and monitoring to ensure enterprise-grade performance and cost efficiency.

Qualifications

• 5+ years of experience in AI/ML, Data Engineering, or Software Engineering.
• 3+ years of hands-on experience with Databricks and Apache Spark.
• Strong proficiency in Python, PySpark, and SQL.
• Experience with MLflow, Delta Lake, Unity Catalog, and Databricks Workflows.
• Experience building and deploying Generative AI, LLM, and RAG-based applications.
• Knowledge of cloud platforms such as Azure, AWS, or GCP.
• Understanding of MLOps, model deployment, and AI governance best practices.

Responsibilities

• Design and implement AI/ML and Generative AI solutions using Databricks.
• Develop scalable data pipelines with Apache Spark, Delta Lake, and Unity Catalog.
• Build, train, deploy, and monitor ML models using MLflow and Databricks ML.
• Develop RAG-based applications, AI agents, and LLM-powered solutions.
• Implement MLOps best practices including CI/CD, model governance, and monitoring.
• Collaborate with business, data, and engineering teams to translate requirements into technical solutions.
• Optimize performance, scalability, security, and cost efficiency of AI platforms.

Skills

AI/ML

Python

PySpark

SQL

Cloud computing

MLOps

Generative AI

RAG

LLMs

Tools

Databricks

Apache Spark

MLflow

Delta Lake

Unity Catalog

LangChain

LangGraph

Azure

AWS

GCP

Job description

Perficient is seeking a Databricks AI Engineer to design, develop, and deploy scalable AI, Machine Learning, and Generative AI solutions on the Databricks Lakehouse Platform. This role combines expertise in Databricks, data engineering, MLOps, and modern AI technologies to deliver innovative, enterprise-grade solutions.

Key Responsibilities

• Design and implement AI/ML and Generative AI solutions using Databricks.
• Develop scalable data pipelines with Apache Spark, Delta Lake, and Unity Catalog.
• Build, train, deploy, and monitor ML models using MLflow and Databricks ML.
• Develop RAG-based applications, AI agents, and LLM-powered solutions.
• Implement MLOps best practices including CI/CD, model governance, and monitoring.
• Collaborate with business, data, and engineering teams to translate requirements into technical solutions.
• Optimize performance, scalability, security, and cost efficiency of AI platforms.

Required Qualifications

• 5+ years of experience in AI/ML, Data Engineering, or Software Engineering.
• 3+ years of hands-on experience with Databricks and Apache Spark.
• Strong proficiency in Python, PySpark, and SQL.
• Experience with MLflow, Delta Lake, Unity Catalog, and Databricks Workflows.
• Experience building and deploying Generative AI, LLM, and RAG-based applications.
• Knowledge of cloud platforms such as Azure, AWS, or GCP.
• Understanding of MLOps, model deployment, and AI governance best practices.

Preferred Qualifications

• Databricks certifications (Data Engineer, Machine Learning Engineer, or Generative AI Engineer).
• Experience with LangChain, LangGraph, vector databases, and agentic AI frameworks.
• Experience delivering enterprise-scale AI and analytics solutions.

Key Skills

Databricks, Generative AI, LLMs, RAG, MLflow, Apache Spark, PySpark, Python, SQL, Delta Lake, Unity Catalog, MLOps, Azure, AWS, GCP, AI Agents, LangChain, LangGraph.