We are seeking a highly skilled Data Engineer with Agentic AI and Python expertise to design, develop, and deploy scalable data platforms and AI-powered solutions.
The ideal candidate will have strong experience in Data Engineering, Python development, cloud-based data platforms, and modern Generative AI/Agentic AI frameworks.
This role combines Data Engineering, Python development, and Agentic AI implementation, making it ideal for candidates with Databricks, Snowflake, RAG, and LLM-based solution experience.
Key Responsibilities:
- Design, build, and maintain scalable ETL/ELT pipelines using Python, PySpark, Databricks, and Snowflake.
- Develop and optimize data ingestion, transformation, and orchestration workflows.
- Build Agentic AI solutions leveraging frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or LangChain.
- Design and implement Retrieval-Augmented Generation (RAG) architectures, vector search, and semantic retrieval solutions.
- Develop REST APIs and microservices using Python (FastAPI/Flask).
- Work with structured and unstructured data from multiple sources.
- Implement data quality, governance, monitoring, and observability frameworks.
- Collaborate with Data Scientists, Architects, and Business Stakeholders to deliver AI-driven solutions.
- Follow Agile/Scrum methodologies and contribute to code reviews and best practices.
Required Skills:
- 7+ years of experience in Data Engineering and Python development.
- Strong hands-on experience with:
- Python, PySpark, SQL
- Databricks
- Snowflake
- Azure Data Factory (ADF) or similar ETL tools
- Experience building scalable data pipelines and lakehouse architectures.
- Knowledge of Spark, Delta Lake, Airflow, Kafka, and Data Warehousing concepts.
- Experience with Git, CI/CD, and DevOps practices.
Agentic AI & GenAI Requirements:
- Hands-on experience with:
- LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel, or LlamaIndex
- Prompt Engineering
- AI Agents and Multi-Agent Systems
- Understanding of LLMOps, AI Governance, and Responsible AI practices.
Preferred Qualifications:
- Experience with Snowpark, Cortex AI, or Snowflake Cortex Search/Analyst.
- Knowledge of MLOps tools such as MLflow and Azure ML.
- Exposure to conversational AI, copilots, and intelligent automation solutions.
- Relevant cloud or Databricks certifications.
Nice to Have:
- Real-time streaming using Kafka/Event Hubs.
- Experience with Docker and Kubernetes.
- Exposure to enterprise AI transformation initiatives.