We are looking for a Lead Data Engineer to design, build, and deliver scalable enterprise data and AI-ready solutions. The role requires strong hands‑on expertise in Snowflake, Nexla, Apache Airflow, SQL, and Python, along with practical experience using GenAI‑powered development tools.
You will collaborate with Data Engineering, Product Engineering, AI/ML, Architecture, and business teams to build data foundations supporting analytics, ML, LLMs, RAG, and Agentic AI initiatives while mentoring engineers and driving Agile delivery.
Key Responsibilities
- Lead design, development, optimization, and support of enterprise data platforms and pipelines.
- Build scalable ETL/ELT solutions using Snowflake, Nexla, SQL, and Python.
- Develop and manage Apache Airflow DAGs, workflows, schedules, and monitoring.
- Design AI-ready data architectures supporting ML, GenAI, RAG, vector databases, and Agentic AI.
- Use Microsoft Copilot, Claude, Cursor, GitHub Copilot, and similar tools to improve engineering productivity.
- Partner with AI/ML teams on data ingestion, transformation, governance, and model‑serving requirements.
- Implement data quality, lineage, observability, governance, security, and compliance practices.
- Lead code reviews, architecture discussions, troubleshooting, and root‑cause analysis.
- Support Agile sprint planning, prioritization, estimation, dependency management, and delivery.
- Coordinate distributed onsite/offshore teams and mentor engineers.
- Create technical designs, data flows, architecture documentation, and operational runbooks.
Required Skills
- Bachelor's degree in Computer Science, IT, Engineering, Data Science, or related field.
- Strong hands‑on experience with Snowflake, including SQL, data modeling, stored procedures, and performance tuning.
- Experience with Nexla or similar ETL/ELT platforms.
- Strong experience with Apache Airflow and workflow orchestration.
- Advanced SQL and Python skills.
- Practical experience with AI‑assisted development tools such as Copilot, Claude, Cursor, or similar.
- Understanding of LLMs, RAG, vector embeddings, prompt engineering, and Agentic AI.
- Experience building data platforms supporting AI/ML and analytics workloads.
- Knowledge of data governance, security, privacy, lineage, and compliance.
- Strong leadership, communication, problem‑solving, and stakeholder management skills.
- Experience working in Agile and distributed team environments.
Preferred Skills
- Snowflake on Azure and Azure data services.
- LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar AI frameworks.
- RAG, semantic search, vector databases, AI agents, or AI‑enabled data products.
- Experience in insurance, insurtech, healthcare, or financial services.