SEMANTIC & GENIE ENGINEER (DATABRICKS)

Vrinda International; India; Remote; INR 2.800.000 - 5.000.000; Vollzeit

Zusammenfassung

Vrinda International is seeking a seasoned Semantic & Genie Engineer to join our IT Services team, focusing on data architecture, AI-enabled automation, and robust governance within the Databricks ecosystem.

The role emphasizes bridging advanced data engineering with generative AI capabilities, ensuring high data quality and seamless semantic layer integration across platforms.

Qualifikationen

• 5+ years of professional experience in data engineering.
• Strong Python and SQL expertise.
• Deep experience with Databricks ecosystem incl. Unity Catalog, Delta Lake, Genie.
• Experience building dashboards with Streamlit.
• Knowledge of data governance, profiling, and validation practices.

Aufgaben

• Design Unity Catalog strategies for data access, lineage and governance.
• Architect and maintain Delta Lake structures for high-performance storage and processing.
• Profile data and build validation frameworks to ensure integrity.
• Optimize SQL queries and pipelines for real-time analytics.
• Develop apps using Python, Streamlit, and Databricks Apps to visualize insights.

Kenntnisse

Python

SQL

Databricks

Unity Catalog

Delta Lake

Databricks Genie

Streamlit

Git

MCP

Data governance

Jobbeschreibung

Welcome to Vrinda International's Talent Portal

Discover your next career milestone with Vrinda International, a leading recruitment firm specializing in IT talent acquisition, international education consultancy, and language coaching opportunities. Our portal streamlines your job search with cutting-edge Applicant Tracking System (ATS) technology.

Why Choose Us

• Seamless Candidate Experience: Upload your resume, browse live openings in Java, Spring Boot, microservices, Angular, and more – or explore roles in global education programs (Germany, UK, France).
  
• Advanced Matching: Our AI-driven tools match your skills to top IT, business development, and overseas study consultancy positions.
  
• Easy Application Process: One-click apply, real-time status updates, and integration with LinkedIn for effortless profile imports.
  
• Exclusive Opportunities: Access hidden job markets, visa guidance for international roles, and personalized career coaching.
  

Join thousands of professionals who've advanced their careers through Vrinda International. Create your profile today and let's connect your talent to global opportunities!

Job Description

Summary

We are seeking a highly skilled Semantic & Genie Engineer to join our IT Services team, focusing on advanced data architecture and intelligent automation within the Databricks ecosystem. This pivotal role is designed to leverage cutting-edge technologies to enhance data governance, optimize query performance, and develop sophisticated multi-agent orchestration solutions. The ideal candidate will bridge the gap between complex data engineering and generative AI capabilities, ensuring robust data quality and seamless integration of semantic layers across our platforms.

Responsibilities

• Design and implement Unity Catalog strategies to manage data access, lineage, and governance effectively.
  
• Architect and maintain Delta Lake structures to ensure high-performance data storage and processing.
  
• Conduct comprehensive data profiling and establish rigorous validation frameworks to uphold data integrity.
  
• Optimize complex SQL queries and database performance to support real-time analytics and reporting needs.
  
• Develop and deploy custom applications using Python, Streamlit, and Databricks Apps to visualize insights and automate workflows.
  
• Engineer advanced multi-agent systems and Databricks Genie Agents to drive autonomous data operations and decision-making.
  
• Configure and monitor Databricks Metric Views to track key performance indicators and system health.
  
• Manage version control and collaborative development processes using Git for all codebases and configurations.
  
• Integrate Machine Learning Pipelines (MCP) to enhance the predictive capabilities of data solutions.
  

Requirements

• Minimum of 5 years of professional experience in data engineering or related technical fields.
  
• Proficiency in Python programming and advanced SQL for data manipulation and analysis.
  
• Deep expertise with Databricks ecosystem components, including Unity Catalog, Delta Lake, and Databricks Genie.
  
• Strong background in data quality assurance, profiling techniques, and validation methodologies.
  
• Demonstrated ability to perform query optimization and tune data pipelines for maximum efficiency.
  
• Experience building interactive dashboards and applications using Streamlit and Databricks Apps.
  
• Knowledge of multi-agent orchestration patterns and the implementation of Genie Agents/Spaces.
  
• Familiarity with Git version control systems and collaborative development practices.
  
• Understanding of Machine Learning Pipelines (MCP) and their integration into data workflows.