Solutions Architect (Databricks COE)

Apexon Technology; Pune District; On-site; INR 2,000,000 - 3,000,000; Full time

Benefits offered by this job

Group Health Insurance covering family of 4

Term Insurance and Accident Insurance

Paid Holidays & Earned Leaves

Employee Wellness

Continuous skill-based development opportunities

Job summary

Apexon Technology is seeking a Senior Data Architect specialized in Databricks for their Pune location. This role requires defining the vision for the Databricks Center of Excellence and leading initiatives in data processing, analytics, and AI. Candidates should have a Bachelor’s or Master’s degree in data engineering and at least 15 years of experience, with a focus on Databricks solutions. The position includes a robust perks package, including comprehensive health benefits and career advancement opportunities.

Qualifications

• 15+ years of experience, with 3+ years focused on Databricks-based data solutions.
• Must have strong expertise in cloud ecosystem deployments.
• Databricks certifications are a plus.

Responsibilities

• Lead the vision for the Databricks Center of Excellence.
• Design and implement data architectures leveraging Databricks.
• Drive the adoption of Databricks for large-scale processing.
• Integrate Databricks with cloud platforms and ecosystem tools.
• Implement robust data governance and security policies.

Skills

Spark

Delta Lake

Databricks Workflows

Python

Scala

SQL

Kafka

Event Hub

Education

Bachelor's or master's degree in data engineering or Computer Science

Tools

AWS

Azure

GCP

Snowflake

Tableau

Power BI

Job description

Senior Data Architect (Databricks)

Responsibilities

• Strategic Leadership: Define and lead the vision for the Databricks Center of Excellence (CoE), positioning Databricks as a strategic data and AI platform for Apexon and its clients.
• Strategic Leadership: Drive the adoption of Databricks for large-scale data processing, analytics, and AI/ML workloads.
• Architecture and Implementation: Design and implement data architectures leveraging Databricks, Delta Lake, and Spark for big data processing and analytics.
• Architecture and Implementation: Develop frameworks for ETL pipelines, real-time streaming, and batch processing using Databricks.
• Architecture and Implementation: Optimize the use of Databricks features, such as MLFlow, AutoML, and Delta Lake for enterprise solutions.
• Performance and Optimization: Lead efforts to optimize Spark jobs, cluster configurations, and data storage for performance and cost efficiency.
• Performance and Optimization: Ensure scalability of Databricks solutions to handle increasing data volumes and compute requirements.
• Integration with Cloud and Ecosystem Tools: Integrate Databricks with cloud platforms (AWS, Azure, GCP) and tools such as Snowflake, Tableau, or Power BI.
• Integration with Cloud and Ecosystem Tools: Implement CI/CD pipelines for Databricks workflows and model deployment.
• Governance and Security: Implement robust data governance, access control, and security policies in Databricks environments.
• Governance and Security: Ensure compliance with industry regulations and best practices for data privacy.
• Thought Leadership and Team Development: Build and manage a team of Databricks professionals, fostering innovation and technical excellence.
• Thought Leadership and Team Development: Stay updated on Databricks advancements and advocate for their adoption through client presentations, workshops, and internal knowledge-sharing.
• Core Expertise: Proficiency in Spark, Delta Lake, and Databricks Workflows.
• Programming: Advanced skills in Python, Scala, and SQL for data engineering tasks.
• Real-Time Processing: Experience with real-time data integration using Kafka, Event Hub, or Databricks Structured Streaming.
• Cloud Ecosystem: Expertise in deploying Databricks on AWS, Azure, or GCP.

Qualifications

• Must Have: Bachelor’s or master’s degree in data engineering, Computer Science, or related field.
• Must Have: 15+ years of experience, with 3+ years focused on Databricks-based data solutions.
• Nice to Have/Preferred: Databricks certifications such as Databricks Certified Data Engineer Professional.
• Nice to Have/Preferred: Experience with multi-cloud and hybrid Databricks deployments.

Our Commitment to Diversity & Inclusion

• Apexon is an equal opportunity employer and promotes diversity in the workplace. We prohibit discrimination and harassment of any kind and provide equal employment opportunities to employees and applicants without regard to gender, race, color, ethnicity or national origin, age, disability, religion, sexual orientation, gender identity or expression, veteran status, or any other protected characteristic by law.

Our Commitment to Environment

• Actively contribute to Apexon's commitment to environmental responsibility by following sustainable practices and supporting ESG initiatives.

Our Perks and Benefits

• Our benefits and rewards program is designed to recognize skills and contributions, elevate learning and provide care and support for you and your loved ones. Includes continuous skill-based development, opportunities for career advancement, and comprehensive health and well-being benefits.
• Group Health Insurance covering family of 4
• Term Insurance and Accident Insurance
• Paid Holidays & Earned Leaves
• Employee Wellness

Job Location

• Pune, India