Technical Lead (Databricks)

Bridgenext

Pune District

Presencial

INR 3 500 000 - 6 000 000

Tempo integral

14 dias+

Recebe uma resposta deste empregador

O teu currículo e carta de apresentação, adaptados exatamente ao que estão a contratar.

Ultrapassa os filtros ATS

Resumo da oferta

Bridgenext is seeking a Lead Data Engineer specializing in Databricks to design, optimize, and scale our next-generation data platform for an enterprise client. You will lead a team of data engineers, set standards, and ensure high-performance data solutions.

You will architect data pipelines, mentor engineers, and collaborate with product managers, data scientists, and software engineers to deliver robust, scalable data systems.

Qualificações

  • 8+ years of experience in data engineering, with 2–3 years in a lead role.
  • Databricks expertise with Delta Lake, Unity Catalog, and Spark.
  • Proficiency in Python (PySpark) and SQL.

Responsabilidades

  • Lead design and implementation of scalable data pipelines on Databricks.
  • Mentor and guide junior data engineers and conduct code reviews.
  • Collaborate with product managers, data scientists, and software engineers.

Conhecimentos

Databricks
Python PySpark
SQL
Spark Architecture
Git
CI/CD
Airflow
Data Modeling
Cloud Platforms

Formação académica

Bachelor's or Master's in CS/Engineering

Ferramentas

Delta Lake
Unity Catalog
Parquet/ORC/JSON/CSV

Descrição da oferta de emprego

Company Overview

At Bridgenext, we engineer Growth Operating Systems. Most enterprises have spent millions on their revenue stack and still aren't seeing the growth they expected.They have tools that function, but no system that wins. We help growth-hungry companies close that gap by turning fragmented platforms, siloed teams, and disconnected data into one integrated Growth OS. More than a technology company or marketing agency, we're a global digital consultancy with experts in engineering, data, AI, creative and more.

Our teams are made up of experts who believe in engineering impact, starting with putting people at the center of everything we do. Every team member directly shapes our work, culture, and values. Nothing matters more to us than a kind, respectful, fulfilling environment that supports everyone. Our flexible, inclusive culture gives you the autonomy, resources, and opportunities to thrive.

Position Description

As a Lead Data Engineer specializing in Databricks, you will be a key player in designing, developing, and optimizing ourenterprisetransportation client's next-generation data platform. You will lead a team of data engineers, providing technical guidance, mentorship, and ensuring the scalable, and high-performance data solutions.

Key Responsibilities:

  • Technical Leadership:
    • Lead the design, development, and implementation of scalable and reliable data pipelines using Databricks, Spark, and other relevant technologies
    • Define and enforce data engineering best practices, coding standards, and architectural patterns
    • Provide technical guidance and mentorship to junior and mid-level data engineers
    • Conduct code reviews and ensure the quality, performance, and maintainability of data solutions
  • Databricks Expertise:
    • Architect and implement data solutions on the Databricks platform, including Databricks Lakehouse, Delta Lake, and Unity Catalog
    • Optimize Spark workloads for performance and cost efficiency on Databricks
    • Develop and manage Databricks notebooks, jobs, and workflows
    • Proficiently use Databricks features such as Delta Live Tables (DLT), Photon, and SQL Analytics
  • Pipeline Development & Operations:
    • Develop, test, and deploy robust ETL/ELT pipelines for data ingestion, transformation, and loading from various sources (e.g., relational databases, APIs, streaming data)
    • Implement monitoring, alerting, and logging for data pipelines to ensure operational excellence
    • Troubleshoot and resolve complex data-related issues
  • Collaboration & Communication:
    • Work closely with cross-functional teams including product managers, data scientists, and software engineers
    • Communicate complex technical concepts clearly to both technical and non-technical stakeholders
    • Stay updated with industry trends and emerging technologies in data engineering and Databricks

Must Have Skills:

  • Experience:
    • 8+ years of experience in data engineering, with at least 2-3 years in a lead or senior capacity
    • Proven experience designing and building large-scale data platforms
  • Primary Skills - Databricks:
    • Extensive hands-on experience with Databricks platform, including Databricks Workspace, Spark on Databricks, Delta Lake, and Unity Catalog
    • Strong proficiency in optimizing Spark jobs and understanding Spark architecture
    • Experience with Databricks features like Delta Live Tables (DLT), Photon, and Databricks SQL Analytics
  • Programming:
    • Expertise in Python (PySpark) is essential
    • Strong proficiency in SQL
  • Data Warehousing/Lakes:
    • Deep understanding of data warehousing concepts, dimensional modeling, and data lake architectures
    • Experience with various data storage formats (Parquet, ORC, JSON, CSV)
  • Cloud Platforms:
    • Experience with at least one major cloud platform (AWS, Azure, or GCP) and their data-related services (e.g., S3, ADLS, GCS, EC2, Azure VMs, Google Compute Engine)
  • Tools & Technologies:
    • Experience with version control systems (Git)
    • Familiarity with CI/CD pipelines for data solutions
    • Knowledge of workflow orchestration tools (e.g., Apache Airflow, Databricks Workflows)
  • Education:
    • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field

Preferred Skills:

  • Experience with stream processing technologies (e.g., Kafka, Kinesis) is a plus
  • Contributions to open-source data projectsExperience with Scala or Java is a plus
  • Databricks certifications (e.g., Databricks Certified Data Engineer Associate/Professional)
  • Experience with MLOps and integrating data pipelines with machine learning workflows

Professional Skills:

  • Excellent problem-solving and analytical skills
  • Strong leadership, communication, and interpersonal skills
  • Ability to work independently and as part of a team in a fast-paced environment

Bridgenext is an Equal Opportunity Employer

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Technical Lead (Databricks)
Technical Lead (Databricks)

UNAVAILABLE • Pune District

Presencial
INR 3 500 000 - 5 200 000
Lead Data Engineer (Databricks)
Lead Data Engineer (Databricks)

Bridgenext Digital Engineering • Pune District

Presencial
INR 2 500 000 - 4 500 000
Databricks - Data Engineer
Databricks - Data Engineer

Tredence Inc. • Bengaluru

Presencial
INR 1 500 000 - 2 500 000
Senior Databricks Engineer
Senior Databricks Engineer

DATABEAT • Hyderabad

Presencial
INR 1 500 000 - 2 500 000
Sr. Databricks Engineer
Sr. Databricks Engineer

ExlService Holdings, Inc. • Pune District

Híbrido
INR 3 500 000 - 6 500 000
Sr. Data Engineer - Databricks
Sr. Data Engineer - Databricks

Anblicks Inc. • Hyderabad

Presencial
INR 3 000 000 - 5 500 000
Senior Databricks Engineer / Tech Lead
Senior Databricks Engineer / Tech Lead

Innover Digital Inc. • Bengaluru

Presencial
INR 1 800 000 - 3 000 000
Databricks Developer / Data Architect
Databricks Developer / Data Architect

Mployee.me • Pune District

Híbrido
INR 2 500 000 - 4 500 000
Senior Databricks Engineer
Senior Databricks Engineer

Value Vision • Bengaluru

Presencial
INR 4 200 000 - 6 600 000
Databricks Lead
Databricks Lead

Ascendion • Bengaluru

Híbrido
INR 400 000 - 700 000
Hybrid work in Bengaluru
Relocation allowed