Data Lead

Hitachi, Ltd.; Bengaluru, Hyderabad; Hybrid; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Flexible work schedules

Work from home options

Annual performance bonuses

Family medical insurance

Group insurance and pensions

Job summary

GlobalLogic in Bengaluru is seeking a senior data engineer to architect and build enterprise-scale data pipelines and Lakehouse solutions on AWS and Databricks.

You will design and optimize Spark workflows (PySpark), Delta Lake, and data governance while collaborating with data scientists and product teams to deliver scalable analytics, AI/ML-ready data platforms, and reliable data services.

Qualifications

• Bachelor's degree or equivalent professional experience.
• 7+ years building enterprise-scale data pipelines and distributed data systems.
• Expert Databricks and Lakehouse architectures experience.
• Deep Spark (PySpark) expertise for large-scale processing.
• Hands-on with Delta Lake, Unity Catalog, Databricks Workflows.
• Python and Java proficiency.
• Advanced SQL with relational and non-relational databases.
• AWS cloud platform expertise including S3, Redshift, IAM.
• Experience designing modern data lakes, warehouses, and Lakehouse solutions.
• Expertise with Kafka and AWS Kinesis for streaming.
• Data modeling, warehousing, schema design, lifecycle management.
• dbt and Looker for data transformation and BI.
• Building and exposing APIs for data consumption and integration.
• Git, CI/CD, infra automation, and engineering best practices.
• Data governance, security, and compliance controls.
• Cluster optimization, performance tuning, cost management.

Responsibilities

• Lead data engineers with mentorship and performance management.
• Design and maintain scalable data pipelines from multiple sources.
• Architect and optimize Databricks Lakehouse solutions for analytics and AI/ML.
• Develop large-scale data processing with Databricks, Spark, Delta Lake.
• Implement Medallion Architecture (Bronze-Silver-Gold) for data quality and governance.
• Build data infrastructure to support analytics and AI/ML workloads.
• Implement streaming solutions with Kafka and AWS Kinesis.
• Leverage Databricks features (Auto Loader, Delta Live Tables, Workflows, Unity Catalog, MLflow).
• Establish best practices for workspace administration, clustering, monitoring, cost optimization.
• Apply governance, security, and regulatory controls across platforms.
• Collaborate with data scientists, engineers, and product leaders on data solutions.
• Drive adoption of modern data engineering practices and tooling.
• Troubleshoot complex data issues while maintaining quality and availability.
• Contribute to architectural decisions and roadmaps.
• Mentor engineers across multiple initiatives and teams.

Skills

Databricks Lakehouse

Apache Spark (PySpark)

Delta Lake

Unity Catalog

Databricks Workflows

Python

Java

SQL (SQL Server, PostgreSQL, MySQL, or

MongoDB

AWS (S3, Redshift, IAM)

Kafka

Kinesis

dbt

Looker

APIs for data consumption

Git, CI/CD, infra automation

Data governance, security, compliance

Troubleshooting data issues

Mentoring engineers

Education

Bachelor's degree in Computer Science or related

Tools

Looker

Databricks

Job description

Job Description

Dear Candidate ,Job DescriptionBachelor's degree in Computer Science, Engineering, Data Science, or related field (or equivalent professional experience).7+ years of experience building enterprise-scale data pipelines, platforms, and distributed data systems.Expert-level experience with Databricks and modern Lakehouse architectures.Deep expertise in Apache Spark, preferably PySpark, for large-scale distributed data processing.Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and enterprise Databricks deployments.Proficiency in Python and strong familiarity with Java.Advanced SQL skills and experience with relational and non-relational databases such as SQL Server, PostgreSQL, MySQL, and MongoDB.Strong expertise with cloud platforms, preferably AWS, including services such as S3, Redshift, IAM, and cloud-native data architectures.Experience designing and implementing modern data lakes, data warehouses, and Lakehouse solutions.Expertise with streaming platforms such as Kafka and AWS Kinesis and event-driven architectures.Strong understanding of data modeling, warehousing, schema design, and data lifecycle management.Experience with data transformation tools such as dbt and business intelligence platforms such as Looker.Experience building and exposing APIs for data consumption and integration.Strong understanding of Git, CI/CD pipelines, infrastructure automation, and software engineering best practices.Experience implementing enterprise data governance, security, and compliance controls across modern data platforms.Experience optimizing Databricks clusters, Spark workloads, storage formats, and platform costs.Strong analytical, troubleshooting, and problem-solving skills with a passion for continuous learning and innovation.Excellent communication and collaboration skills across technical and non-technical audiences.Demonstrated experience mentoring engineers, leading technical projects, and influencing architectural decisions.

Requirements

• Bachelor's degree in Computer Science, Engineering, Data Science, or related field (or equivalent professional experience).
• 7+ years of experience building enterprise-scale data pipelines, platforms, and distributed data systems.
• Expert-level experience with Databricks and modern Lakehouse architectures.
• Deep expertise in Apache Spark, preferably PySpark, for large-scale distributed data processing.
• Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and enterprise Databricks deployments.
• Proficiency in Python and strong familiarity with Java.
• Advanced SQL skills and experience with relational and non-relational databases such as SQL Server, PostgreSQL, MySQL, and MongoDB.
• Strong expertise with cloud platforms, preferably AWS, including services such as S3, Redshift, IAM, and cloud-native data architectures.
• Experience designing and implementing modern data lakes, data warehouses, and Lakehouse solutions.
• Expertise with streaming platforms such as Kafka and AWS Kinesis and event-driven architectures.
• Strong understanding of data modeling, warehousing, schema design, and data lifecycle management.
• Experience with data transformation tools such as dbt and business intelligence platforms such as Looker.
• Experience building and exposing APIs for data consumption and integration.
• Strong understanding of Git, CI/CD pipelines, infrastructure automation, and software engineering best practices.
• Experience implementing enterprise data governance, security, and compliance controls across modern data platforms.
• Experience optimizing Databricks clusters, Spark workloads, storage formats, and platform costs.
• Strong analytical, troubleshooting, and problem-solving skills with a passion for continuous learning and innovation.
• Excellent communication and collaboration skills across technical and non-technical audiences.
• Demonstrated experience mentoring engineers, leading technical projects, and influencing architectural decisions.

Job Responsibilities

• Lead a team of data engineers, providing technical direction, mentorship, and performance management.
• Design, develop, and maintain scalable data pipelines to collect, process, and store data from multiple sources.
• Architect, build, and optimize Databricks Lakehouse solutions supporting enterprise analytics, reporting, AI/ML, and data products.
• Develop large-scale distributed data processing solutions using Databricks, Apache Spark (PySpark), and Delta Lake.
• Implement and manage Medallion Architecture (Bronze, Silver, Gold) data layers to improve data quality, governance, and usability.
• Build and optimize data infrastructure to support analytics, reporting, and AI/ML workloads at scale.
• Implement event sourcing and streaming architectures using platforms such as Kafka and AWS Kinesis for real-time data processing.
• Leverage Databricks capabilities such as Auto Loader, Delta Live Tables, Databricks Workflows, Unity Catalog, and MLflow to improve platform efficiency and governance.
• Establish best practices for Databricks workspace administration, cluster management, performance tuning, monitoring, and cost optimization.
• Apply data governance, security principles, and compliance frameworks to ensure data quality and regulatory adherence.
• Collaborate with data scientists, software engineers, analytics teams, and product leaders to deliver scalable and reliable data solutions.
• Drive the adoption of modern data engineering practices, frameworks, and tooling across the organization.
• Troubleshoot and resolve complex data-related issues while maintaining data quality, availability, and integrity.
• Contribute to architectural decisions and technology roadmaps for the organization's data platform.
• Mentor data engineers and provide technical leadership across multiple initiatives and teams.

What we offer

Exciting Projects: We focus on industries like High-Tech, communication, media, healthcare, retail and telecom. Our customer list is full of fantastic global brands and leaders who love what we build for them.

Collaborative Environment: You Can expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment — or even abroad in one of our global centers or client facilities!

Work-Life Balance: GlobalLogic prioritizes work-life balance, which is why we offer flexible work schedules, opportunities to work from home, and paid time off and holidays.

Professional Development: Our dedicated Learning & Development team regularly organizes Communication skills training(GL Vantage, Toast Master),Stress Management program, professional certifications, and technical and soft skill trainings.

Excellent Benefits:

• competitive salaries
• family medical insurance
• Group Term Life Insurance
• Group Personal Accident Insurance
• NPS(National Pension Scheme )
• Periodic health awareness program
• extended maternity leave
• annual performance bonuses
• referral bonuses

Fun Perks: We want you to love where you work, which is why we host sports events, cultural activities, offer food on subsidies rates, Corporate parties. Our vibrant offices also include dedicated GL Zones, rooftop decks and GL Club where you can drink coffee or tea with your colleagues over a game of table and offer discounts for popular stores and restaurants!

About GlobalLogic

GlobalLogic is a full-lifecycle product development services leader that combines chip-to-cloud software engineering expertise and vertical industry experience to help our customers design, build, and deliver their next-generation products and digital experiences. We expertly integrate design, complex engineering, and agile delivery capabilities to deliver superior business outcomes for global brands in telecom, automotive, healthcare, technology, media and entertainment, manufacturing, and semiconductor industries.

Headquartered in Silicon Valley, GlobalLogic employs over 25,000 designers and engineers across the globe. Analysts like NASSCOM and Zinnov have recognized us for being a top company in our field, and we are consistently nominated as a preferred employer by both global HR consultative firms and local job boards. By creating a work environment that is exciting and flexible, and by fostering growth through ongoing learning and development programs, we empower our employees to achieve both their professional and personal goals.

At GlobalLogic, we make amazing products — and careers.