Principal Data Engineer- Databricks & AI platform

GoTo; Bengaluru; On-site; INR 4,000,000 - 12,000,000; Full time

Benefits offered by this job

Comprehensive benefits

Paid time off and holidays

Learning and development programs

Job summary

GoTo is seeking a Principal Data Engineer in Bengaluru to own the Databricks data platform, including Unity Catalog, Delta Lake, and Spark. The role is hands-on and architectural, with responsibility to define multi-year roadmaps, standards, and platform improvements that enable AI-enabled data products.

You will mentor engineers, steer migrations of legacy systems to Databricks Lakehouse, and collaborate across data, analytics, and product teams to deliver scalable data solutions.

Qualifications

• 12+ years in data engineering
• Hands-on Databricks in production
• Experience designing medallion/lakehouse architectures
• Strong SQL and Python
• Experience with CI/CD for data pipelines
• Experience leading technical decisions and mentoring engineers
• Experience with AI agents or AI-assisted tooling

Responsibilities

• Own architecture and roadmap for the Databricks platform (Unity Catalog, Delta Lake)
• Lead data pipelines and AI-ready datasets for GenAI/ML use cases
• Design medallion lakehouse architecture at scale
• Set data quality, security, and access standards
• Tune Spark jobs and cluster configs for performance and cost
• Mentor senior engineers and drive incident response
• Embed IaC and CI/CD into pipelines and platform infra

Skills

SQL

Python

Databricks

Lakehouse design

Cloud platforms

Mentoring

Stakeholder comms

Tools

Databricks

Airflow

dbt

CI/CD

Job description

Engineering at GoTo

We’re trailblazers in remote work technology—building powerful, flexible solutions that empower everyone to live their best life, both at work and beyond. With us, you’ll have the opportunity to chart new paths and help redefine how the world works. For us, AI isn’t just a buzzword; it’s a tool we use to deliver real, practical value to our customers and teams. We focus on solving meaningful problems, not just adding features for the sake of using AI. Here, growth takes many forms: you can expand your skills, take on new challenges, lead initiatives, and explore creative ideas. Join a GoTo product team and play a key role in transforming the workplace for millions of users worldwide—your work will truly make a difference.

Where you’ll work

Bangalore, KA, IN

About The Role

We're looking for a Principal Data Engineer to own our Databricks data platform. You'll be the go-to expert on Unity Catalog, Delta Lake, and Spark, and you'll set the technical direction for how our teams build, govern, and scale data pipelines. This is a hands‑on role: you'll write code, review designs, and fix hard problems.

We're looking for a Principal Data Engineer who operates at architect level across our data platform and ML/AI infrastructure. You'll own the technical vision and multi‑year roadmap for our Databricks Lakehouse, set the architectural standards that govern how Data Engineering teams build and ship, and be an active contributor to our ML and AI platform.

You won't be maintaining someone else's architecture. You'll be designing the next version of it, and you'll have the scope to make it matter.

Your Day to Day

• Own the architecture and roadmap for our Databricks platform: Unity Catalog, Delta Lake, cluster configuration, and workflow orchestration.
• Lead the architecture of our data platform, pipelines, and AI‑ready datasets, so they're built to support GenAI and ML use cases, not just BI.
• Design and evolve our medallion (Bronze/Silver/Gold) lakehouse architecture so it scales with the business.
• Set standards for data quality, security, and access control across the platform, and make sure teams actually follow them.
• Tune Spark jobs and cluster configs for performance and cost. Find the waste, right‑size the clusters, kill the idle spend.
• Build and maintain the core ETL frameworks and reusable components other teams build on top of.
• Lead migrations from legacy platforms onto Databricks, including data modeling and pipeline redesign.
• Embed infrastructure‑as‑code and CI/CD practices into how we build, deploy, and run pipelines and platform infrastructure, so deployments stay consistent and repeatable.
• Troubleshoot production issues, do root cause analysis, and put fixes in place that prevent repeat incidents.
• Mentor senior engineers and act as the technical escalation point across the Data teams.
• You’ll set the standard for spec‑driven development and agentic AI tooling, then drive adoption for pipeline generation, testing, documentation, and incident response across the Data teams.
• Stay current on Databricks releases and bring in new capabilities when they solve a real problem, not just because they’re new.

What We’re Looking For

• 12+ years in data engineering, with at least 3+ years working hands‑on with Databricks in production.
• Deep expertise in Unity Catalog, Delta Lake, and Spark performance tuning at scale.
• Real experience designing and running medallion/lakehouse architectures.
• Strong SQL and Python.
• Experience with at least one major cloud platform (AWS, Azure, or GCP) and infrastructure‑as‑code tooling.
• Experience with CI/CD for data pipelines, including Databricks Asset Bundles.
• A track record of leading technical decisions and mentoring other engineers.
• Experience rolling out AI agents or AI‑assisted tooling across an engineering team. You've set standards and gotten a team to adopt them.
• Clear communicator who can work across engineering, analytics, and business stakeholders.

Nice to Have

• Experience with orchestration tools like Airflow or dbt.
• Experience with streaming/messaging systems like Kafka.
• Exposure to GenAI, RAG, or LLM‑based data applications.
• Experience leading a legacy data warehouse to Databricks migration.

What We Offer

At GoTo, we care about helping our people succeed at work and feel supported in life. Our employee benefits and programs are designed to support your well‑being, growth, and sense of belonging. Here’s what you can expect as part of our team:

• Comprehensive health benefits
• Generous paid time off, including paid holidays, volunteer days, quarterly self‑care days, and company‑designated no‑meeting days
• Tuition reimbursement and access to instructor‑led and on‑demand learning and development programs
• The Thrive Global Wellness Program, a confidential Employee Assistance Program (EAP), a wellness app and one‑on‑one wellness coaching
• Employee‑led communities and programs, including Employee Resource Groups (ERGs), GoTo Gives, and charitable matching

We work hard to create an environment where everyone feels welcome, respected, and able to contribute. Building a culture of belonging isn’t just something we talk about - it’s part of how we work every day.

Specific benefits and offerings may vary by country in line with local regulations and market practices.

At GoTo, you’ll find the flexibility, resources, and support you need to thrive—at work, at home, and everywhere in between. You’ll work towards a shared goal with an open‑minded, cohesive team that’s greater than the sum of its parts. We’re committed to creating an inclusive space for everyone, because we know unique perspectives make us a stronger company and community. Join us and be part of a company that invests in your future, where together we Be Real, Think Big, Move Fast, Keep Growing, and stay Customer Obsessed. Learn more.