Lead Data Engineer

Searce Inc; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Searce Inc in Bengaluru seeks a senior Data Architect to lead the design and development of scalable, cloud-native data platforms. You will be hands-on, guiding architectural decisions on batch vs streaming, ETL vs ELT, and delivering production-grade code.

You will mentor a squad of data analysts and engineers, implement CI/CD for data, champion AI-ready data foundations using GCP and AWS, and ensure data accuracy, latency management, and cost efficiency for mission-critical analytics.

Qualifications

• Architect scalable, cloud-native data platforms with end-to-end responsibility.
• Lead and mentor a team of data analysts and engineers in delivering high-quality code.
• Design data foundations for AI/ML, leveraging GCP and AWS tools.
• Translate client questions into elegant data services using Agile practices.
• Champion data integrity, testing, and modular, cost-efficient pipelines.

Responsibilities

• Architect end-to-end data platforms and guide critical architectural decisions.
• Lead hands-on development with Python, SQL, Spark and production-grade code.
• Mentor and elevate the squad through code reviews and best practices.
• Drive AI-ready data strategy using GCP (Dataflow, BigQuery) and AWS (Lambda, EMR).
• Ensure reliability of reporting pipelines, optimize performance and costs.

Skills

Data Architecture

Python

SQL

Spark

Cloud-native

CI/CD for data

BigQuery

Snowflake

Redshift

Tools

BigQuery

Snowflake

Redshift

EMR

Job description

Solver? Absolutely. But not the usual kind. We're searching for the architects of the audacious & the pioneers of the possible. If you're the type to dismantle assumptions, re-engineer ‘best practices,’ and build solutions that make the future possible NOW, then you're speaking our language.

Your Responsibilities

What you will wake up to solve.

• Lead Technical Design & Data Architecture: Architect and lead the end‑to‑end development of scalable, cloud‑native data platforms. You’ll guide the squad on critical architectural decisions—choosing between Batch vs. Streaming or ETL vs. ELT—while remaining 100% hands‑on, contributing high‑quality, production‑grade code.
• Build High‑Velocity Data Pipelines: Drive the implementation of robust data transports and ingestion frameworks using Python, SQL, and Spark. You will build integration layers that connect heterogeneous sources (SaaS, RDBMS, NoSQL) into unified, high‑availability environments like BigQuery, Snowflake, or Redshift.
• Mentor & Elevate the Squad: Foster a culture of technical excellence by mentoring and inspiring a team of data analysts and engineers. Lead deep‑diving code reviews, promote best‑practice data modeling (Star/Snowflake schema), and ensure the squad adopts modern engineering standards like CI/CD for data.
• Drive AI‑Ready Data Strategy: Be the expert in designing data foundations optimized for AI and Machine Learning. You will champion the use of GCP (Dataflow, Pub/Sub, BigQuery) and AWS (Lambda, Glue, EMR) to create “clean room” environments that fuel advanced analytics and generative AI models.
• Partner with Clients as a Technical DRI: Act as the Directly Responsible Individual for client success. Translate ambiguous business questions into elegant data services, manage project deliverables using Agile methodologies, and ensure that the data provided is accurate, consistent, and mission‑critical.
• Troubleshoot & Optimize for Scale: Own the reliability of the reporting layer. You will proactively monitor pipelines, troubleshoot complex transformation bottlenecks, and propose ways to improve platform performance and cost‑efficiency.
• Innovate and Build Reusable IP: Spearhead the creation of reusable data frameworks, custom operators, and transformation libraries that accelerate future projects and establish Searce’s unique technical advantage in the market.

Functional Skills

• The Data Architect: This persona deconstructs ambiguous business goals into scalable, elegant data blueprints. They don't just move data; they design the foundation—from schema design to partitioning strategies—that allows data scientists and analysts to thrive, foreseeing technical bottlenecks and making pragmatic trade‑offs.
• The Player‑Coach: As a hands‑on leader, this persona leads from the front by writing exemplary, production‑grade SQL and Python while simultaneously mentoring and elevating the skills of the squad. Their success is measured by the team's ability to deliver high‑quality, maintainable code and their growth as engineers.
• The Pragmatic Innovator: This individual balances a passion for modern data tech (like Generative AI and Real‑time Streaming) with a sharp focus on business outcomes. They champion new tools where they add real value but are disciplined enough to choose stable, cost‑effective solutions to meet deadlines and deliver robust products.
• The Client‑Facing Technologist: This persona acts as the crucial technical bridge between the data squad and the client. They build trust by listening actively, explaining complex data concepts (like data latency or idempotency) in simple terms, and demonstrating how engineering decisions align with the client’s strategic goals.
• The Quality Craftsman: This individual possesses an unwavering commitment to data integrity and treats data engineering as a craft. They are the guardian of the reporting layer, advocating for robust testing, data validation frameworks, and clean, modular code to ensure the long‑term reliability of the data platform.

Experience & Relevance

• Engineering Depth: 7‑10 years of professional experience in end‑to‑end data product development. You have a portfolio that proves your ability to build complex, high‑velocity pipelines for both Batch and Streaming workloads.
• Cloud‑Native Fluency: Deep, hands‑on experience designing and deploying scalable data solutions on at least one major cloud platform (AWS, GCP, or Azure). You are comfortable navigating the nuances of EMR, BigQuery, or Synapse at scale.
• AI‑Native Workflow: You don’t just build for AI; you build AI to accelerate delivery and have a track record of building the data foundations required for Generative AI. You must be proficient in using AI coding assistants (e.g., GitHub Copilot) to accelerate your delivery.
• Architectural Portfolio: Evidence of leading 2‑3 large‑scale transformations—including platform migrations, data lakehouse builds, or real‑time analytics architectures.
• Client‑Facing Acumen: You have direct experience in a consultative, client‑facing role. You can confidently translate a CEO’s business vision into a Lead Engineer’s technical specification without losing anything in translation.