Lead Data Engineer

Searce Inc; Pune District; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

Searce Inc is seeking a Senior Data Architect to design and deliver cloud-native data platforms in a hands‑on leadership role. Lead end‑to‑end data pipelines and data models, guiding analysts and engineers to produce production‑grade solutions.

The ideal candidate will drive AI‑ready data foundations, collaborate with clients, and ensure scalable, cost‑efficient architectures using AWS, GCP, and related technologies. Pune‑based role with high impact.

Qualifications

• 7–10 years of professional experience in end‑to‑end data product development.
• Cloud‑Native fluency with hands‑on experience on AWS, GCP, or Azure.
• AI‑Native workflow experience and using AI coding assistants to accelerate delivery.
• Evidence of leading 2–3 large‑scale transformations such as data lakehouse or real‑time analytics.
• Direct client‑facing experience translating business needs into technical data solutions.

Responsibilities

• Lead technical design and data architecture for scalable, cloud‑native platforms.
• Build high‑velocity data pipelines using Python, SQL, Spark across diverse data sources.
• Mentor and elevate the squad with code reviews and best practices (CI/CD for data).
• Drive AI‑ready data strategy using Dataflow, BigQuery, Lambda, Glue, EMR as applicable.
• Act as a technical DRI for client success and ensure accurate, consistent data delivery.

Skills

Data architecture

Team leadership

Tools

Python

SQL

Spark

BigQuery

Snowflake

Redshift

AWS

GCP

Job description

Solver? Absolutely. But not the usual kind. We're searching for the architects of the audacious & the pioneers of the possible. If you're the type to dismantle assumptions, re-engineer ‘best practices,’ and build solutions that make the future possible NOW, then you're speaking our language.

Your Responsibilities

What you will wake up to solve.

• Lead Technical Design & Data Architecture: Architect and lead the end‑to‑end development of scalable, cloud‑native data platforms. You’ll guide the squad on critical architectural decisions—choosing between Batch vs. Streaming or ETL vs. ELT—while remaining 100% hands‑on, contributing high‑quality, production‑grade code.
• Build High‑Velocity Data Pipelines: Drive the implementation of robust data transports and ingestion frameworks using Python, SQL, and Spark. You will build integration layers that connect heterogeneous sources (SaaS, RDBMS, NoSQL) into unified, high‑availability environments like BigQuery, Snowflake, or Redshift.
• Mentor & Elevate the Squad: Foster a culture of technical excellence by mentoring and inspiring a team of data analysts and engineers. Lead deep‑diving code reviews, promote best‑practice data modeling (Star/Snowflake schema), and ensure the squad adopts modern engineering standards like CI/CD for data.
• Drive AI‑Ready Data Strategy: Be the expert in designing data foundations optimized for AI and Machine Learning. You will champion the use of GCP (Dataflow, Pub/Sub, BigQuery) and AWS (Lambda, Glue, EMR) to create “clean room” environments that fuel advanced analytics and generative AI models.
• Partner with Clients as a Technical DRI: Act as the Directly Responsible Individual for client success. Translate ambiguous business questions into elegant data services, manage project deliverables using Agile methodologies, and ensure that the data provided is accurate, consistent, and mission‑critical.
• Troubleshoot & Optimize for Scale: Own the reliability of the reporting layer. You will proactively monitor pipelines, troubleshoot complex transformation bottlenecks, and propose ways to improve platform performance and cost‑efficiency.
• Innovate and Build Reusable IP: Spearhead the creation of reusable data frameworks, custom operators, and transformation libraries that accelerate future projects and establish Searce’s unique technical advantage in the market.

Functional Skills

• The Data Architect: This persona deconstructs ambiguous business goals into scalable, elegant data blueprints. They don't just move data; they design the foundation—from schema design to partitioning strategies—that allows data scientists and analysts to thrive, foreseeing technical bottlenecks and making pragmatic trade‑offs.
• The Player‑Coach: As a hands‑on leader, this persona leads from the front by writing exemplary, production‑grade SQL and Python while simultaneously mentoring and elevating the skills of the squad. Their success is measured by the team's ability to deliver high‑quality, maintainable code and their growth as engineers.
• The Pragmatic Innovator: This individual balances a passion for modern data tech (like Generative AI and Real‑time Streaming) with a sharp focus on business outcomes. They champion new tools where they add real value but are disciplined enough to choose stable, cost‑effective solutions to meet deadlines and deliver robust products.
• The Client‑Facing Technologist: This persona acts as the crucial technical bridge between the data squad and the client. They build trust by listening actively, explaining complex data concepts (like data latency or idempotency) in simple terms, and demonstrating how engineering decisions align with the client’s strategic goals.
• The Quality Craftsman: This individual possesses an unwavering commitment to data integrity and treats data engineering as a craft. They are the guardian of the reporting layer, advocating for robust testing, data validation frameworks, and clean, modular code to ensure the long‑term reliability of the data platform.

Experience & Relevance

• Engineering Depth: 7‑10 years of professional experience in end‑to‑end data product development. You have a portfolio that proves your ability to build complex, high‑velocity pipelines for both Batch and Streaming workloads.
• Cloud‑Native Fluency: Deep, hands‑on experience designing and deploying scalable data solutions on at least one major cloud platform (AWS, GCP, or Azure). You are comfortable navigating the nuances of EMR, BigQuery, or Synapse at scale.
• AI‑Native Workflow: You don’t just build for AI; you build AI to accelerate delivery and have a track record of building the data foundations required for Generative AI. You must be proficient in using AI coding assistants (e.g., GitHub Copilot) to accelerate your delivery.
• Architectural Portfolio: Evidence of leading 2‑3 large‑scale transformations—including platform migrations, data lakehouse builds, or real‑time analytics architectures.
• Client‑Facing Acumen: You have direct experience in a consultative, client‑facing role. You can confidently translate a CEO’s business vision into a Lead Engineer’s technical specification without losing anything in translation.