Lead Data Engineer

Searce Inc; Pune District; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Searce Inc is hiring a Data Architect to lead end‑to‑end design of cloud‑native data platforms in a client‑facing role. The ideal candidate will architect scalable data blueprints, guide pipelines end‑to‑end, and mentor a squad of analysts and engineers.

The role emphasizes hands‑on development with Python, SQL, and Spark, and a focus on AI‑ready data foundations across AWS, GCP, and Azure.

Qualifications

• 7–10 years of professional experience in end‑to‑end data product development.
• Cloud‑native data solution design on AWS, GCP, or Azure.
• AI‑native workflow and generative AI data foundations experience.

Responsibilities

• Lead Technical Design & Data Architecture for scalable cloud data platforms.
• Build high‑velocity data pipelines using Python, SQL, and Spark.
• Mentor and elevate the squad with code reviews and best practices.
• Drive AI‑Ready data strategy leveraging GCP and AWS services.
• Partner with clients as the DRI to translate business problems into data services.
• Troubleshoot and optimize pipeline reliability and cost efficiency.
• Spearhead reusable data frameworks and libraries for future projects.

Skills

Python

SQL

Spark

Tools

BigQuery

Snowflake

Redshift

EMR

Dataflow

Pub/Sub

Lambda

Glue

Synapse

Job description

Solver? Absolutely. But not the usual kind. We're searching for the architects of the audacious & the pioneers of the possible. If you're the type to dismantle assumptions, re-engineer ‘best practices,’ and build solutions that make the future possible NOW, then you're speaking our language.

Your Responsibilities

What you will wake up to solve.

• Lead Technical Design & Data Architecture: Architect and lead the end-to-end development of scalable, cloud-native data platforms. You’ll guide the squad on critical architectural decisions—choosing between Batch vs. Streaming or ETL vs. ELT—while remaining 100% hands-on, contributing high‑quality, production‑grade code.
• Build High-Velocity Data Pipelines: Drive the implementation of robust data transports and ingestion frameworks using Python, SQL, and Spark. You will build integration layers that connect heterogeneous sources (SaaS, RDBMS, NoSQL) into unified, high‑availability environments like BigQuery, Snowflake, or Redshift.
• Mentor & Elevate the Squad: Foster a culture of technical excellence by mentoring and inspiring a team of data analysts and engineers. Lead deep‑Dive code reviews, promote best‑practice data modeling (Star/Snowflake schema), and ensure the squad adopts modern engineering standards like CI/CD for data.
• Drive AI‑Ready Data Strategy: Be the expert in designing data foundations optimized for AI and Machine Learning. You will champion the use of GCP (Dataflow, Pub/Sub, BigQuery) and AWS (Lambda, Glue, EMR) to create "clean room" environments that fuel advanced analytics and generative AI models.
• Partner with Clients as a Technical DRI: Act as the Directly Responsible Individual for client success. Translate ambiguous business questions into elegant data services, manage project deliverables using Agile methodologies, and ensure that the data provided is accurate, consistent, and mission‑critical.
• Troubleshoot & Optimize for Scale: Own the reliability of the reporting layer. You will proactively monitor pipelines, troubleshoot complex transformation bottlenecks, and propose ways to improve platform performance and cost‑efficiency.
• Innovate and Build Reusable IP: Spearhead the creation of reusable data frameworks, custom operators, and transformation libraries that accelerate future projects and establish Searce’s unique technical advantage in the market.

Functional Skills

the solver personas.

• The Data Architect: This persona deconstructs ambiguous business goals into scalable, elegant data blueprints. They don't just move data; they design the foundation—from schema design to partitioning strategies—that allows data scientists and analysts to thrive, foreseeing technical bottlenecks and making pragmatic trade‑offs.
• The Player‑Coach: As a hands‑on leader, this persona leads from the front by writing exemplary, production‑grade SQL and Python while simultaneously mentoring and elevating the skills of the squad. Their success is measured by the team's ability to deliver high‑quality, maintainable code and their growth as engineers.
• The Pragmatic Innovator: This individual balances a passion for modern data tech (like Generative AI and Real‑time Streaming) with a sharp focus on business outcomes. They champion new tools where they add real value but are disciplined enough to choose stable, cost‑effective solutions to meet deadlines and deliver robust products.
• The Client‑Facing Technologist: This persona acts as the crucial technical bridge between the data squad and the client. They build trust by listening actively, explaining complex data concepts (like data latency or idempotency) in simple terms, and demonstrating how engineering decisions align with the client’s strategic goals.
• The Quality Craftsman: This individual possesses an unwavering commitment to data integrity and treats data engineering as a craft. They are the guardian of the reporting layer, advocating for robust testing, data validation frameworks, and clean, modular code to ensure the long‑term reliability of the data platform.

Experience & Relevance

• Engineering Depth: 7-10 years of professional experience in end‑to‑end data product development. You have a portfolio that proves your ability to build complex, high‑velocity pipelines for both Batch and Streaming workloads.
• Cloud‑Native Fluency: Deep, hands‑on experience designing and deploying scalable data solutions on at least one major cloud platform (AWS, GCP, or Azure). You are comfortable navigating the nuances of EMR, BigQuery, or Synapse at scale.
• AI‑Native Workflow: You don’t just build for AI; you build with AI. You must be proficient in using AI coding assistants (e.g., GitHub Copilot) to accelerate your delivery and have a track record of building the data foundations required for Generative AI.
• Architectural Portfolio: Evidence of leading 2-3 large‑scale transformations—including platform migrations, data lakehouse builds, or real‑time analytics architectures.
• Client‑Facing Acumen: You have direct experience in a consultative, client‑facing role. You can confidently translate a CEO’s business vision into a Lead Engineer’s technical specification without losing anything in translation.

Join the ‘real solvers’

ready to futurify?

If you are excited by the possibilities of what an AI‑native engineering‑led, modern tech consultancy can do to futurify businesses, and experience the ‘Art of the possible’.