Technical Lead - Databricks Champion

sigmoid; Bengaluru; On-site; INR 4,200,000 - 7,000,000; Full time

Job summary

Sigmoid is seeking a Technical Lead - Data Engineering to build a scalable data platform capable of collecting, storing, and analyzing massive datasets from multiple channels. You will work directly with clients and senior leaders, shaping technical direction and delivering robust data solutions.

The role requires deep expertise in Hadoop/Spark, cloud infrastructure, and distributed systems, with hands-on coding and architectural leadership in a fast-growing environment.

Qualifications

• 8+ years of experience in data engineering or a related field.
• Experience architecting enterprise-scale big data platforms.
• Proficient with Hadoop, Spark and cloud infrastructure (GCP/AWS).

Responsibilities

• Align Sigmoid with client initiatives and strategic goals.

Skills

Big data

Python

Scala

Spark

Hadoop

Cloud platforms

Distributed systems

API design

Leadership

Communication

Education

BS in CS / related field

Tools

Hadoop

Spark

Oozie

Kafka

GCS

Job description

Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world's leading data-rich organizations across sectors such as CPG-retail, BFSI, life sciences, manufacturing, and more to solve complex business challenges. Our global team specializes in cloud data modernization, predictive analytics, generative AI, and DataOps, supported by 10+ delivery centers and innovation hubs, including a major global presence in Bengaluru and operations across the USA, Canada, UK, Netherlands, Poland, Singapore, and India.

Recognized as a leader in the data and analytics space, Sigmoid is backed by Peak XV Partners and has consistently received accolades for innovation and rapid growth. Highlights include being named a 'Leader' in ISG's Specialty Analytics Services for Supply Chain (2024), a two-time 'India Future Unicorn' by Hurun India, and a four-time honoree on both the Inc. 500 and Deloitte Technology Fast 500 lists.

Technical Lead - Data Engineering

As a Technical Lead, you will be responsible in building a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels.

This position reports to Engineering Manager.

Responsibilities

• Align Sigmoid with key Client initiatives
• Interface daily with customers across leading Fortune 500 companies to understand strategic requirements
• Connect with VP and Director level clients on a regular basis.
• Travel to client locations
• Ability to understand business requirements and tie them to technology solutions
• Facilitate in Technical Aspects
• Design, develop and evolve highly scalable and fault-tolerant distributed components using Big data technologies.
• Excellent experience in Application development and support, integration development and data management.
• Provide technical leadership and manage it day to day basis
• Guiding developers in day to day design and coding tasks
• Play a key role in hiring technical talents to build the future of Sigmoid.
• Stay up-to-date on the latest technology to ensure the greatest ROI for customer & Sigmoid
• Hands on coder with good understanding on enterprise level code
• Design and implement APIs, abstractions and integration patterns to solve challenging distributed computing problems
• Experience in defining technical requirements, data extraction, data transformation, automating jobs, productionizing jobs, and exploring new big data technologies within a Parallel Processing environment

Culture

Must be a strategic thinker with the ability to think unconventional / out:of:box.

Analytical and data driven orientation.

Raw intellect, talent and energy are critical.

Entrepreneurial and Agile : understands the demands of a private, high growth company.

Ability to be both a leader and hands on "doer".

Qualifications

• 8+ years of track record of relevant work experience and a computer Science or a related technical discipline is required
• Experience in architecture and delivery of Enterprise scale applications and capable in developing framework, design patterns etc. Should be able to understand and tackle technical challenges, propose comprehensive solutions and guide junior staff
• Proven track record of building and shipping large-scale engineering products and/or knowledge of cloud infrastructure such as GCP/AWS preferred
• Experience working with large, complex data sets from a variety of sources
• Experience with Hadoop, Spark, or similar stack a must
• Experience with functional and object-oriented programming, Python or Scala a must
• Effective communication skills (both written and verbal)
• Ability to collaborate with a diverse set of engineers, data scientists and product managers
• Technical knowledge in Spark, Hadoop & GCS Stack.
• Comfort in a fast-paced start-up environment

Preferred Qualification

• Experience in agile methodology
• Development and support experience in Big Data domain
• Architecting, developing, implementing and maintaining Big Data solutions
• Experience with database modeling and development, data mining and warehousing.
• Experience with Hadoop ecosystem (HDFS, MapReduce, Oozie, Hive, Impala, Spark, Kerberos, KAFKA, etc)