AI Lead Data Engineer

Generac; Ramban district; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

Generac in India seeks a Senior Data/Software Engineer to lead design and implementation of scalable data pipelines and cross-domain systems, from data ingestion to ML deployment. You’ll shape architecture, drive reliability, and partner with ML and IoT teams.

The role emphasizes mentoring engineers, delivering with SLAs, and guiding initiatives that boost performance and business impact across squads.

Qualifications

• 10+ years in data/software engineering delivering cross-domain projects.
• Experience with scalable data pipelines and architectures.
• Cloud data platforms with IoT data integration.
• End-to-end data systems from ingestion to ML deployment.
• Data security, governance, and regulatory compliance.
• Technical leadership and mentoring across teams.

Responsibilities

• Lead the design and implementation of scalable data pipelines and systems that solve cross-domain problems.
• Contribute to and align with the company’s systemic architecture and standards.
• Mentor engineers and drive knowledge sharing through reviews and talks.
• Collaborate with principal engineers and directors to ensure designs support the broader vision.
• Define and track SLAs and reliability metrics for owned components.
• Proactively resolve blockers and guide cross-team initiatives.

Skills

Cross-domain thinking

Mentor engineers

Technical leadership

Stakeholder alignment

Problem solving

System design

Tools

Spark

Kafka

Airflow

AWS

GCP

Job description

• Cross-Domain Problem Solving: Lead the design and implementation of scalable data pipelines and systems for complex problems that require detailed understanding across multiple domains (e.g., data, machine learning, IoT, cloud infrastructure). These problems will often come with high levels of ambiguity, incomplete data, and evolving requirements.
• Architectural Impact: Contribute to the company's system architecture with designs that have been battle-tested, resulting in significant, long-lasting impact within a specific domain. Solutions are expected to integrate with the company's broader enterprise architecture and align with company-wide standards.
• Enterprise-Wide Architecture: Collaborate with principal engineers and directors to ensure designs complement the company’s vision.
• Technical Proposals: Propose technical solutions and strategies that have a significant impact on the company's data ecosystem. These solutions should drive improvement in the scalability, performance, and resilience of the company’s products and services.
• Component Ownership: Take end-to-end ownership of full components within your domain of expertise, ensuring that their design, implementation, testing, deployment, and operations meet high standards. These components will likely interact with systems in other domains, requiring careful consideration of cross-team dependencies.
• System Operations & SLAs: Define and track SLAs for the components you own, ensuring they meet operational excellence standards and contribute to the system’s overall reliability.
• Maintainability & Scalability: Systematically consider maintainability in designs and implementations, with a focus on ensuring systems can scale to support the organization's growing data needs.
• Mentor & Lead: Actively mentor engineers across the organization, helping them achieve concrete technical and professional goals. Drive knowledge-sharing initiatives through code reviews, technical talks, and training sessions.
• Cross-Team Collaboration: Facilitate and guide technical discussions across squads, ensuring decisions are aligned with the organization's strategic goals. You’ll help foster an inclusive environment where all team members feel heard and respected.
• Technical Expertise Development: Participate in “bar-raiser” groups that focus on elevating engineering standards across the company, including leading post-mortem reviews, design sessions, and code reviews.
• Challenging Best Practices: Continuously review existing processes, best practices, and rituals across the company's engineering organization. Propose and implement improvements that enhance efficiency, collaboration, and quality.
• Delivery Metrics & Quality: Educate teams on key software delivery metrics and help track progress. Ensure that the team’s testing approaches align with accepted frameworks, and work to close gaps in quality metrics.
• Documentation & Knowledge Sharing: Foster a culture of documentation and transparency within the team and across stakeholders, ensuring that key processes and decisions are well-documented and accessible.
• Forward-Thinking Design: Anticipate future data challenges, such as scalability and security concerns, and propose strategies to avoid roadblocks. You’ll look for opportunities to improve existing solutions and identify novel approaches that haven’t been tried before.
• Technology Evaluation: Stay ahead of industry trends by evaluating and recommending new technologies that align with the organization's goals in data engineering, machine learning, and IoT.
• Domain-Wide Impact: Your work will have a measurable impact across multiple teams within the Data Engineering & Machine Learning Services group. This impact will often have significant customer implications, driving improvements in performance, scalability, and product capabilities.
• Economic Thinking & Risk Management: Drive a culture of thoughtful decision-making, balancing technical innovation with practical constraints like time, cost, and risk. Work closely with partner teams to prioritize capabilities that will deliver the highest business impact.
• Proactive Issue Resolution: Anticipate blockers and delays in projects before they require escalation. Proactively work to resolve these challenges by engaging with stakeholders and partner teams.

Job Requirements

• 10+ years of experience in data/software engineering, with a proven track record of owning and delivering complex, cross-domain projects at scale.
• Extensive experience in building and maintaining scalable data pipelines and architecture using tools like Apache Spark, Kafka, and Airflow.
• Expertise in cloud data platforms (AWS, GCP, or Azure), with a strong focus on distributed systems, cloud managed open source frameworks and services, and IoT data integration.
• Solid understanding of end-to-end data systems, from ingestion to machine learning model deployment and inference.
• Expertise in data security, data governance, and compliance regulations relevant to the industry.
• Extensive experience in data architecture, database design and data engineering methodologies across multiple industries, with at least 5 years in a technical leadership role.
• Ability to solve problems that span multiple domains, including data engineering, machine learning, IoT, and cloud infrastructure. A deep understanding of how these domains interact is essential.
• Experience with real-time data processing, analytics platforms, and machine learning integration is highly valued.
• Proven ability to mentor and guide engineers, from juniors to senior engineers, across multiple teams. Experience facilitating technical discussions and driving consensus.
• Demonstrated ability to lead cross-functional initiatives and work effectively across squads.
• A strategic mindset, with the ability to think ahead about potential roadblocks and design systems that can scale and evolve with the company's needs.
• Experience driving large technical initiatives from ideation through implementation, with a focus on creating systems that deliver high business impact.
• Demonstrated track record of contributing to new processes and practices within engineering teams. You’re comfortable challenging the status quo and driving improvements.
• Experience with software delivery metrics and ensuring that teams follow best practices in testing, code quality, and maintainability.