AI Lead Data Engineer

Generac Power Systems; Pune District; On-site; INR 4,000,000 - 6,500,000; Full time

Job summary

Generac Power Systems in Pune seeks a senior data/software engineer to own cross-domain data initiatives, design scalable data pipelines, and mentor engineers across squads.

You will drive architectural decisions, collaborate with cloud platforms, and champion data governance while delivering reliable, scalable solutions across the organization.

Qualifications

• 10+ years of experience in data/software engineering with ownership of cross-domain projects.
• Extensive experience building scalable data pipelines and architecture with Spark, Kafka and Airflow.
• Expertise in cloud data platforms (AWS, GCP, or Azure) and distributed systems.
• Strong understanding of data security, governance and compliance.

Responsibilities

• Lead design and implementation of scalable data pipelines across domains.
• Architect enterprise-wide solutions aligned with the company’s standards.
• Mentor engineers and drive knowledge-sharing through reviews and talks.
• Collaborate with principal engineers to ensure system-wide impact.

Skills

Data engineering

Leadership

Mentoring

Cross-domain collaboration

Apache Spark

Kafka

Airflow

Cloud platforms

Distributed systems

Data governance

Education

Bachelor’s or Master’s in CS/Engineering

Tools

Apache Spark

Kafka

Airflow

AWS

GCP

Azure

Job description

We are Generac, a leading energy technology company committed to powering a smarter world.

Over the 60 plus years of Generac’s history, we’ve been dedicated to energy innovation. From creating the home standby generator market category, to our current evolution into an energy technology solutions company, we continue to push new boundaries.

• Cross-Domain Problem Solving: Lead the design and implementation of scalable data pipelines and systems for complex problems that require detailed understanding across multiple domains (e.g., data, machine learning, IoT, cloud infrastructure). These problems will often come with high levels of ambiguity, incomplete data, and evolving requirements.
• Architectural Impact: Contribute to the company’s system architecture with designs that have been battle-tested, resulting in significant, long-lasting impact within a specific domain. Solutions are expected to integrate with the company’s broader enterprise architecture and align with company-wide standards.
• Enterprise-Wide Architecture: Collaborate with principal engineers and directors to ensure designs complement the company’s vision.
• Technical Proposals: Propose technical solutions and strategies that have a significant impact on the company’s data ecosystem. These solutions should drive improvement in the scalability, performance, and resilience of the company’s products and services.
• Component Ownership: Take end-to-end ownership of full components within your domain of expertise, ensuring that their design, implementation, testing, deployment, and operations meet high standards. These components will likely interact with systems in other domains, requiring careful consideration of cross-team dependencies.
• System Operations & SLAs: Define and track SLAs for the components you own, ensuring they meet operational excellence standards and contribute to the system’s overall reliability.
• Maintainability & Scalability: Systematically consider maintainability in designs and implementations, with a focus on ensuring systems can scale to support the organization’s growing data needs.
• Mentor & Lead: Actively mentor engineers across the organization, helping them achieve concrete technical and professional goals. Drive knowledge-sharing initiatives through code reviews, technical talks, and training sessions.
• Cross-Team Collaboration: Facilitate and guide technical discussions across squads, ensuring decisions are aligned with the organization’s strategic goals. You’ll help foster an inclusive environment where all team members feel heard and respected.
• Technical Expertise Development: Participate in "bar-raiser" groups that focus on elevating engineering standards across the company, including leading post-mortem reviews, design sessions, and code reviews.
• Challenging Best Practices: Continuously review existing processes, best practices, and rituals across the company’s engineering organization. Propose and implement improvements that enhance efficiency, collaboration, and quality.
• Delivery Metrics & Quality: Educate teams on key software delivery metrics and help track progress. Ensure that the team’s testing approaches align with accepted frameworks, and work to close gaps in quality metrics.
• Documentation & Knowledge Sharing: Foster a culture of documentation and transparency within the team and across stakeholders, ensuring that key processes and decisions are well-documented and accessible.
• Forward-Thinking Design: Anticipate future data challenges, such as scalability and security concerns, and propose strategies to avoid roadblocks. You’ll look for opportunities to improve existing solutions and identify novel approaches that haven’t been tried before.
• Technology Evaluation: Stay ahead of industry trends by evaluating and recommending new technologies that align with the organization’s goals in data engineering, machine learning, and IoT.
• Domain-Wide Impact: Your work will have a measurable impact across multiple teams within the Data Engineering & Machine Learning Services group. This impact will often have significant customer implications, driving improvements in performance, scalability, and product capabilities.
• Economic Thinking & Risk Management: Drive a culture of thoughtful decision-making, balancing technical innovation with practical constraints like time, cost, and risk. Work closely with partner teams to prioritize capabilities that will deliver the highest business impact.
• Proactive Issue Resolution: Anticipate blockers and delays in projects before they require escalation. Proactively work to resolve these challenges by engaging with stakeholders and partner teams.

Job Requirements:

• 10+ years of experience in data/software engineering, with a proven track record of owning and delivering complex, cross-domain projects at scale.
• Extensive experience in building and maintaining scalable data pipelines and architecture using tools like Apache Spark, Kafka, and Airflow.
• Expertise in cloud data platforms (AWS, GCP, or Azure), with a strong focus on distributed systems, cloud managed open source frameworks and services, and IoT data integration.
• Solid understanding of end-to-end data systems, from ingestion to machine learning model deployment and inference.
• Expertise in data security, data governance, and compliance regulations relevant to the industry.
• Extensive experience in data architecture, database design and data engineering methodologies across multiple industries, with at least 5 years in a technical leadership role.
• Ability to solve problems that span multiple domains, including data engineering, machine learning, IoT, and cloud infrastructure. A deep understanding of how these domains interact is essential.
• Experience with real-time data processing, analytics platforms, and machine learning integration is highly valued.
• Proven ability to mentor and guide engineers, from juniors to senior engineers, across multiple teams. Experience facilitating technical discussions and driving consensus.
• Demonstrated ability to lead cross-functional initiatives and work effectively across squads.
• A strategic mindset, with the ability to think ahead about potential roadblocks and design systems that can scale and evolve with the company’s needs.
• Experience driving large technical initiatives from ideation through implementation, with a focus on creating systems that deliver high business impact.
• Demonstrated track record of contributing to new processes and practices within engineering teams. You’re comfortable challenging the status quo and driving improvements.
• Experience with software delivery metrics and ensuring that teams follow best practices in testing, code quality, and maintainability.