AI Lead Data Engineer

Generac; Ramban district; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Generac seeks a senior data/ML systems leader to drive scalable data pipelines and cross-domain architecture. You will own end-to-end components, guide engineering teams, and ensure SLAs while aligning with enterprise architecture.

The role emphasizes mentoring, collaboration across squads, and proactive improvement of processes. Ideal candidates bring 10+ years in data engineering, cloud platforms (AWS/GCP/Azure), and experience leading technical initiatives with a strong focus on security,

Qualifications

• 10+ years of experience in data/software engineering owning and delivering complex cross-domain projects.
• Extensive experience building and maintaining scalable data pipelines and architectures (Spark, Kafka, Airflow).
• Expertise in cloud data platforms (AWS, GCP, Azure) with distributed systems and IoT data integration.
• End-to-end data systems from ingestion to ML model deployment and inference.
• Experience in data security, governance, and compliance relevant to industry.
• 5+ years in a technical leadership role.

Responsibilities

• Lead design and implementation of scalable data pipelines and systems across data, ML, IoT, and cloud infrastructure.
• Architect and contribute to scalable enterprise data architecture aligned with company standards.
• Collaborate with principal engineers and directors to ensure architectural alignment with the company vision.
• Propose technical solutions to improve scalability, performance, and resilience of products.
• Own full components end-to-end including design, testing, deployment, and operations.
• Define and track SLAs for owned components to ensure reliability.
• Mentor engineers, lead code reviews, talks, and training sessions.
• Foster cross-team collaboration and influence technical discussions across squads.
• Participate in bar-raiser groups to elevate engineering standards.
• Review and improve engineering processes, rituals, and best practices.
• Educate teams on delivery metrics and quality frameworks.
• Foster documentation and knowledge sharing across stakeholders.
• Anticipate future data challenges and propose scalable, secure designs.
• Evaluate new technologies aligned with data, ML, and IoT goals.
• Drive measurable domain-wide impact across Data Eng & ML Services.
• Balance technical innovation with time, cost, and risk in decision making.
• Identify blockers early and coordinate with stakeholders to resolve.

Skills

Cross-domain problem-solving

Mentorship & leadership

Cross-team collaboration

Strategic thinking

Technical leadership

Tools

Apache Spark

Kafka

Airflow

AWS

GCP

Azure

Job description

• Cross-Domain Problem Solving: Lead the design and implementation of scalable data pipelines and systems for complex problems that require detailed understanding across multiple domains (e.g., data, machine learning, IoT, cloud infrastructure). These problems will often come with high levels of ambiguity, incomplete data, and evolving requirements.
• Architectural Impact: Contribute to the company's system architecture with designs that have been battle-tested, resulting in significant, long-lasting impact within a specific domain. Solutions are expected to integrate with the company's broader enterprise architecture and align with company-wide standards.
• Enterprise-Wide Architecture: Collaborate with principal engineers and directors to ensure designs complement the company’s vision.
• Technical Proposals: Propose technical solutions and strategies that have a significant impact on the company's data ecosystem. These solutions should drive improvement in the scalability, performance, and resilience of the company’s products and services.
• Component Ownership: Take end-to-end ownership of full components within your domain of expertise, ensuring that their design, implementation, testing, deployment, and operations meet high standards. These components will likely interact with systems in other domains, requiring careful consideration of cross-team dependencies.
• System Operations & SLAs: Define and track SLAs for the components you own, ensuring they meet operational excellence standards and contribute to the system’s overall reliability.
• Maintainability & Scalability: Systematically consider maintainability in designs and implementations, with a focus on ensuring systems can scale to support the organization's growing data needs.
• Mentor & Lead: Actively mentor engineers across the organization, helping them achieve concrete technical and professional goals. Drive knowledge-sharing initiatives through code reviews, technical talks, and training sessions.
• Cross-Team Collaboration: Facilitate and guide technical discussions across squads, ensuring decisions are aligned with the organization's strategic goals. You’ll help foster an inclusive environment where all team members feel heard and respected.
• Technical Expertise Development: Participate in “bar-raiser” groups that focus on elevating engineering standards across the company, including leading post-mortem reviews, design sessions, and code reviews.
• Challenging Best Practices: Continuously review existing processes, best practices, and rituals across the company's engineering organization. Propose and implement improvements that enhance efficiency, collaboration, and quality.
• Delivery Metrics & Quality: Educate teams on key software delivery metrics and help track progress. Ensure that the team’s testing approaches align with accepted frameworks, and work to close gaps in quality metrics.
• Documentation & Knowledge Sharing: Foster a culture of documentation and transparency within the team and across stakeholders, ensuring that key processes and decisions are well-documented and accessible.
• Forward-Thinking Design: Anticipate future data challenges, such as scalability and security concerns, and propose strategies to avoid roadblocks. You’ll look for opportunities to improve existing solutions and identify novel approaches that haven’t been tried before.
• Technology Evaluation: Stay ahead of industry trends by evaluating and recommending new technologies that align with the organization's goals in data engineering, machine learning, and IoT.
• Domain-Wide Impact: Your work will have a measurable impact across multiple teams within the Data Engineering & Machine Learning Services group. This impact will often have significant customer implications, driving improvements in performance, scalability, and product capabilities.
• Economic Thinking & Risk Management: Drive a culture of thoughtful decision-making, balancing technical innovation with practical constraints like time, cost, and risk. Work closely with partner teams to prioritize capabilities that will deliver the highest business impact.
• Proactive Issue Resolution: Anticipate blockers and delays in projects before they require escalation. Proactively work to resolve these challenges by engaging with stakeholders and partner teams.

Job Requirements

• 10+ years of experience in data/software engineering, with a proven track record of owning and delivering complex, cross-domain projects at scale.
• Extensive experience in building and maintaining scalable data pipelines and architecture using tools like Apache Spark, Kafka, and Airflow.
• Expertise in cloud data platforms (AWS, GCP, or Azure), with a strong focus on distributed systems, cloud managed open source frameworks and services, and IoT data integration.
• Solid understanding of end-to-end data systems, from ingestion to machine learning model deployment and inference.
• Expertise in data security, data governance, and compliance regulations relevant to the industry.
• Extensive experience in data architecture, database design and data engineering methodologies across multiple industries, with at least 5 years in a technical leadership role.
• Ability to solve problems that span multiple domains, including data engineering, machine learning, IoT, and cloud infrastructure. A deep understanding of how these domains interact is essential.
• Experience with real-time data processing, analytics platforms, and machine learning integration is highly valued.
• Proven ability to mentor and guide engineers, from juniors to senior engineers, across multiple teams. Experience facilitating technical discussions and driving consensus.
• Demonstrated ability to lead cross-functional initiatives and work effectively across squads.
• A strategic mindset, with the ability to think ahead about potential roadblocks and design systems that can scale and evolve with the company's needs.
• Experience driving large technical initiatives from ideation through implementation, with a focus on creating systems that deliver high business impact.
• Demonstrated track record of contributing to new processes and practices within engineering teams. You’re comfortable challenging the status quo and driving improvements.
• Experience with software delivery metrics and ensuring that teams follow best practices in testing, code quality, and maintainability.