Databricks Engineer

Persistent Systems; Bengaluru; Hybrid; INR 1,600,000 - 2,100,000; Full time

Benefits offered by this job

Hybrid work

Flexible hours

Education sponsorship

Long Service awards

Insurance coverage

Job summary

Persistent Systems is seeking a Data Engineer - AI & Analytics to design scalable cloud data solutions and optimize pipelines for AI initiatives. You will work with Python, PySpark, and SQL to deliver data platforms that enable GenAI capabilities.

The role offers hybrid work in Bengaluru, with emphasis on Agile/DevOps practices, cross-functional collaboration, and opportunities to work with cutting-edge GenAI technologies. Strong focus on data warehousing, data modeling, and cost efficiency.

Qualifications

• 6+ years of experience in data engineering with SQL, Python and PySpark.
• Experience building cloud-based data and BI solutions on AWS or Azure.
• Strong understanding of data pipelines, ETL/ELT, data modeling, and warehousing.
• Knowledge of GenAI, LLMs, GPT, and Retrieval-Augmented Generation (RAG).
• Excellent analytical, problem-solving, stakeholder management and communication skills.
• Experience working in Agile environments and collaborating with distributed teams.

Responsibilities

• Design, build, and optimize batch and real-time data pipelines using Python, PySpark, and SQL.
• Perform source-to-target data analysis, mapping, and requirement gathering.
• Collaborate with stakeholders and project teams to deliver data solutions.
• Support cloud data platforms, architecture enhancements, and AWS/Azure analytics initiatives.
• Develop GenAI/LLM use cases, including RAG-based productivity improvements.
• Follow Agile, DevOps, DataOps, and DevSecOps practices for project delivery.

Skills

SQL

Python

PySpark

Generative AI

LLMs

Tools

AWS

Azure

Job description

We are seeking a Data Engineer - AI & Analytics to design and deliver scalable cloud-based data solutions, optimize data pipelines, and support AI-driven initiatives. The role combines data engineering, cloud technologies, and Generative AI capabilities to build high-quality, cost-efficient data platforms.

• Experience: 6 to 8 years
• Job Type: Full Time Employment

What You'll Do:

• Design, build, and optimize batch and real-time data pipelines using Python, PySpark, and SQL.
• Perform source-to-target data analysis, mapping, and requirement gathering.
• Collaborate with business stakeholders, data engineers, and project teams to deliver data solutions.
• Support cloud data platforms, architecture enhancements, and AWS/Azure-based analytics initiatives.
• Develop and enable GenAI/LLM use cases, including RAG-based solutions for improved productivity.
• Follow Agile, DevOps, DataOps, and DevSecOps practices to ensure successful project delivery.

Expertise You'll Bring:

• 6+ years of experience in data engineering with strong expertise in SQL, Python, and PySpark.
• Experience building cloud-based data and BI solutions on AWS or Azure.
• Strong understanding of data pipelines, ETL/ELT, data modeling, and data warehousing concepts.
• Knowledge of Generative AI, Large Language Models (LLMs), GPT, and Retrieval-Augmented Generation (RAG).
• Excellent analytical, problem-solving, stakeholder management, and communication skills.
• Experience working in Agile environments and collaborating with globally distributed teams.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

“Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.”