Data Engineer

Persistent Systems; Bengaluru; Hybrid; INR 1,200,000 - 2,400,000; Full time

Benefits offered by this job

Hybrid work

Flexible hours

Long Service awards

Insurance coverage

Education sponsorship

Job summary

Persistent Systems seeks a data engineer in Bengaluru to design, develop and optimize scalable data pipelines with Databricks, PySpark, Python and SQL. You will help build modern lakehouse architectures and enable AI-powered analytics using Databricks Genie, collaborating with analytics teams to deliver actionable insights.

You will work on enterprise data platforms, ensuring data quality, governance, security, and scalability, while optimizing processing for performance and cost.

Qualifications

• 5–7 years of data engineering experience.
• Strong knowledge of Databricks Lakehouse architecture.
• Experience designing scalable data pipelines for enterprise data platforms.
• Proficient in SQL and Python.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, and SQL.
• Build and maintain enterprise-grade ETL/ELT pipelines for structured and unstructured data processing.
• Implement modern Lakehouse architectures leveraging Databricks capabilities.
• Develop AI-driven analytics solutions using Databricks Genie and related AI-powered technologies.
• Collaborate with business users, analysts, and data scientists to understand requirements and deliver data-driven solutions.

Skills

Databricks

PySpark

Python

SQL

AI analytics

Tools

Databricks Genie

AWS

Job description

We are seeking a highly skilled Data Engineer with strong expertise in Databricks, PySpark, Python, SQL, and AI-powered analytics solutions. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines, implementing modern Lakehouse architectures, and enabling intelligent data access through Databricks AI capabilities such as Genie. This role offers an opportunity to work on enterprise-scale data platforms, build modern data engineering solutions, and collaborate with business and analytics teams to deliver meaningful insights that drive business decisions. Experience with AWS cloud services and modern data architecture patterns will be an added advantage.

• Experience: 5 to 7 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, and SQL.
• Build and maintain enterprise-grade ETL/ELT pipelines for structured and unstructured data processing.
• Implement modern Lakehouse architectures leveraging Databricks capabilities.
• Develop AI-driven analytics solutions using Databricks Genie and related AI-powered technologies.
• Collaborate with business users, analysts, and data scientists to understand requirements and deliver data-driven solutions.
• Create data models, transformation frameworks, and reusable data engineering components.
• Optimize data processing workloads for performance, scalability, and cost efficiency.
• Implement data quality checks, monitoring, and validation processes across data platforms.
• Develop and maintain data integration solutions across multiple enterprise systems.
• Support self-service analytics initiatives by enabling governed and accessible data platforms.
• Ensure data security, governance, compliance, and privacy standards are implemented across all solutions.
• Work closely with architecture and cloud teams to implement best practices for modern data platforms.
• Troubleshoot and resolve data pipeline issues and production support incidents.
• Participate in code reviews, design discussions, and continuous improvement initiatives.
• Stay updated with emerging trends in Databricks, AI-powered analytics, cloud data engineering, and modern data architectures.

Expertise You'll Bring:

• 5 to 7 years of experience in Data Engineering and Data Platform development.
• Strong hands‑on experience with Databricks, PySpark, Python, and SQL.
• Proven experience designing and developing large-scale data engineering solutions.
• Strong understanding of Databricks Lakehouse architecture and best practices.
• Experience in building and optimizing ETL/ELT pipelines for enterprise data platforms.
• Expertise in data transformation, data integration, and pipeline orchestration.
• Experience working with structured, semi-structured, and unstructured data.
• Strong knowledge of SQL query optimization and performance tuning.
• Experience with data modeling, data warehousing, and analytics solutions.
• Hands‑on experience with Databricks AI capabilities, including Genie, conversational analytics, or GenAI-enabled use cases.
• Understanding of AI-driven analytics and intelligent data access solutions.
• Knowledge of data governance, data quality, security, and compliance principles.
• Experience working within Agile development environments.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and stakeholder management abilities.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let's unleash your full potential at Persistent - persistent.com/careers

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."