Databricks, PySpark, Python - Data Engineer

Infosys Limited; Kerala; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

Infosys Limited in Trivandrum is seeking a Data Engineer with Databricks, PySpark, and Python to design, develop, and deploy data engineering solutions.

You will build scalable ETL/ELT pipelines, optimize Spark jobs, ensure data quality and governance, mentor teammates, and collaborate with stakeholders in Agile sprints.

Qualifications

• Lead the design, development, and deployment of data engineering solutions using Databricks and PySpark.
• Develop scalable ETL/ELT pipelines and data transformation frameworks.
• Write efficient Python and SQL code for data processing and analytics.
• Work closely with stakeholders to understand business requirements and translate them into technical solutions.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Ensure data quality, governance, and security standards are followed.
• Review code, mentor team members, and provide technical guidance.
• Support production deployments, incident troubleshooting, and root cause analysis.
• Participate in Agile ceremonies and drive sprint deliverables.

Responsibilities

• Lead the design and development of data pipelines using Spark-based technologies.
• Collaborate with stakeholders to translate business needs into data solutions.
• Optimize Spark jobs for efficiency and cost savings.
• Mentor junior engineers and participate in code reviews.
• Support production deployments and incident analyses.

Skills

Databricks

PySpark

Python

SQL

Spark

Data pipelines

Education

MBA

MCA

ME

MSc

MTech

B.E.

BCA

BSc

BTech

Tools

Databricks

SQL Tools

Job description

TRIVANDRUM , Infosys Limited Databricks, PySpark, Python - Data Engineer

Job ID/Reference Code INFSYS-EXTERNAL-254130

Work Experience 7 - 9 Years

Job Title Databricks, PySpark, Python - Data Engineer

Educational Requirements MBA,MCA,ME,MSc,MTech,Bachelor of Engineering,BCA,BSc,BTech

Service Line Data & Analytics Unit

Responsibilities Lead the design, development, and deployment of data engineering solutions using Databricks and PySpark.Develop scalable ETL/ELT pipelines and data transformation frameworks.Write efficient Python and SQL code for data processing and analytics.Work closely with stakeholders to understand business requirements and translate them into technical solutions.Optimize Spark jobs for performance, scalability, and cost efficiency.Ensure data quality, governance, and security standards are followed.Review code, mentor team members, and provide technical guidance.Support production deployments, incident troubleshooting, and root cause analysis.Participate in Agile ceremonies and drive sprint deliverables.

Technical and Professional Requirements: 7-9 years of overall IT experience.Strong hands‑on expertise in Databricks and PySpark.Proficiency in Python programming and SQL.Experience building enterprise-scale data pipelines and data lake solutions.Strong understanding of distributed computing concepts and Spark architecture.Experience in performance tuning and optimization.Excellent stakeholder management and team coordination skills.

Preferred Skills: Technology->AI-Data science->PYTHON Technology->Data Engineering->Databricks Technology->Big Data - Data Processing->PySpark