Senior Data Engineer

Experian; Hyderabad; Hybrid; INR 1,400,000 - 2,300,000; Full time

Job summary

Experian is hiring a Data Engineer - Scala in Hyderabad for a hybrid role requiring 40% travel to the office. You will build and maintain batch pipelines with Scala and Spark, optimize workloads in AWS, and collaborate with engineers and analysts to deliver robust data solutions.

The ideal candidate has 3–6 years of software development experience, strong Scala and Spark skills, and hands-on experience with AWS, DevOps tools, and CI/CD practices in an Agile environment.

Qualifications

• 3-6 years of software development experience with at least 2 years in Scala.
• Clean, scalable, and reusable code using SOLID and design patterns.
• Understanding of functional programming and distributed computing.
• Experience with DevOps and IaC tools including Terraform and CI/CD.
• Hands-on Apache Spark for batch data processing in AWS.

Responsibilities

• Develop and maintain batch data pipelines using Scala and Spark.
• Optimize Spark workloads for performance and scalability in AWS.
• Collaborate with data engineers and analysts to deliver data solutions.
• Ensure data integrity and reliability across systems.
• Participate in code reviews, technical design, and Agile activities.
• Monitor and troubleshoot production data pipelines and resolve issues.

Skills

Scala

Apache Spark

Functional programming

Distributed computing

Agile

Tools

Terraform

CI/CD pipelines

Git

Jenkins

Docker

Kubernetes

Bitbucket

Jira

Job description

Company Description

Experian is a global data and technology company, powering opportunities for people and businesses around the world. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more. Experian invests in people and new advanced technologies to unlock the power of data. We have an amazing team of 25,200 people in 32 countries. Our uniqueness is that we celebrate yours. Experian's people first, inclusive and purpose driven culture is multi award-winning; World's Best Workplaces™ 2025 (Fortune Global Top 25), Great Place To Work™ in 26 countries to name a few. Check out Experian Life on social or explore our Careers Site (experian.com/careers) to understand why.

Job Description

We are looking for Data Engineer - Scala. This is a hybrid role requiring travelling to Hyderabad office 40% per month.

Main Responsibilities

• You will be based in Hyderabad and reporting to Director Engineering.
• Develop and maintain batch data processing pipelines using Scala and Apache Spark
• Optimize Spark workloads for performance and scalability in an AWS cloud environment.
• You will partner with data engineers and analysts to understand data needs and deliver high-quality data solutions.
• Ensure data integrity, reliability, and consistency across systems.
• Participate in code reviews, technical design discussions, and Agile development activities.
• Monitor and troubleshoot production data pipelines and support issue resolution

Qualifications

• 3-6 years of software development experience, including at least 2 years working with Scala.
• Experience writing clean, scalable, and reusable code using SOLID principles, coding standards, and common design patterns where appropriate.
• Understanding of functional programming concepts and distributed computing.
• Experience with DevOps and Infrastructure as Code (IaC) tools, including Terraform, CI/CD pipelines, Git, and Jenkins.
• Hands-on experience using Apache Spark for batch data processing.
• Familiarity with AWS services such as Amazon S3, AWS Glue, AWS Lambda, AWS Step Functions, AWS Batch, and CloudWatch.
• Experience working with structured and semi-structured data formats, including Parquet and JSON.
• Experience across the full software development lifecycle and application lifecycle management tools such as Bitbucket and Jira.
• Experience with data lake architectures and ETL frameworks.
• Exposure to containerization technologies such as Docker and Kubernetes.
• Knowledge of data governance and data security best practices.
• Relevant certifications in AWS, Spark, or Scala will be an advantage.

Additional Information

Our uniqueness is that we celebrate yours. Experian's culture and people are important differentiators. We take our people agenda very seriously and focus on what matters; DEI, work/life balance, development, authenticity, collaboration, wellness, reward & recognition, volunteering... the list goes on. Experian's people first approach is award-winning; World's Best Workplaces™ 2024 (Fortune Global Top 25), Great Place To Work™ in 24 countries, and Glassdoor Best Places to Work 2024 to name a few. Check out Experian Life on social or our Careers Site and Glassdoor to understand why. Experian is also proud to be an Equal Opportunity and Affimative Action employer. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity. Recruitment Fraud Awareness - Experian's recruitment process is conducted only through authorised channels. Recruitment communications will only be sent from an @experian.com email address. Experian will never ask candidates to make any payment as part of an application, interview, assessment, onboarding, or recruitment process.

Benefits

Experian care for employee's work life balance, health, safety and wellbeing. In support of this endeavour, we offer best-in-class family well-being benefits, enhanced medical benefits and paid time off. This is a hybrid remote/in-office role. Experian Careers - Creating a better tomorrow together