Data Engineer - CB2

Bridgestone Americas; Bengaluru; In loco; INR 1.400.000 - 2.800.000; Tempo pieno

Descrizione del lavoro

Bridgestone Americas is seeking a Data Engineer in Bengaluru to design, develop, and maintain scalable data solutions on AWS, Databricks, Spark, and PySpark. You will build robust pipelines using Medallion architecture, optimize performance, and implement orchestration and data quality checks.

Work with Product, Analytics, and Engineering teams in an Agile environment to deliver production-ready data platforms and foster reusable frameworks and best practices.

Competenze

• 5+ years of experience in Data Engineering and building large-scale data pipelines; capable of providing technical guidance.
• Hands-on AWS experience with S3, Glue, Aurora/RDS, Lambda, and Step Functions; cloud-native data engineering practices.
• 2+ years with Databricks, Spark, and PySpark; knowledge of performance optimization, partitioning, caching, and efficient job design.
• Strong SQL and Python/PySpark skills with data modeling, database design, and ETL/ELT strategies.
• Experience designing and optimizing data pipelines and orchestration workflows, including scheduling, retries, and monitoring.
• Understanding of software engineering practices including coding standards, code reviews, testing, Git, CI/CD, and automation.
• Experience with data quality, validation, reconciliation, monitoring, troubleshooting, and root-cause analysis.
• Understanding of REST APIs and integration with third-party data sources.
• Experience creating, reviewing, and maintaining functional and technical documentation throughout delivery.
• Strong communication and collaboration skills in Agile environments.

Mansioni

• Design, develop, and maintain scalable, high-performance data engineering solutions on AWS.
• Leverage Databricks, Spark, and PySpark to build robust data pipelines across ingestion, transformation, and modeling.
• Implement Medallion Architecture (Raw, Silver, Gold) for data lifecycle management.
• Optimize data platforms for performance, cost efficiency, and reliability across AWS and Databricks.
• Develop orchestration and workflows for scheduling, retries, monitoring, and error handling.
• Contribute to code reviews, testing, CI/CD, deployment, and automation; ensure production-readiness.
• Collaborate with Product, Business, Architecture, QA, API, and Engineering in Agile teams.
• Provide technical guidance and mentorship, promote reusable frameworks and engineering standards.

Conoscenze

Data engineering

ETL/ELT pipelines

AWS cloud

SQL

Python

Spark

PySpark

Data modeling

Strumenti

Databricks

Spark

PySpark

Glue

S3

Lambda

Step Functions

Aurora/RDS

Descrizione del lavoro

Data Engineer

Overview Of Role

• Design, develop, and maintain scalable, high-performance data engineering solutions using AWS
• Databricks, Spark and PySpark, leveraging strong technical acumen to solve complex data and engineering challenges
• Build robust data pipelines following Medallion Architecture (Raw, Silver, and Gold layers), covering data ingestion, transformation, processing, data modeling, validation, quality checks, and reconciliation
• Develop and optimize data solutions using AWS services including S3, Glue, Aurora/RDS, Lambda, and Step Functions, along with Databricks, Spark, and PySpark to improve performance, scalability, reliability, and cloud cost efficiency
• Implement reliable orchestration and operational workflows covering scheduling, dependencies, retries, error handling, monitoring, and failure recovery, while troubleshooting complex data and production issues and driving root-cause resolution
• Develop reusable, maintainable, and production-ready code following engineering standards and best practices; contribute to code reviews, testing, CI/CD, deployment, automation, and continuous improvement
• Support data migration and modernization initiatives across AWS and Databricks, including legacy platform migrations, source-to-target mapping, data validation, reconciliation, and production readiness
• Collaborate with Product, Business, Architecture, QA, API, and Engineering teams in a cross-functional Agile environment, contributing to technical design discussions, estimation, sprint planning, backlog refinement, and delivery
• Provide technical guidance and mentorship to other engineers as applicable, promote reusable frameworks and engineering standards, and contribute to resolving complex technical challenges

Required Qualifications

• 5+ years of experience in Data Engineering, ETL/ELT, and developing large-scale data pipelines, with experience providing technical guidance or leadership as applicable
• Strong hands-on experience with AWS services including S3, Glue, Aurora/RDS, Lambda, and Step Functions, with a strong understanding of cloud-native data engineering practices
• 2+ years of hands-on experience with Databricks, Spark, and PySpark, including Spark performance optimization, partitioning, joins, caching, file formats, data skew, and efficient job design
• Strong SQL and Python/PySpark skills, with experience in data modeling, database design, schema mapping, ETL/ELT, and source-to-target transformations
• Experience designing and optimizing data pipelines and orchestration workflows, including scheduling, dependencies, retries, error handling, monitoring, failure recovery, and performance optimization
• Strong understanding of software engineering practices including coding standards, code reviews, testing, Git, CI/CD, Azure DevOps, reusable frameworks, and automation
• Experience with data quality, validation, reconciliation, monitoring, troubleshooting, and root-cause analysis
• Understanding of REST APIs, API request/response flows, and integration with APIs and third-party source systems
• Experience creating, reviewing, and maintaining functional and technical documentation throughout the delivery lifecycle
• Strong communication and collaboration skills, with experience working across Product, Business, Architecture, QA, API, and Engineering teams in Agile environments
• Strong understanding of Agile practices, including sprint planning, backlog refinement, estimation, iterative delivery, and production support

5

Preferred Qualifications

• Experience with API development and integration, including hands-on experience working with API-driven data solutions
• Experience with Redis or other in-memory databases and caching technologies
• Experience with Databricks and advanced Spark performance optimization techniques
• Experience developing reusable frameworks, automation, and engineering standards
• Relevant AWS certifications in Cloud, Data Engineering, or Solutions Architecture
• Experience developing, debugging, and supporting data solutions within large, cross-functional engineering teams
• Strong analytical and problem-solving skills, with the ability to manage multiple priorities in a deadline-driven environment
• Strong attention to detail and commitment to data quality, reliability, and engineering excellence