Data Engineer - NMFC

Bridgestone Americas; Bengaluru; In loco; INR 1.800.000 - 3.200.000; Tempo pieno

Descrizione del lavoro

Bridgestone Americas is seeking aData Engineer in Bengaluru to design, develop, and maintain scalable data pipelines using AWS, Databricks, Spark, and PySpark. You will build Medallion-style data layers, optimize performance, and ensure data quality across ingestion, transformation, and governance.

You will collaborate with Product, Architecture, QA, and Engineering in an Agile environment, mentor junior engineers, and drive improvements in automation, CI/CD, and production readiness.

Competenze

• 5+ years of data engineering and ETL/ELT experience.
• 2+ years with Databricks, Spark and PySpark.
• Strong SQL and Python/PySpark skills.
• Experience with data quality, validation, reconciliation and monitoring.

Mansioni

• Design, develop, and maintain scalable data engineering solutions using AWS, Databricks, Spark, and PySpark.
• Build robust data pipelines following Medallion Architecture (Raw, Silver, Gold).
• Develop and optimize data solutions using AWS services (S3, Glue, Aurora/RDS, Lambda, Step Functions).
• Implement orchestration and operational workflows for scheduling, retries, monitoring and error handling.
• Collaborate with cross-functional teams in an Agile environment and contribute to design, estimation and delivery.
• Provide technical guidance and mentorship to other engineers and promote reusable frameworks.

Conoscenze

SQL & Python

Communication

Agile

Collaboration

Mentorship

Strumenti

AWS

Databricks

Spark

PySpark

Azure DevOps

Git

Descrizione del lavoro

Data Engineer

Overview Of Role

• Design, develop, and maintain scalable, high-performance data engineering solutions using AWS,
• Databricks, Spark, and PySpark, leveraging strong technical acumen to solve complex data and engineering challenges.
• Build robust data pipelines following Medallion Architecture (Raw, Silver, and Gold layers), covering data ingestion, transformation, processing, data modeling, validation, quality checks, and reconciliation.
• Develop and optimize data solutions using AWS services including S3, Glue, Aurora/RDS, Lambda, Step Functions, along with Databricks, Spark, and PySpark to improve performance, scalability, reliability, and cloud cost efficiency.
• Implement reliable orchestration and operational workflows covering scheduling, dependencies, retries, error handling, monitoring, and failure recovery, while troubleshooting complex data and production issues and driving root-cause resolution.
• Develop reusable, maintainable, and production-ready code following engineering standards and best practices; contribute to code reviews, testing, CI/CD, deployment, automation, and continuous improvement.
• Support data migration and modernization initiatives across AWS and Databricks, including legacy platform migrations, source-to-target mapping, data validation, reconciliation, and production readiness.
• Collaborate with Product, Business, Architecture, QA, API, and Engineering teams in a cross-functional Agile environment, contributing to technical design discussions, estimation, sprint planning, backlog refinement, and delivery.
• Provide technical guidance and mentorship to other engineers as applicable, promote reusable frameworks and engineering standards, and contribute to resolving complex technical challenges.

Required Qualifications

• 5+ years of experience in Data Engineering, ETL/ELT, and developing large-scale data pipelines, with experience providing technical guidance or leadership as applicable.
• Strong hands-on experience with AWS services including S3, Glue, Aurora/RDS, Lambda, and Step Functions, with a strong understanding of cloud-native data engineering practices.
• 2+ years of hands-on experience with Databricks, Spark, and PySpark, including Spark performance optimization, partitioning, joins, caching, file formats, data skew, and efficient job design.
• Strong SQL and Python/PySpark skills, with experience in data modeling, database design, schema mapping, ETL/ELT, and source-to-target transformations.
• Experience designing and optimizing data pipelines and orchestration workflows, including scheduling, dependencies, retries, error handling, monitoring, failure recovery, and performance optimization.
• Strong understanding of software engineering practices including coding standards, code reviews, testing, Git, CI/CD, Azure DevOps, reusable frameworks, and automation.
• Experience with data quality, validation, reconciliation, monitoring, troubleshooting, and root-cause analysis.
• Understanding of REST APIs, API request/response flows, and integration with APIs and third-party source systems.
• Experience creating, reviewing, and maintaining functional and technical documentation throughout the delivery lifecycle.
• Strong communication and collaboration skills, with experience working across Product, Business, Architecture, QA, API, and Engineering teams in Agile environments.
• Strong understanding of Agile practices, including sprint planning, backlog refinement, estimation, iterative delivery, and production support.

Preferred Qualifications

• Experience with API development and integration, including hands-on experience working with API-driven data solutions.
• Experience with Redis or other in-memory databases and caching technologies.
• Experience with Databricks and advanced Spark performance optimization techniques.
• Experience developing reusable frameworks, automation, and engineering standards.
• Relevant AWS certifications in Cloud, Data Engineering, or Solutions Architecture.
• Experience developing, debugging, and supporting data solutions within large, cross-functional engineering teams.
• Strong analytical and problem-solving skills, with the ability to manage multiple priorities in a deadlinedriven environment.
• Strong attention to detail and commitment to data quality, reliability, and engineering excellence