Associate Developer - Data Engineering

3M; Bengaluru; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

3M is hiring an Associate Developer in Data Engineering to design, develop, test, and support scalable data pipelines on AWS using PySpark and Databricks. You will build data models, data ingestion frameworks, and data dictionaries while collaborating with global teams.

The role requires 2–4 years in data warehousing, strong Python/PySpark and SQL skills, and experience with AWS services. Hybrid work location in Hebbal, Bangalore offers rich collaboration opportunities.

Qualifications

• Bachelor's degree or higher, completed and verified prior to start.
• 2–4 years of data warehousing, including data pipelines and data modeling on AWS/Databricks.
• Experience with Python, PySpark, and SQL scripting on AWS services.

Responsibilities

• Design, develop, test, and support scalable data pipelines using AWS and PySpark.
• Build and govern data ingestion and data models in Databricks.
• Create design docs, data lineage, and metadata management.
• Collaborate with global teams and stay current with industry best practices in data engineering, Databricks, and AWS services.

Skills

Python

PySpark

SQL

Analytical skills

English fluency

Education

Bachelor's degree or higher

Tools

AWS

Databricks

Spark SQL

S3

EKS

EMR

CloudFormation

Athena

Job description

Job Description:

3M has a long-standing reputation as a company committed to innovation. We provide the freedom to explore and encourage curiosity and creativity. We gain new insight from diverse thinking, and take risks on new ideas. Here, you can apply your talent in bold ways that matter.

Collaborate with Innovative 3Mers Around the World

Choosing where to start and grow your career has a major impact on your professional and personal life, so it's equally important you know that the company that you choose to work at, and its leaders, will support and guide you. With a wide variety of people, global locations, technologies and products, 3M is a place where you can collaborate with other curious, creative 3Mers.

The Impact You'll Make in this Role

As an Associate Developer, Data Engineering, you will be part of the Data & Analytics Platform Engineering Team, designing, developing, testing, and supporting various platform foundational frameworks in Amazon Web Services (AWS) and Databricks. Here, you will make an impact by:

• Designing, developing, and maintaining scalable data pipelines with delta mechanism in the AWS Lake using AWS services and PySpark.


• Analysing, designing, developing, testing, deploying, and supporting data models in AWS S3 using Spark SQL based on extracts from various sources and functional design specifications.


• Building and administering data ingestion frameworks in Databricks using PySpark and SQL and managing Platform Engineering tasks.


• Reviewing business requirements and creating functional and technical design documents, data lineage, data dictionaries, and metadata management, including SQL scripting.


• Staying current with industry trends and best practices in data engineering, Databricks, AWS services, AWS Lakehouse, PySpark, and Athena while participating actively in local and global teams.

Your Skills and Expertise

To set you up for success in this role from day one, 3M requires (at a minimum) the following qualifications:

• Bachelor's degree or higher (completed and verified prior to start).


• Two (2) to four (4) years of solid experience in data warehousing, including data pipelines and data modeling, particularly on AWS and Databricks platforms, in a private, public, government or military environment.

Additional qualifications that could help you succeed even further in this role include:

• Two (2) to three (3) years of strong experience in Python and PySpark with SQL scripting, plus hands-on experience with AWS services such as S3, Amazon EKS, Amazon EMR, CloudFormation, AWS Secrets Manager, Glue, and Athena.


• Two (2) to three (3) years of solid experience with Databricks, including automated user provisioning, RBACs, ABACs, AI/ML, and workspace management, along with good experience in relational databases and concepts.


• Extensive working knowledge of one or more business processes such as sales, finance, supply chain, manufacturing, or Master Data.


• Detail-oriented, self-directed problem solver with strong analytical and multitasking skills, a strong interest in technology, and English fluency.

Work Location: Hebbal, Bangalore (Karnataka), Hybrid Mode

Supporting Your Well-being

3M offers many programs to help you live your best life - both physically and financially. To ensure competitive pay and benefits, 3M regularly benchmarks with other companies that are comparable in size and scope.

Learn more about 3M's creative solutions to the world's problems at www.3M.com or on Instagram, Facebook, and LinkedIn @3M.

Safety

Safety is a core value at 3M. All employees are expected to contribute to a strong Environmental Health and Safety (EHS) culture by following safety policies, identifying hazards, and engaging in continuous improvement.