Databricks Engineer

Tata Consultancy Services; BLOOMINGTON (MN); On-site; USD 80,000 - 120,000; Full time

Benefits offered by this job

Medical insurance

401(k)

Job summary

Tata Consultancy Services is looking for a skilled Data Engineer with over 7 years of experience to lead data pipeline optimization on AWS. The ideal candidate will excel in working with Databricks, SQL, and PySpark while collaborating closely with product and project management teams. This full-time role is located in Bloomington, MN, and offers a competitive salary range of $80,000 to $120,000 annually. Join TCS to take on challenges in a dynamic environment and provide mentorship to fellow engineers as part of a fast-paced team.

Qualifications

• 7+ years of experience in data engineering.
• Strong knowledge of Databricks development.
• Experience in optimizing ETL/ELT data pipelines using Databricks on AWS.
• Good understanding of AWS native services.
• Proficient in SQL and PySpark.
• Familiarity with CI/CD pipelines.

Responsibilities

• Design and develop ETL/ELT data pipelines.
• Act as technical lead for medium to large projects.
• Collaborate with teams to identify and prioritize initiatives.
• Troubleshoot issues and performance tuning.
• Guide and coach other Software Engineers.

Skills

Data Engineering

Databricks

Spark

SQL

PySpark

AWS

CI/CD

Problem Solving

Communication

Job description

Tata Consultancy Services provided pay range

This range is provided by Tata Consultancy Services. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$80,000.00/yr - $120,000.00/yr

Direct message the job poster from Tata Consultancy Services

• 7+ years of experience in data engineering with Databricks and Spark
• Strong knowledge of Databricks development, tools and capabilities
• Experience in Design, develop, and optimize ETL/ELT data pipelines using Databricks on AWS.
• Good understanding of AWS native services – S3, Data Sync, DMS, Lambda, SNS, SQS.
• SQL and PySpark : strong command over querying databases and proficiency in PySpark.
• Experience with Delta Lake and building data lakes.
• Familiarity with CI/CD pipelines using tools such as Gitlab, or Code Pipeline.
• Strong analytical and problem-solving skills, with the ability to optimize complex data workflows.
• Excellent communication skills and the ability to work in a fast-paced, collaborative environment
• Ability to drive technical and business conversations.
• Prior experience with coding in a core programming language (i.e., Python, Java, Scala) and willingness to learn a base level of Spark.
• Hands-on expertise with complex proofs-of-concept and public cloud platform(s).
• Accountable for the design and development of new and current systems
• Act as technical lead for medium to large complex project/feature enhancements
• Work with the Product Management team, Project Management, Business and User Experience teams
• to identify and prioritize initiatives and features
• Participate in all grooming and priority setting sessions and accountable for project sizing and estimation
• Work closely with other development teams across organization to troubleshoot issues, performance
• tuning and identify opportunities for seamless integrations across systems
• Works closely with the Production Support team to train, guide and troubleshoot production issues
• Provides coaching and technical guidance for Software Engineers at different levels of experience
• Thrives on working in a fast-paced environment.

#LI-RJ2

Salary Range-$80,000-$120,000 a year

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Tata Consultancy Services by 2x

Inferred from the description for this job

Medical insurance

401(k)

Get notified about new Data Engineer jobs in Bloomington, MN.

Minneapolis, MN $110,000.00-$130,000.00 4 months ago

Minneapolis, MN $90,000.00-$120,000.00 1 month ago

Bloomington, MN $140,000.00-$160,000.00 1 week ago

Minneapolis, MN $98,000.00-$115,000.00 2 weeks ago

Minneapolis, MN $62,000.00-$93,000.00 2 weeks ago

Minneapolis, MN $68,763.00-$89,392.00 1 week ago

Minneapolis, MN $90,000.00-$100,000.00 1 month ago

Bloomington, MN $153,440.00-$230,160.00 1 month ago

Maple Grove, MN $82,600.00-$156,900.00 2 weeks ago

Data Engineer - Insights & Analytics using Qlik & Power BI (Plymouth, MN)

Minneapolis, MN $62,000.00-$93,000.00 2 weeks ago

Eden Prairie, MN $71,200.00-$127,200.00 7 hours ago

Minneapolis, MN $100,000.00-$145,000.00 2 days ago

Minneapolis, MN $110,000.00-$130,000.00 3 weeks ago

Eden Prairie, MN $89,800.00-$176,700.00 1 month ago

Senior Data Engineer - Local to MN only (Hybrid)

Principal GCP Data Engineer_Remote_USC/GC or H4EAD_ONLY ON W2

Azure Data Engineer - Ab Initio_only on W2

Minneapolis, MN $152,000.00-$263,000.00 6 days ago

Minneapolis, MN $123,300.00-$205,500.00 1 week ago

Data Engineer/Data Architect (IS Technical Architect)

Minneapolis–Saint Paul, WI $10,000.00-$50,000.00 6 days ago

Greater Minneapolis-St. Paul Area $190,000.00-$195,000.00 3 weeks ago

Minneapolis, MN $75,000.00-$95,000.00 2 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.