Aws Data Engineer

Tata Consultancy Services; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Tata Consultancy Services (TCS) is hiring an AWS Data Engg (Databricks) with 4-8 years of experience in Bengaluru. You will design and implement robust ETL/ELT pipelines in Databricks using Spark, integrating with AWS services like S3, Glue, Lambda, and Redshift.

The role includes data lake/warehouse modeling, workflow orchestration, performance optimization, and governance. Required skills include Python/Scala, advanced SQL, and strong Spark expertise, plus knowledge of DevOps, Git, and

Qualifications

• Bachelor's degree typical for the role.
• Certifications like AWS Data Engineer or Databricks are beneficial.

Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines in Databricks using Spark.
• Integrate Databricks with AWS services like S3, Glue, Lambda, and Redshift.
• Contribute to data lake/warehouse architecture and modeling.
• Orchestrate workflows with Databricks Workflows or Airflow and optimize performance.
• Collaborate with stakeholders to define data requirements, implement quality checks, and ensure data governance.

Skills

Databricks

Spark

Python/Scala

SQL

AWS data services

ETL/ELT

DevOps/Git

Orchestration (Airflow)

Education

Bachelor's degree

AWS Data Engineer / Databricks certification (beneficial)

Tools

Databricks

Airflow

Git

AWS Glue

Lambda

Redshift

Job description

Greetings from Tata Consultancy Services (TCS)!

AWS Data Engg (Databricks):

Exp: 4-8 yrs

Key responsibilities for this role include designing, developing, and maintaining ETL/ELT data pipelines in Databricks using Spark, and integrating Databricks with various AWS services like S3, Glue, Lambda, and Redshift. They contribute to data lake and data warehouse architecture and modeling, orchestrate workflows with tools like Databricks Workflows or Airflow, and optimize performance and troubleshoot issues. Collaboration with stakeholders to define data requirements, implement quality checks, and ensure data governance is also crucial.

Required Skills & Qualifications

Required skills include proven experience in data engineering with Databricks and AWS data services, proficiency in Python or Scala and advanced SQL, and expertise in Apache Spark. In-depth knowledge of core AWS data services and a solid understanding of data warehousing principles, ETL/ELT, and data modeling are essential. Familiarity with DevOps, Git, and orchestration tools is also needed. A bachelor's degree is typical, and relevant certifications like AWS Certified Data Engineer or Databricks Certified Data Engineer are beneficial.

Note: Associate level AWS Databrick certificate is mandatory. /AWS Data Engineer