Data Engineer

Tekskills; Ernakulam; Hybrid; INR 1,800,000 - 2,800,000; Full time

Job summary

Tekskills is seeking a Senior Data Engineer to design, develop, and optimize scalable data pipelines using Azure Data Services and Databricks. The role requires deep Python expertise and hands-on experience with PySpark, Databricks Unity Catalog, and Git for version control.

You will collaborate with cross-functional teams in an Agile environment, implement data transformations, and set up CI/CD pipelines for data workflows, ensuring performance and cost efficiency.

Qualifications

• 10+ years of data engineering experience.
• Deep expertise in Python and PySpark.
• Experience with Databricks, Git and Azure cloud platform.

Responsibilities

• Design, develop, and optimize scalable data pipelines and ETL processes using Azure Data Services and Databricks.
• Implement data transformations and analytics using PySpark and Python.
• Configure and manage Databricks Unity Catalog for secure data governance and access control.

Skills

Python

Databricks

PySpark

Git

Azure

Job description

JOB Description:

Experience: 10+ Years

Location: Pan India

Mandatory skills

• Deep expertise in python
• Experience with Databricks, pyspark and Git.
• Experience with Azure cloud platform.

Detailed JD

• Design, develop, and optimize scalable data pipelines and ETL processes using Azure Data Services and Databricks.
• Implement data transformations and analytics using PySpark and Python.
• Configure and manage Databricks Unity Catalog for secure data governance and access control.
• Orchestrate workflows using Apache Airflow for scheduling and automation.
• Collaborate with cross-functional teams to deliver high-quality data solutions in an Agile environment.
• Set up and maintain CI/CD pipelines for data workflows and deployments.
• Manage version control and branching strategies using Git.
• Write and execute unit tests to ensure code quality and reliability.
• Monitor, troubleshoot, and optimize data processes for performance and cost efficiency