Data Engineer

Tekskills; Jaipur; Hybrid; INR 4,000,000 - 7,000,000; Full time

Job summary

Tekskills in India seeks a senior data engineer to design, develop, and optimize scalable data pipelines using Python, PySpark on Databricks, andAzure cloud services. The role involves implementing data transformations, governance, and cost-efficient data workflows in an Agile setup.

You will collaborate with cross-functional teams, build CI/CD pipelines, and ensure code quality with unit tests, while managing data governance and security across the platform.

Qualifications

• Deep expertise in Python.
• Experience with Databricks, PySpark and Git.
• Experience with Azure cloud platform.

Responsibilities

• Design, develop, and optimize scalable data pipelines and ETL processes using Azure Data Services and Databricks.
• Implement data transformations and analytics using PySpark and Python.
• Configure and manage Databricks Unity Catalog for secure data governance and access control.
• Orchestrate workflows using Apache Airflow for scheduling and automation.
• Collaborate with cross-functional teams to deliver high-quality data solutions in an Agile environment.
• Set up and maintain CI/CD pipelines for data workflows and deployments.
• Manage version control and branching strategies using Git.
• Write and execute unit tests to ensure code quality and reliability.
• Monitor, troubleshoot, and optimize data processes for performance and cost efficiency

Skills

Python

Databricks

PySpark

Git

Azure Cloud

Tools

Azure Data Services

Airflow

Unity Catalog

CI/CD

Job description

JOB Description:

Experience: 10+ Years

Location: Pan India

Mandatory skills

• Deep expertise in python
• Experience with Databricks, pyspark and Git.
• Experience with Azure cloud platform.

Job Details

• Design, develop, and optimize scalable data pipelines and ETL processes using Azure Data Services and Databricks.
• Implement data transformations and analytics using PySpark and Python.
• Configure and manage Databricks Unity Catalog for secure data governance and access control.
• Orchestrate workflows using Apache Airflow for scheduling and automation.
• Collaborate with cross-functional teams to deliver high-quality data solutions in an Agile environment.
• Set up and maintain CI/CD pipelines for data workflows and deployments.
• Manage version control and branching strategies using Git.
• Write and execute unit tests to ensure code quality and reliability.
• Monitor, troubleshoot, and optimize data processes for performance and cost efficiency