Opening For Data architecture

Tekskills Inc.; Dadri; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Tekskills Inc. in India is seeking a data engineer to design and implement scalable data pipelines using Azure Data Services, Databricks, PySpark, and Python. The role involves building robust ETL processes, ensuring governance, and delivering analytics-ready data assets.

You will collaborate with cross-functional teams in an Agile environment, manage CI/CD for data workloads, and optimize performance and costs while maintaining quality and reliability across the data platform.

Qualifications

• Experience designing scalable data pipelines and ETL processes.
• Proficiency in PySpark and Python for data transformations.
• Hands-on experience with Databricks Unity Catalog and data governance.

Responsibilities

• Design, build, and optimize scalable data pipelines and ETL processes.
• Implement data transformations and analytics with PySpark and Python.
• Configure Databricks Unity Catalog for secure governance and access control.
• Orchestrate workflows with Apache Airflow for scheduling and automation.
• Collaborate with cross-functional teams in an Agile environment.
• Set up and maintain CI/CD pipelines for data workflows and deployments.
• Manage version control and branching strategies using Git.
• Write and run unit tests to ensure code quality and reliability.
• Monitor, troubleshoot, and optimize data processes for performance and cost.

Skills

Azure Data Services

Databricks

PySpark

Python

Airflow

Git

CI/CD

Unit testing

Agile

Data governance

Tools

Databricks Unity Catalog

Apache Airflow

Git

CI/CD tooling

Job description

• Design, develop, and optimize scalable data pipelines and ETL processes using Azure Data Services and Databricks.
• Implement data transformations and analytics using PySpark and Python.
• Configure and manage Databricks Unity Catalog for secure data governance and access control.
• Orchestrate workflows using Apache Airflow for scheduling and automation.
• Collaborate with cross-functional teams to deliver high-quality data solutions in an Agile environment.
• Set up and maintain CI/CD pipelines for data workflows and deployments.
• Manage version control and branching strategies using Git.
• Write and execute unit tests to ensure code quality and reliability.
• Monitor, troubleshoot, and optimize data processes for performance and cost efficiency.

Location: Bangalore,Hyd,Pune,Mysore,Chennai