Data Engineer

Botmet Technologies; Hyderabad; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Botmet Technologies in Hyderabad seeks a Data Engineer to design, implement, and optimize data pipelines using Python, Snowflake, and Azure Data Factory. The role focuses on ELT/ETL processing, data modeling, and scalable cloud-based data platforms to support business applications.

You will collaborate with data architects, analysts, and IT teams to ensure data quality, security, and efficient data delivery.

Qualifications

• 5+ years of experience in data engineering or backend data development.
• Proficient in Python for scripting, data manipulation, and API interaction.
• Hands-on Snowflake performance tuning, schema design, and SQL development.
• Practical experience with Azure Data Factory pipelines, triggers, datasets, and linked services.

Responsibilities

• Develop and maintain robust data pipelines using Python and ADF.
• Design and optimize ELT/ETL processes to load data into Snowflake from various sources.
• Implement performance tuning and monitoring on Snowflake queries and tables.
• Collaborate with data architects and analysts to understand data requirements and provide scalable solutions.
• Automate recurring data ingestion and transformation tasks using Linux scripting.
• Manage data security, access controls, and data quality standards in Snowflake.
• Schedule and monitor ADF pipelines for production loads and troubleshooting failures.
• Integrate external APIs or databases for data enrichment and ingestion.
• Perform root cause analysis and provide timely resolutions to data issues.

Skills

Python scripting

Data pipelines

ETL design

SQL

Cloud data platforms

Tools

Snowflake

Azure Data Factory

Linux

Job description

We are seeking a skilled and motivated Data Engineer with strong expertise in Python, Snowflake, Azure Data Factory (ADF), and Linux. The ideal candidate will be responsible for developing, optimizing, and maintaining data pipelines, ETL workflows, and cloud-based data platforms to support business applications.

Responsibilities

• Develop and maintain robust data pipelines using Python and ADF.
• Design and optimize ELT/ETL processes to load data into Snowflake from various sources.
• Implement performance tuning and monitoring on Snowflake queries and tables.
• Collaborate with data architects and analysts to understand data requirements and provide scalable solutions.
• Automate recurring data ingestion and transformation tasks using scripting in Linux environments.
• Manage data security, access controls, and data quality standards in Snowflake.
• Schedule and monitor ADF pipelines for production loads and troubleshooting failures.
• Integrate external APIs or databases as needed for data enrichment and ingestion.
• Perform root cause analysis and provide timely resolutions to data issues.

Qualifications

• Required Qualifications:
• 5+ years of experience in data engineering or backend data development.
• Strong proficiency in Python for scripting, data manipulation, and API interaction.
• Hands-on experience with Snowflake including schema design, SQL development, and performance tuning.
• Practical experience with Azure Data Factory: pipelines, triggers, datasets, and linked services.
• Proficiency with Linux/Unix command line, shell scripting, and cron jobs.
• Experience working with large-scale datasets and cloud data platforms.
• Strong understanding of data modeling concepts (star/snowflake schema).
• Familiarity with version control systems such as GitLab.
• Preferred Qualifications:
• Experience with Azure services.
• Knowledge of CI/CD pipelines in a data engineering context.
• Background in data warehousing or big data solutions.