Data Engineer

Opella; Hyderabad; On-site; INR 1,500,000 - 2,200,000; Full time

Job summary

Opella is seeking a Data Engineer based in Hyderabad to design and optimize data pipelines using Snowflake and Databricks. The role involves collaboration across teams to implement scalable data solutions using Python, AWS, and Azure. Candidates should have a minimum of 7 years of experience in Data Engineering, strong knowledge of ETL/ELT processes, and a Bachelor's degree in a related field. Opella offers a collaborative work environment focused on data-driven solutions.

Qualifications

• 7+ years of experience in Data Engineering with a focus on database development.
• Strong experience with Snowflake/Databricks, including data modeling and performance tuning.
• Proficiency in Python for building data pipelines and automation.

Responsibilities

• Design, develop, and maintain data pipelines using Python and dbt.
• Build and manage Snowflake/Databricks data warehouses and data marts.
• Leverage AWS/Azure services to develop cloud-based data pipelines.

Skills

Snowflake

Databricks

Python

AWS

Azure

dbt

Git

Education

Bachelor’s degree in computer science, Information Technology, Data Engineering, or a related field

Job description

Job title: Data engineer

Location: Hyderabad

Position Overview

This position will report to the Senior Data Architect within Opella. The Data Engineer will collaborate across multiple business functions and work closely with data scientists, data architects, and data governance teams to implement scalable data pipelines and platforms that support various business needs. They should have strong expertise in Snowflake/Databricks, Python, AWS/Azure, dbt, and Git.

They will be responsible for designing, building, and optimizing data pipelines to support advanced analytics and business intelligence initiatives, ensuring data is accessible, clean, and optimized. They will work with data architects, AI engineers, analysts, and business stakeholders to ensure data is intelligent, trusted, and ready for AI consumption.

Key Responsibilities

• Design, develop, and maintain data pipelines using Python and dbt to ingest, transform, and store data in Snowflake/Databricks.
• Build and manage Snowflake/Databricks data warehouses and data marts, ensuring efficient and scalable data architectures.
• Leverage AWS/Azure services (e.g., S3, Lambda, Glue, EC2) to develop cloud-based data pipelines and automate data workflows.
• Develop ETL/ELT pipelines to load data into Snowflake, ensuring performance, scalability, and reliability.
• Optimize data queries and performance within Snowflake, ensuring efficient use of resources and minimizing query costs.
• Use dbt for transforming and modeling data in Snowflake, implementing robust testing, documentation, and version control processes.
• Support development of AI-ready data platforms by designing high-quality, well-modeled datasets for advanced analytics and machine learning workloads.
• Support AI and machine learning workloads by preparing high-quality, analytics-ready datasets.
• Implement and manage CI/CD pipelines for data engineering projects using Git to ensure smooth development, testing, and deployment processes.
• Ensure data quality, integrity, and consistency through best practices in data governance, documentation, and monitoring.
• Troubleshoot and optimize existing data workflows, resolving any issues with data ingestion, processing, or query performance.

Qualifications

• 7+ years of experience in Data Engineering, with a strong focus on database development, ETL/ELT processes, and cloud platforms.
• Strong experience with Snowflake/DataBricks as a data warehouse platform, including data modeling, performance tuning, and optimization.
• Proficiency in Python for building data pipelines, automation, and data transformations.
• Hands-on experience with AWS cloud services, particularly S3, Lambda, Glue and EC2. Expertise in dbt for managing and transforming data within Snowflake.
• Experience with SQL for complex data queries, transformations, and performance optimization.
• Familiarity with Git for version control, branching, and collaborating on data engineering projects.
• Experience building and maintaining ETL/ELT pipelines in cloud environments. Excellent troubleshooting skills with the ability to optimize and improve existing data processes.
• Ability to work independently and in collaboration with cross-functional teams. Strong communication skills to effectively interact with technical and non-technical stakeholders.

Education

Bachelor’s degree in computer science, Information Technology, Data Engineering, or a related field.