Director Data Engineer

Embarkgcc Services; Hyderabad; On-site; INR 2,000,000 - 4,000,000; Full time

Job summary

Embarkgcc Services is seeking a highly skilled Sr Data Engineer to design, build, and maintain scalable data pipelines and infrastructure for enterprise-wide data initiatives. You will collaborate with cross-functional teams to ensure reliable data systems and performance across cloud platforms and distributed environments.

You will lead complex data engineering tasks, contribute to architectural decisions, and promote best practices for secure, high-quality data solutions.

Qualifications

• Bachelor’s degree in computer science or related field.
• 10+ years of data engineering or related roles.
• 10+ years in Python, Java, SQL, Scala.
• 10+ years of experience with database systems and advanced SQL.
• 10+ years with ETL tools such as SSIS, Snowflake, Databricks, Azure Data Factory.
• Experience with Hadoop, Spark, Kafka.
• 7+ years on Azure, AWS, or Google Cloud.
• Strong analytical, problem-solving, and collaboration skills.

Responsibilities

• Design, build, and optimize end-to-end data pipelines.
• Model data schemas and architectures for analytics and operations.
• Develop APIs and data integrations with internal and external sources.
• Manage data infrastructure including warehouses and lakehouses.
• Collaborate with data scientists and stakeholders; document designs.

Skills

Python

Java

SQL

Scala

Stored Procedures

ETL Tools

Snowflake

Databricks

Azure Data Factory

Hadoop

Spark

Kafka

Azure

AWS

Google Cloud

Data Modeling

APIs

RESTful APIs

Education

Bachelor’s degree in computer science

Tools

SSIS

Azure Data Factory

Databricks

Job description

SUMMARY

We are seeking a highly skilled and motivated Sr Data Engineerto join our innovative team. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support our enterprise-wide data-driven initiatives. You will collaborate closely with cross-functional teams to ensure the availability, reliability, and performance of our data systems and solutions.

This role involves working across modern data engineering frameworks, cloud platforms, and distributed environments to enable efficient data integration, transformation, governance, and consumption across the organization. You will contribute to architectural decisions, lead complex data engineering tasks, and support best practices for high quality, secure, and scalable data solutions.

Responsibilities

Data Pipeline Development

• Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
  
• Develop robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources into our data ecosystem.
  
• Implement data validation and quality checks to ensure accuracy and consistency across systems.
  

Data Modeling and Architecture

• Design and maintain data models, schemas, and database structures for analytical and operational use cases.
  
• Optimize data storage and retrieval mechanisms to support performance and scalability.
  
• Evaluate and implement data storage solutions including relational databases, NoSQL systems, data lakes, and cloud storage platforms.
  

Data Integration and API Development

• Build and maintain integrations with internal and external data sources and APIs.
  
• Implement RESTful APIs and web services for secure and efficient data access and consumption.
  
• Ensure compatibility, standardization, and interoperability across systems.
  

Data Infrastructure Management

• Configure and manage data infrastructure components including databases, warehouses, lakehouses, and distributed computing frameworks.
  
• Monitor system performance, identify bottlenecks, troubleshoot issues, and implement optimizations.
  
• Implement data security controls, governance, and access management policies to protect sensitive information.
  

Collaboration and Documentation

• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver effective technical solutions.
  
• Document technical designs, workflows, data flow diagrams, and best practices to enable transparency and knowledge sharing.
  
• Provide technical guidance, mentorship, and support to team members and cross-functional partners.
  

Requirements

• Bachelor’s degree in computer science, Engineering, Information Systems, or a related field.
  
• 10+ years of proven experience in data engineering, software development, or related technical roles.
  
• 10+ years of experience in programming languages commonly used in data engineering (Python, Java, SQL, Stored Procedures, Scala, etc.).
  
• 10+ years of experience with database systems, data modeling, and advanced SQL.
  
• 10+ years of experience with ETL tools such as SSIS, Snowflake, Databricks, Azure Data Factory, Stored Procedures, etc.
  
• Experience with big data technologies such as Hadoop, Spark, Kafka, etc.
  
• 7+ years of experience working with cloud platforms like Azure, AWS, or Google Cloud.
  
• Strong analytical, problem-solving, and debugging skills with high attention to detail.
  
• Excellent communication and collaboration skills in a team-oriented, fast-paced environment.
  
• Ability to adapt to rapidly evolving technologies and business requirements.