We are seeking a highly skilled Senior Data Engineer with 8-12 years of experience in designing, developing, and managing scalable data platforms and enterprise data solutions. The ideal candidate will have strong expertise in data engineering, cloud technologies, ETL/ELT development, data warehousing, and big data ecosystems. The candidate will play a key role in building modern data platforms, enabling advanced analytics, business intelligence, AI/ML initiatives, and data-driven decision-making across the organization.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and data integration frameworks.
- Build and optimize ETL/ELT processes for structured and unstructured data sources.
- Develop and manage enterprise data warehouses, data lakes, and lakehouse architectures.
- Implement data ingestion, transformation, and orchestration solutions using cloud-native services.
- Collaborate with business analysts, data scientists, architects, and stakeholders to understand data requirements.
- Ensure data quality, governance, security, and compliance standards are met.
- Optimize database and query performance for large-scale datasets.
- Design and implement real-time and batch data processing solutions.
- Support reporting, analytics, AI/ML, and advanced data engineering initiatives.
- Participate in architecture reviews and provide technical leadership to the data engineering team.
- Develop CI/CD pipelines and automate deployment processes for data platforms.
- Mentor junior engineers and establish engineering best practices.
Required Technical Skills
- Data Engineering
- Data Modeling (Dimensional & Relational)
- ETL/ELT Development
- Data Warehousing Concepts
- Data Lake & Lakehouse Architecture
- Data Migration & Data Integration
- Real-Time Data Processing
- Cloud Platforms (Any One or More)
- Microsoft Azure Azure Data Factory (ADF)
- Azure Synapse Analytics
- Azure Data Lake Storage (ADLS)
- Azure Databricks
- Microsoft Fabric
- AWSGlue Redshift EMR S3
- Google Cloud Platform (GCP) BigQuery Dataflow Cloud Storage
- Big Data Technologies Apache Spark Databricks Hadoop Ecosystem Kafka Delta Lake Databases SQL Server Oracle PostgreSQL MySQL Snowflake Azure SQL Database
- Programming Skills Python SQL PySpark Scala (Preferred)
- Shell Scripting
- DevOps & Version Control Azure DevOps Git CI/CD Pipelines Jenkins
Required Experience
- 8-12 years of overall IT experience.
- Minimum 5+ years of experience in Data Engineering.
- Strong hands-on expertise in SQL and Python.
- Experience with cloud-based data platforms (Azure preferred).
- Expertise in building data pipelines, ETL/ELT frameworks, and data warehouses.
- Experience working with large-scale structured and unstructured datasets.
- Knowledge of data governance, security, and compliance frameworks.
- Experience in Agile/Scrum delivery models.
Preferred Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
- Microsoft Certified: Azure Data Engineer Associate (DP-203).
- Microsoft Fabric Certification.
- Databricks Certified Data Engineer.
- AWS or GCP Data Engineering Certifications.