Data Engineer

Recex.co; Bengaluru; On-site; INR 1,200,000 - 1,500,000; Full time

Job summary

A data consultancy firm is looking for an experienced Data Engineer to help build scalable data solutions. You will design and maintain data pipelines, ensuring data quality and accessibility. The ideal candidate should have strong SQL and Python skills, experience in cloud platforms like Azure, and over 5 years of relevant experience. You will collaborate across teams to support business intelligence initiatives and optimize data workflows.

Qualifications

• 5+ years of experience in software engineering with 3+ years in data engineering or related roles.
• Strong SQL skills with experience in relational and NoSQL databases.
• Proven track record of building production-grade data pipelines.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines.
• Collaborate with architects to implement data warehouse solutions.
• Develop data validation frameworks and quality checks.
• Support analytics teams with data access and optimization.

Skills

Machine learning workflows

Data visualization tools (Tableau, Looker, Power BI)

Stream processing and real-time analytics

Data governance and compliance frameworks (GDPR, CCPA)

Open-source contributions

Microsoft Fabric, Dataiku, Snowflake

Education

Relevant Cloud certifications (e.g., Microsoft Certified: Azure Data Engineer Associate, AWS Certified Data Engineer, Google Cloud Professional Data Engineer)

Tools

Python

Azure (Fabric, Synapse, Data Factory, Event Hubs)

SQL (Microsoft SQL Server, PostgreSQL, MySQL)

NoSQL (MongoDB, Cassandra)

Git

Docker

Kubernetes

Job description

Join to apply for the Data Engineer role at Recex.co.

Position Overview

We are seeking an experienced Data Engineer to join our innovative data team and help build scalable data infrastructure, software consultancy, and development services that power business intelligence, analytics, and machine learning initiatives. The ideal candidate will design, develop, and maintain robust high-performance data pipelines and solutions while ensuring data quality, reliability, and accessibility across the organization.

Experience: 5 to 7 years

Location: Bengaluru

Employment Type: Full-time / Permanent

Key Responsibilities

Data Pipeline Development

• Design, build, and maintain highly efficient and scalable ETL/ELT pipelines to process large volumes of structured and unstructured data.
• Implement real-time and batch data processing solutions using modern data engineering tools (e.g., Fabric, Spark, Kafka, Airflow).
• Optimize data workflows for performance, reliability, and cost-effectiveness.
• Monitor pipeline health and implement automated alerting and recovery mechanisms.

Data Architecture & Infrastructure

• Collaborate with architects to design and implement data warehouse, data lake, and lakehouse solutions (e.g., Fabric, Snowflake).
• Build and maintain cloud-based data infrastructure on Azure, AWS, or GCP.
• Implement data governance frameworks and ensure compliance with data privacy regulations.
• Design schema and data models for optimal query performance and scalability.

Data Quality & Integration

• Develop data validation frameworks and implement quality checks throughout pipelines.
• Integrate data from multiple sources including APIs, databases, streaming platforms, and third-party services.
• Create and maintain data documentation, lineage tracking, and metadata management.
• Troubleshoot data quality issues and implement corrective measures.

Collaboration & Support

• Partner with data scientists, analysts, and business stakeholders to understand data requirements.
• Support analytics teams with data access, modeling, and optimization.
• Participate in code reviews and maintain high standards for data engineering practices.
• Mentor junior team members and contribute to team knowledge sharing.

Skills

• Knowledge of machine learning workflows and MLOps practices.
• Familiarity with data visualization tools (Tableau, Looker, Power BI).
• Experience with stream processing and real-time analytics.
• Experience with data governance and compliance frameworks (GDPR, CCPA).
• Contributions to open-source data engineering projects.
• Relevant Cloud certifications (e.g., Microsoft Certified: Azure Data Engineer Associate, AWS Certified Data Engineer, Google Cloud Professional Data Engineer).
• Specific experience or certifications in Microsoft Fabric, Dataiku, Snowflake.

Required Qualifications

Technical Skills

• Programming Languages: Proficiency in Python.
• Cloud Platforms: Hands‑on experience with Azure (Fabric, Synapse, Data Factory, Event Hubs).
• 3+ years of experience in data engineering or related roles with 5+ years of software engineering experience.
• Databases: Strong SQL skills and experience with relational (Microsoft SQL Server, PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases.
• Version Control: Proficiency with Git and collaborative development workflows.
• Proven track record of building production‑grade data pipelines handling large‑scale data or solutions.
• Desired experience with containerization (Docker) and orchestration (Kubernetes) technologies.