Eucloid - Lead Data Engineer

Eucloid Data Solutions; Chennai District; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Eucloid Data Solutions is seeking a Lead Data Engineer to join our Data Platform team in Chennai. You will lead the design, deployment, and operation of a multi-node big data cluster, coordinating across data lakes, BI, and analytics stakeholders to deliver scalable data infrastructure.

You’ll work with open source and commercial stacks, automate operations with scripts, and develop dashboards and data models to drive business insights. Strong communication and Linux experience are required.

Qualifications

• Design, deploy, and operate multi-node big data clusters.
• Support the full SDLC using open source or commercial stacks.
• Deploy and maintain development, test and production environments for the big data platform.
• Develop scripts to automate operations and configurations in the infrastructure.
• Specify, design, build, and support BI solutions with the datalake team.
• Create dashboards and KPIs to show business performance to management.
• Design and maintain data models for reporting and analytics.
• Identify infrastructure needs and assist developers and business users.
• Research performance issues and optimize platform performance.
• Troubleshoot and resolve issues across all operational environments.
• Work with cross-functional teams delivering software deployments.
• Own the design and development of automated reporting and in-depth analysis.]
• job_responsibilities_description:[

Responsibilities

• Design, deploy, and operate multi-node big data clusters.
• Support the full SDLC using open source or commercial stacks.
• Deploy and maintain development, test and production environments for the big data platform.
• Develop scripts to automate operations and configurations in the infrastructure.
• Specify, design, build, and support BI solutions with the datalake team.
• Create dashboards and KPIs to show business performance to management.
• Design and maintain data models for reporting and analytics.
• Identify infrastructure needs and assist developers and business users.
• Research performance issues and optimize platform performance.
• Troubleshoot and resolve issues across all operational environments.
• Work with cross-functional teams delivering software deployments.
• Own the design and development of automated reporting and in-depth analysis.

Skills

Data lake

Spark

Distributed file systems

YARN

Cloud (GCP/AWS)

SQL tools

Shell scripting

Python scripting

ETL & OLAP pipelines

BI dashboards

Education

Bachelor's/Master's in CS

Tools

Vertica

Dremio

Superset

Tableau

Job description

About The Role

Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components. Candidate will also be involved in projects like integrating data from various sources, managing big data pipelines that are easily accessible with optimised performance of the overall ecosystem.

About The Role

Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components. Candidate will also be involved in projects like integrating data from various sources, managing big data pipelines that are easily accessible with optimised performance of the overall ecosystem.

Responsibilities

• Responsible for the design, deployment, configuration, and operations of a multi-node big data cluster.
• Work with open source and/or commercial stacks to support the full SDLC.
• Deploy, manage, and maintain development, test and production environments for the big data platform.
• Develop scripts to automate and streamline operations and configurations in the infrastructure.
• Specify, design, build, and support BI solutions by working closely with the datalake team.
• Create dashboards and KPIs to show the business performance to management.
• Design and maintain data models used for reporting and analytics.
• Work to identify infrastructure needs and provide support to developers and business users.
• Research performance issues and optimize platform for performance.
• Troubleshoot and resolve issues in all operational environments.
• Work with a cross-functional team delivering software deployments.
• Own the design and development of automated solutions for recurring reporting and in-depth analysis.

Requirements

• Strong experience in Data lake, Spark, distributed file system, Yarn, and Cloud services (GCP / AWS).
• Strong experience in SQL tools like Vertica, Dremio, or any big data SQL.
• Scripting knowledge in Shell and Python.
• Experience with ETL and OLAP concepts in building highly scalable data pipelines.
• Exposure to visualization systems like Apache Superset or Tableau is a plus.
• Experience with Agile, data structures, data analysis, and wrangling tools.
• Familiarity with version control and relational databases.
• Strong experience in monitoring, debugging, troubleshooting, and providing on-call support.

Qualifications

• Bachelor's / Master's Degree in Computer Science or related field from a reputed institution.
• 8+ years of professional experience in software, with most of them from a product company.
• Experience working in a Linux based environment.
• Good communication and design skills.

(ref:hirist.tech)