Interesting Job Opportunity: Eucloid - Lead Data Engineer

Eucloid Data Solutions; Chennai District; On-site; INR 3,000,000 - 5,000,000; Full time

Job summary

Eucloid Data Solutions seeks a Lead Data Engineer to join the Data Platform team to build and optimize data infrastructure for diverse client applications. You will participate in tech selection and component design, handle data integration from multiple sources, and manage scalable data pipelines for high-performance analytics.

The role involves designing data models, deploying and maintaining the platform, and delivering automated reporting solutions in collaboration with cross-functional

Qualifications

• Bachelor's or Master's degree in CS or related field.
• 8+ years of professional software experience, preferably in a product company.
• Experience in a Linux-based environment.
• Strong communication and design skills.

Responsibilities

• Design, deploy, configure, and operate a multi-node big data cluster.
• Maintain development, test, and production environments for the big data platform.
• Develop automation scripts for operations and configurations.
• Collaborate with data lake team to design and support BI solutions.
• Create dashboards and KPIs to monitor business performance.
• Model data for reporting and analytics.
• Identify infrastructure needs and support developers and users.
• Research performance issues and optimize platform performance.
• Troubleshoot issues across environments and work with cross-functional teams.
• Own the design and development of automated recurring reporting and analysis.

Skills

Good communication

Relational databases

Design skills

Education

Bachelor's / Master's Degree in Computer Science or related field

Tools

Data Lake

Spark

Yarn

Cloud (GCP/AWS)

SQL tools (Vertica/Dremio)

Shell

Python

Job description

About The Role

Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components. Candidate will also be involved in projects like integrating data from various sources, managing big data pipelines that are easily accessible with optimised performance of the overall ecosystem.

About The Role

Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components. Candidate will also be involved in projects like integrating data from various sources, managing big data pipelines that are easily accessible with optimised performance of the overall ecosystem.

Responsibilities

• Responsible for the design, deployment, configuration, and operations of a multi-node big data cluster.
• Work with open source and/or commercial stacks to support the full SDLC.
• Deploy, manage, and maintain development, test and production environments for the big data platform.
• Develop scripts to automate and streamline operations and configurations in the infrastructure.
• Specify, design, build, and support BI solutions by working closely with the datalake team.
• Create dashboards and KPIs to show the business performance to management.
• Design and maintain data models used for reporting and analytics.
• Work to identify infrastructure needs and provide support to developers and business users.
• Research performance issues and optimize platform for performance.
• Troubleshoot and resolve issues in all operational environments.
• Work with a cross-functional team delivering software deployments.
• Own the design and development of automated solutions for recurring reporting and in-depth analysis.

Requirements

• Strong experience in Data lake, Spark, distributed file system, Yarn, and Cloud services (GCP / AWS).
• Strong experience in SQL tools like Vertica, Dremio, or any big data SQL.
• Scripting knowledge in Shell and Python.
• Experience with ETL and OLAP concepts in building highly scalable data pipelines.
• Exposure to visualization systems like Apache Superset or Tableau is a plus.
• Experience with Agile, data structures, data analysis, and wrangling tools.
• Familiarity with version control and relational databases.
• Strong experience in monitoring, debugging, troubleshooting, and providing on-call support.

Qualifications

• Bachelor's / Master's Degree in Computer Science or related field from a reputed institution.
• 8+ years of professional experience in software, with most of them from a product company.
• Experience working in a Linux based environment.
• Good communication and design skills.

(ref:hirist.tech)