Senior Data Engineer

LatentView Analytics Ltd.; Chennai District; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

LatentView Analytics Ltd. in Chennai, India, is hiring a Data Engineer/Senior Data Engineer with 4–6 years of experience and strong hands-on Python, SQL, and GCP skills.

You will design, build, and maintain scalable data pipelines across cloud and on-premise warehouses, collaborating with analytics and business teams to deliver robust data solutions. The role emphasizes reliable, scalable pipelines, exposure to BigQuery/Dataflow/GCS, PostgreSQL, and experience with Spark/Kafka/Hadoop is a plus.

Qualifications

• 4-6 years of Data Engineering experience.
• Python & SQL proficiency with 3+ years.
• Experience with GCP services: BigQuery, Dataflow, GCS.
• Experience with PostgreSQL and NoSQL databases.
• ETL/ELT and distributed systems expertise.
• Cloud and traditional data warehouses knowledge.
• Spark, Kafka, Hadoop preferred.
• Strong problem-solving and communication skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines from scratch.
• Build and optimize data solutions using Python and SQL.
• Develop data pipelines and applications using GCP services (BigQuery, Dataflow, GCS, PostgreSQL).
• Work with relational and non-relational databases for data integration.
• Maintain data-focused applications across cloud and traditional warehouses.
• Ensure pipelines are reliable, scalable, performant, and highly available.
• Troubleshoot data pipeline, application, and production issues.
• Collaborate with engineering, analytics, and business teams to deliver data solutions.
• Work with big-data technologies such as Spark, Hadoop, Kafka as needed.
• Adopt new tools/techniques to improve processes.
• Provide guidance to new team members.
• Contribute to code reviews, documentation, and testing.
• Work in dynamic environments with evolving requirements.

Skills

Data engineering

Python & SQL

GCP (BigQuery)

PostgreSQL/NoSQL

ETL/ELT

Data warehouses

Spark/Kafka/Hadoop

Problem solving

Communication

Tools

Cloud Composer

Pub/Sub

Cloud Functions

Cloud SQL

Job description

We are looking for a Data Engineer / Senior Data Engineer with 4-6 years of overall experience and strong hands-on development experience in Python, SQL, and Google Cloud Platform (GCP). The candidate will be responsible for designing, developing, and maintaining scalable, distributed, and fault-tolerant data pipelines and data applications.

The ideal candidate should have practical experience with BigQuery, Dataflow, Google Cloud Storage (GCS), PostgreSQL, and cloud-based/traditional data warehouses. Experience with Spark, Kafka, Hadoop, Cloud Composer, Pub/Sub, Cloud Functions, and Cloud SQL will be an added advantage.

Job Responsibilities

• Design, develop, and maintain scalable and fault-tolerant data pipelines from scratch.
• Build and optimize data solutions using Python and SQL.
• Develop data pipelines and applications using GCP services, including BigQuery, Dataflow, GCS, and PostgreSQL.
• Work with both relational and non-relational databases to support data integration and analytics requirements.
• Develop and maintain data-focused applications across cloud and traditional data warehouse environments.
• Ensure data pipelines are reliable, scalable, performant, and highly available.
• Troubleshoot data pipeline, application, and production issues and implement appropriate solutions.
• Collaborate with engineering, analytics, and business teams to understand requirements and deliver data solutions.
• Work with big-data technologies such as Apache Spark, Hadoop, and Kafka where required.
• Explore and adopt new tools, technologies, and techniques to improve data engineering processes.
• Provide technical guidance and support to new team members/joiners.
• Contribute to code reviews, documentation, testing, and best practices.
• Work effectively in a dynamic environment where requirements and problems may not always be clearly defined.

Required Skills

• 4-6 years of Data Engineering experience
• Python & SQL - 3+ years
• GCP - BigQuery, Dataflow, GCS
• PostgreSQL / relational & NoSQL databases
• ETL/ELT & distributed systems
• Cloud & traditional data warehouses
• Apache Spark / Kafka / Hadoop - preferred
• Strong problem-solving and communication skills

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status.