Data Engineer

Bsri Solutions; Chennai District, Bengaluru; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Bsri Solutions is seeking a Data Engineer with Gen AI to design, develop and maintain scalable data platforms and ETL/ELT pipelines. The role requires 5+ years of hands-on experience with Python, SQL, cloud data services, and modern data engineering frameworks in a fast-paced environment.

You will collaborate with data scientists and analysts, implement validation and CI/CD practices, and build data lakes on cloud platforms (AWS/Azure/GCP).

Qualifications

• 5+ years of experience in Data Engineering.
• Proficient in Python and SQL with cloud data services.
• Experience with ETL/ELT pipelines and data warehousing concepts.
• Strong knowledge of data modeling and large-scale datasets.

Responsibilities

• Design, develop, and maintain scalable data pipelines.
• Build data ingestion from multiple structured and unstructured sources.
• Develop data models and data warehouses for analytics.
• Write optimized SQL queries for high-performance processing.
• Create and maintain data lakes on cloud storage.
• Implement data validation, cleansing, and quality checks.
• Integrate data solutions with AWS, Azure, or GCP.
• Develop batch and real-time data processing pipelines.
• Collaborate with data scientists and analysts to deliver data solutions.
• Follow Git, CI/CD, testing, monitoring, and documentation best practices.

Skills

Python

PySpark

SQL

Cloud data services

ETL/ELT pipelines

Data modeling

Generative AI/LLMs

Data warehousing

Distributed datasets

Git CI/CD

Agile/Scrum

Education

Bachelor's degree in Computer Science / IT

Tools

Apache Airflow

Kafka

Docker

Kubernetes

Terraform

Databricks

Delta Lake

Apache Iceberg

Apache Hudi

Job description

Role: Data Engineer with Gen AI

Loc: Chennai, Bangalore

About the Role

We are looking for a passionate Data Engineer with 5+ years of experience in designing, developing, and maintaining scalable data platforms and ETL/ELT pipelines. The ideal candidate should possess strong expertise in Python, SQL, cloud data services, and modern data engineering frameworks. You will play a key role in building reliable, high-performance data solutions that support analytics, reporting, and AI/ML initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
• Build and optimize data ingestion processes from multiple structured and unstructured data sources.
• Develop robust data models and data warehouses for analytics and reporting.
• Design and optimize SQL queries for high-performance data processing.
• Build and maintain data lakes using cloud storage solutions.
• Implement data validation, cleansing, transformation, and quality checks.
• Integrate data solutions with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Develop batch and real-time data processing pipelines using modern data processing frameworks.
• Collaborate with data scientists, analysts, and application teams to deliver reliable data solutions.
• Follow software engineering best practices, including Git, CI/CD pipelines, testing, monitoring, and technical documentation.

Required Skills

• 5+ years of experience in Data Engineering.
• Strong programming experience in Python.
• Hands-on experience with PySpark and Apache Spark.
• Advanced SQL skills, including query optimization and performance tuning.
• Experience with AWS services/ Azure/ GCP.
• Experience building scalable ETL/ELT pipelines.
• Strong exp in Generative AI and Large Language Models (LLMs)
• Knowledge of data warehousing concepts and dimensional modeling.
• Experience working with large-scale distributed datasets.
• Familiarity with Git and CI/CD pipelines.
• Understanding Agile/Scrum methodologies.

Preferred Qualifications

• Experience with Apache Airflow or similar workflow orchestration tools.
• Knowledge of Kafka and real-time data streaming.
• Familiarity with Docker, Kubernetes, and Terraform (IaC).
• Exposure to Databricks and modern data lake technologies (Delta Lake, Apache Iceberg, or Apache Hudi).
• Understanding data governance, metadata management, and CI/CD practices.
• Strong exp in Generative AI and Large Language Models (LLMs)
• Strong analytical, problem-solving, and communication skills.

Qualifications

Bachelor's degree in computer science, Information Technology, Engineering, or related technical discipline.