Data Engineer / Data Architect

Soul Ai; Delhi; Vor Ort; INR 1.200.000 - 2.400.000; Vollzeit

Zusammenfassung

Soul Ai in Mumbai, Delhi-NCR, Bengaluru and other Indian cities seeks a Data Engineer / Data Architect to design, build, and maintain scalable data infrastructure for client projects. You will enable robust data pipelines, storage, transformation, and access across ecosystems.

The role welcomes developers with strong Python/SQL skills, data modeling expertise, and cloud experience to deliver reliable, high-performance data platforms and collaborate with data scientists, analysts, and engineers.

Qualifikationen

• Strong programming skills in Python, SQL, and Shell scripting.
• Experience with ETL/ELT tools and orchestration frameworks (Airflow, Luigi, dbt).
• Proficiency with relational databases (PostgreSQL, MySQL) and NoSQL (MongoDB, Redis).
• Experience with big data technologies: Spark, Kafka, Hive, Hadoop.
• Data modeling, schema design, and data warehousing concepts.
• Cloud proficiency across AWS/GCP/Azure and services like Redshift, BigQuery, S3, Dataflow, Databricks.
• Knowledge of DevOps/CI/CD for data infra.

Aufgaben

• Design and implement scalable, robust, and secure data pipelines.
• Build ETL/ELT frameworks to collect, clean, and transform data.
• Collaborate with data scientists, analysts, and backend engineers for data access and model integration.
• Maintain data integrity, schema design, lineage, and quality monitoring.
• Optimize performance and reliability of data workflows in production.
• Design and manage data warehousing and lakehouse architectures.
• Set up infrastructure using IaC where applicable.

Kenntnisse

Python

SQL

Shell scripting

Airflow

dbt

PostgreSQL

MongoDB

Spark

Kafka

Hive

Hadoop

AWS

GCP

Azure

CI/CD

Ausbildung

Bachelor's or Master's in CS/Data Engineering/Information Systems

Tools

Redshift

BigQuery

S3

Databricks

Jobbeschreibung

We specialize in delivering high-quality human-curated data and AI-first scaled operations services.

Based in San Francisco and Hyderabad, we are a fast-moving team on a mission to build AI for Good, driving innovation and societal impact.

Role Overview

We are seeking a Data Engineer / Data Architect who will be responsible for designing, building, and maintaining scalable data infrastructure and systems for a client.

Youll play a key role in enabling efficient data flow, storage, transformation, and access across our organization or client ecosystems.

Whether youre just beginning or already an expert, we value strong technical skills, curiosity, and the ability to translate complex requirements into reliable data pipelines.

Responsibilities

• Design and implement scalable, robust, and secure data pipelines
• Build ETL/ELT frameworks to collect, clean, and transform structured and unstructured data
• Collaborate with data scientists, analysts, and backend engineers to enable seamless data access and model integration
• Maintain data integrity, schema design, lineage, and quality monitoring
• Optimize performance and ensure reliability of data workflows in production environments
• Design and manage data warehousing and lakehouse architecture
• Set up and manage infrastructure using IaC (Infrastructure as Code) when applicable

Required Skills

• Strong programming skills in Python, SQL, and Shell scripting
• Hands-on experience with ETL tools and orchestration frameworks (e g, Airflow, Luigi, dbt)
• Proficiency in relational databases (e g , PostgreSQL, MySQL) and NoSQL databases (e g, MongoDB, Redis)
• Experience with big data technologies: Apache Spark, Kafka, Hive, Hadoop, etc
• Deep understanding of data modeling, schema design, and data warehousing concepts
• Proficient with cloud platforms (AWS/GCP/Azure) and services like Redshift, BigQuery, S3, Dataflow, or Databricks
• Knowledge of DevOps and CI/CD tools relevant to data infrastructure

Nice to Have

• Experience working in real-time streaming environments
• Familiarity with containerization and Kubernetes
• Exposure to MLOps and collaboration with ML teams
• Experience with security protocols, data governance, and compliance frameworks

Educational Qualifications

• Bachelors or Masters in Computer Science, Data Engineering, Information Systems, or a related technical field

Location

Location - Mumbai, Delhi / NCR, Bengaluru , Kolkata, Chennai, Hyderabad, Ahmedabad, Pune, India