Data Architect

Soul Ai; Hyderabad; Sur place; INR 800 000 - 1 400 000; Plein temps

Résumé du poste

Soul Ai in Hyderabad is seeking a senior data platform engineer to architect enterprise-scale data platforms, integrating diverse sources and tools to support analytics and AI initiatives.

You will build real-time and batch data pipelines, define data governance, and implement real-time streaming with Kafka, Flink, and cloud services, while adopting DevOps and DataOps best practices.

Qualifications

• Experience designing scalable distributed data systems.
• Hands-on with CI/CD and data infrastructure optimization.
• Proficiency with cloud data warehouses and governance.

Responsabilités

• Design and architect enterprise-scale data platforms.
• Develop real-time and batch data pipelines.
• Define data governance strategies and ensure security, integrity and compliance.
• Implement real-time streaming solutions and DevOps/DataOps practices.

Connaissances

Python/Scala/Java

Apache Spark

Kafka/Flink

Data modelling

Cloud platforms

CI/CD

Kubernetes

Outils

Apache Spark

Hadoop

Flink

Kafka

Snowflake/Redshift/BigQuery

Terraform

Airflow

Kubernetes

Prometheus/Grafana

Description du poste

Responsibilities:

• Design and architect enterprise-scale data platforms, integrating diverse data sources and tools
• Develop real-time and batch data pipelines to support analytics and machine learning
• Define and enforce data governance strategies to ensure security, integrity, and compliance along with optimizing data pipelines for high performance, scalability, and cost efficiency in cloud environments
• Implement solutions for real-time streaming data (Kafka, AWS Kinesis, Apache Flink) and adopt DevOps/DataOps best practices

Required Skills:

• Strong experience in designing scalable, distributed data systems and programming (Python, Scala, Java) with expertise in Apache Spark, Hadoop, Flink, Kafka, and cloud platforms (AWS, Azure, GCP)
• Proficient in data modeling, governance, warehousing (Snowflake, Redshift, Big Query), and security/compliance standards (GDPR, HIPAA)
• Hands‑on experience with CI/CD (Terraform, Cloud Formation, Airflow, Kubernetes) and data infrastructure optimization (Prometheus, Grafana)

Nice to Have:

• Experience with graph databases, machine learning pipeline integration, real-time analytics, and IoT solutions
• Contributions to open‑source data engineering communities