Data Engineer (Freelancer)

Soul Ai; Hyderabad; Hybrid; INR 2,009,000 - 3,571,000; Full time

Benefits offered by this job

Above-market pay

Remote work possible

Office Hyderabad/Bangalore

Job summary

Soul Ai is seeking top-tier Data Engineers to design, build and optimize enterprise-scale data pipelines. You will work on real-time and batch processing, data governance, and cloud deployments across AWS/Azure/GCP.

The role emphasizes scalable architectures, DevOps/DataOps practices, and solutions for streaming data and analytics. Join a high-caliber team focused on AI challenges, with opportunities to influence data governance and security while delivering cost-efficient data platforms in

Qualifications

• Strong experience in designing scalable, distributed data systems and programming (Python, Scala, Java) with expertise in Apache Spark, Hadoop, Flink, Kafka, and cloud platforms (AWS, Azure, GCP).
• Proficient in data modeling, governance, warehousing (Snowflake, Redshift, BigQuery), and security/compliance standards (GDPR, HIPAA).
• Hands-on experience with CI/CD (Terraform, Cloud Formation, Airflow, Kubernetes) and data infrastructure optimization (Prometheus, Grafana).
• Nice to Have: Experience with graph databases, machine learning pipeline integration, real-time analytics, and IoT solutions.
• Contributions to open-source data engineering communities.

Responsibilities

• Design and architect enterprise-scale data platforms, integrating diverse data sources and tools.
• Develop real-time and batch data pipelines to support analytics and machine learning.
• Define and enforce data governance strategies to ensure security, integrity, and compliance along with optimizing data pipelines for high performance, scalability, and cost efficiency in cloud environments.
• Implement solutions for real-time streaming data (Kafka, AWS Kinesis, Apache Flink) and adopt DevOps/DataOps best practices.

Skills

Python

Scala

Java

Data modeling

Data governance

Data warehousing

Security & compliance

CI/CD

DevOps

Cloud platforms (AWS/GCP/Azure)

Tools

Apache Spark

Hadoop

Flink

Kafka

Snowflake

Redshift

BigQuery

Terraform

CloudFormation

Airflow

Kubernetes

Prometheus

Grafana

Job description

We are looking for top-tier Data Engineers who are proficient in designing, building and optimizing data pipelines. If you have experience in this field then this is your chance to collaborate with industry leaders.

Whats in it for you.

• Pay above market standards. The role is going to be contract based with project timelines from 2 12 months, or freelancing.
• Be a part of an Elite Community of professionals who can solve complex AI challenges.
• Work location could be:. Remote (Highly likely). Onsite on client location. Deccan AIs OfficeHyderabad or Bangalore.
• Responsibilities:. Design and architect enterprise-scale data platforms, integrating diverse data sources and tools.
• Develop real-time and batch data pipelines to support analytics and machine learning.
• Define and enforce data governance strategies to ensure security, integrity, and compliance along with optimizing data pipelines for high performance, scalability, and cost efficiency in cloud environments.
• Implement solutions for real-time streaming data (Kafka, AWS Kinesis, Apache Flink) and adopt DevOps/DataOps best practices.

Required Skills:

• Strong experience in designing scalable, distributed data systems and programming (Python, Scala, Java) with expertise in Apache Spark, Hadoop, Flink, Kafka, and cloud platforms (AWS, Azure, GCP).
• Proficient in data modeling, governance, warehousing (Snowflake, Redshift, BigQuery), and security/compliance standards (GDPR, HIPAA).
• Hands-on experience with CI/CD (Terraform, Cloud Formation, Airflow, Kubernetes) and data infrastructure optimization (Prometheus, Grafana).
• Nice to Have:. Experience with graph databases, machine learning pipeline integration, real-time analytics, and IoT solutions.
• Contributions to open-source data engineering communities.