Data Engineer (Freelancer)

Soul Ai; Kolkata District; Remoto; INR 2.009.000 - 3.348.000; Part-time

Vantaggi offerti da questo lavoro

Competitive compensation

Contract-based projects

Elite community access

Descrizione del lavoro

Soul AI invites India’s top 1% Data Engineers to join a contract-based opportunity to design, build and optimize enterprise-scale data pipelines for analytics and ML.

You will work with industry leaders, implement real-time streaming (Kafka, Kinesis, Flink), govern data security and cost-efficient cloud pipelines across AWS/Azure/GCP, with flexible remote options.

Competenze

• Experience designing scalable, distributed data systems and programming (Python/Scala/Java).
• Strong knowledge of Spark, Hadoop, Flink, Kafka and cloud platforms (AWS/Azure/GCP).
• Hands-on with CI/CD and data infra tools like Terraform, Airflow, Kubernetes.

Mansioni

• Design and architect enterprise-scale data platforms, integrating diverse data sources and tools.
• Develop real-time and batch data pipelines to support analytics and ML.
• Define and enforce data governance for security, integrity and compliance; optimize pipelines for performance, cost in cloud environments.
• Implement real-time streaming solutions (Kafka, Kinesis, Flink) and adopt DevOps/DataOps practices.

Conoscenze

Data pipelines

Python

Spark

Cloud platforms

Formazione

Bachelor's in CS/IEL or related

Strumenti

Kafka

Flink

Snowflake

Redshift

BigQuery

Terraform

Airflow

Kubernetes

Descrizione del lavoro

Step into the world of AI innovation with the Experts Community of Soul AI (By Deccan AI).

We are looking for Indias top 1% Data Engineers for a unique job opportunity to work with the industry leaders.

Who can be a part of the community.

We are looking for top-tier Data Engineers who are proficient in designing, building and optimizing data pipelines. If you have experience in this field then this is your chance to collaborate with industry leaders.

What's in it for you.

• Pay above market standards.
• The role is going to be contract based with project timelines from 2 12 months, or freelancing.
• Be a part of an Elite Community of professionals who can solve complex AI challenges.

Work location could be:

• Remote (Highly likely)
• Onsite on client location
• Deccan AIs OfficeHyderabad or Bangalore

Responsibilities:

• Design and architect enterprise-scale data platforms, integrating diverse data sources and tools.
• Develop real-time and batch data pipelines to support analytics and machine learning.
• Define and enforce data governance strategies to ensure security, integrity, and compliance along with optimizing data pipelines for high performance, scalability, and cost efficiency in cloud environments.
• Implement solutions for real-time streaming data (Kafka, AWS Kinesis, Apache Flink) and adopt DevOps/DataOps best practices.

Required

• Skills:
• Strong experience in designing scalable, distributed data systems and programming (Python, Scala, Java) with expertise in Apache Spark, Hadoop, Flink, Kafka, and cloud platforms (AWS, Azure, GCP).
• Proficient in data modeling, governance, warehousing (Snowflake, Redshift, BigQuery), and security/compliance standards (GDPR, HIPAA).
• Hands-on experience with CI/CD (Terraform, Cloud Formation, Airflow, Kubernetes) and data infrastructure optimization (Prometheus, Grafana).

Nice to Have.

• Experience with graph databases, machine learning pipeline integration, real-time analytics, and IoT solutions.
• Contributions to open-source data engineering communities.