Data Engineer (Python, PySpark & Generative AI) | Prominent Staffing | PAN India

Prominent Staffing

India

A distancia

INR 1.500.000 - 2.500.000

Jornada completa

Hace 4 días
Sé de los primeros/as/es en solicitar esta vacante

Una candidatura completa en un minuto

Currículum y carta de presentación personalizados, listos para enviar.

Supera los filtros ATS

Descripción de la vacante

Prominent Staffing is seeking a Data Engineer with 4+ years of experience for a permanent role on a PAN India basis. You will drive data pipeline development, ETL optimization, and cloud data warehousing using Python, PySpark, and SQL, while integrating Gen AI capabilities.

You will collaborate with data scientists, software architects, and business stakeholders to ensure data integrity, security, and scalable solutions across enterprise environments. Remote work within PAN India is supported.

Formación

  • 4+ years of professional data engineering experience.
  • Strong Python, SQL, and PySpark skills.
  • Experience with ML workflows and Gen AI integration.
  • Proficiency with AWS or Azure.
  • Knowledge of data modeling, warehousing, and relational/NoSQL databases.
  • Gen AI tools and cloud data ecosystems familiarity.
  • Legal work authorization for PAN India operations.

Responsabilidades

  • Design, build, test, and scale scalable data pipelines using Python and PySpark.
  • Write and optimize complex SQL for large datasets.
  • Integrate ML models and Gen AI capabilities into data architectures.
  • Deploy and manage cloud-native data infrastructure on AWS or Azure.
  • Collaborate with data scientists, architects, and stakeholders.
  • Ensure data quality, security, and documentation across storage systems.
  • Maintain runbooks and data lineage maps.
  • Participate in agile sprint planning and code reviews.

Conocimientos

Python programming
SQL
PySpark
ML workflows
Gen AI integration
Data engineering
Cloud technologies

Educación

Bachelor's degree in Computer Science / Software Engineering / Data Engineering / IT

Herramientas

Docker
Kubernetes
Terraform
GitHub Actions
Jenkins
Jira
Confluence
Git

Descripción del empleo

Position Summary

Driving advanced data engineering, scalable data pipeline architecture, machine learning model integration, and generative AI innovation across PAN India, Prominent Staffing is actively seeking a passionate and talented Data Engineer with 4+ years of professional experience for a permanent full-time engagement. In this vital big data and analytics engineering role, you will take full operational, tactical, and technical ownership of designing, building, testing, and scaling robust, data-driven solutions utilizing Python, PySpark, SQL, and advanced cloud technologies. Bringing deep technical proficiency in distributed data processing, ETL pipelines, machine learning, and GenAI applications, you will empower an elite international staffing and technology consulting corporation to achieve uncompromised data integrity, lightning-fast processing speeds, and superior client delivery. This prestigious career opportunity offers competitive remuneration, enterprise client exposure, and exceptional professional growth within a leading global organization.

Detailed Job Description

As a Data Engineer supporting Prominent Staffing on a professional PAN India basis, you will take full operational, tactical, and technical ownership of enterprise data pipeline development, ETL optimization, cloud data warehousing, and cross-functional engineering collaboration. Your day-to-day responsibilities include designing, developing, and optimizing large-scale data ingestion and transformation pipelines using Python and PySpark, building robust Extract, Transform, Load (ETL) workflows, writing advanced SQL queries for relational data manipulation, integrating machine learning (ML) models and Generative AI (Gen AI) capabilities into enterprise data architectures, deploying scalable workloads across cloud technologies (AWS or Azure), collaborating proactively with data scientists, software architects, business stakeholders, and technical recruitment leadership like Meghna Yaduvanshi at Prominent Staffing, adhering to strict corporate data engineering and security standards, and managing application requirements.

Key Responsibilities
  • Architect, design, develop, and optimize scalable data pipelines, ETL workflows, and data ingestion architectures using Python and PySpark.
  • Write, tune, and execute complex SQL queries and relational data transformations for high-performance data processing.
  • Integrate machine learning (ML) models and Generative AI (Gen AI) frameworks into modern enterprise data ecosystems.
  • Deploy, configure, and manage cloud-native data infrastructure across Amazon Web Services (AWS) or Microsoft Azure.
  • Execute rigorous performance tuning, query optimization, and latency reduction for large-scale distributed datasets.
  • Collaborate proactively with data scientists, software engineers, analytics teams, and product managers.
  • Ensure high availability, fault tolerance, data quality, and security compliance across all enterprise data storage systems.
  • Maintain meticulous technical documentation, data lineage maps, pipeline schematics, and operational runbooks.
  • Participate actively in agile sprint planning, code reviews, architectural design sessions, and continuous improvement reviews.
  • Deliver high-quality data engineering solutions strictly within established project timelines and milestone objectives.
Required Qualifications & Skills
  • Bachelor's degree in Computer Science, Software Engineering, Data Engineering, Information Technology, or a related technical discipline.
  • Minimum of 4+ years of progressive professional hands-on experience in data engineering, big data processing, and analytics.
  • Strong, demonstrable operational proficiency in Python programming, SQL, and PySpark distributed computing.
  • Comprehensive working knowledge and practical hands-on experience in machine learning (ML) workflows and ETL pipeline development.
  • Proven expertise in cloud technologies including Amazon Web Services (AWS) or Microsoft Azure.
  • Solid practical grasp of modern data modeling, data warehousing concepts, and relational/NoSQL databases.
  • Practical familiarity with Generative AI (Gen AI) tools, large language model integration, and cloud data ecosystems.
  • Mandatory legal work authorization and physical availability for professional operations across PAN India.
Nice-to-Have Skills
  • Professional advanced certifications such as Databricks Certified Data Engineer, AWS Certified Data Analytics, or Microsoft Certified: Azure Data Engineer Associate.
  • Prior professional working experience supporting Tier-1 global technology enterprises, financial institutions, or digital consulting firms.
  • Hands-on familiarity with containerization (Docker, Kubernetes), infrastructure as code (Terraform), and CI/CD pipelines (GitHub Actions, Jenkins).
  • Experience utilizing enterprise agile tracking and version control platforms (Jira, Confluence, Git/GitHub).
  • Active participation in data engineering user groups, Apache Spark meetups, or open-source AI communities.
Application Information
  • Salary/Rate: Market Competitive / Enterprise Data Engineering Package
  • Work Schedule: Professional Remote / PAN India Operations
  • Deadline: Open until filled (Urgent PAN India Data Engineer Opening)
  • Notice Period: Immediate to 30 Days Preferred
  • Contract Duration: Permanent / Full-Time Employment
Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Data Engineer - AWS & Azure
Data Engineer - AWS & Azure

HCL Technologies Limited • Dadri, Chennai, Bengaluru

Híbrido
INR 2.400.000 - 4.000.000
Data Engineer (Python, PySpark & ETL Pipelines) | Zenbyte Technologies | PAN India
Data Engineer (Python, PySpark & ETL Pipelines) | Zenbyte Technologies | PAN India

Zenbyte Technologies • Bengaluru, Chennai

Presencial
INR 1.200.000 - 2.400.000
Technical Lead – Data Engineering | Team Computers | Mumbai | Delhi
Technical Lead – Data Engineering | Team Computers | Mumbai | Delhi

Tech Junction Ltd • Mumbai

Presencial
INR 2.500.000 - 4.200.000
GCP Data Engineer (Google Cloud, BigQuery, PySpark & Cloud Composer) | Techedge Solution | Pan India
GCP Data Engineer (Google Cloud, BigQuery, PySpark & Cloud Composer) | Techedge Solution | Pan India

Techedge Solution • India

A distancia
INR 1.500.000 - 2.700.000
Sr. Data Engineer 4B
Sr. Data Engineer 4B

Genpact Ltd. • Hyderabad

Híbrido
INR 900.000 - 1.500.000
Senior Data Engineer
Senior Data Engineer

RBM Software • Pune District

Presencial
INR 1.200.000 - 1.500.000
Opportunity to work on large-scale data projects
Exposure to modern cloud technologies
Collaborative work environment
Data Engineer
Data Engineer

Persistent Systems • Maharashtra

Híbrido
INR 900.000 - 1.300.000
Competitive salary
Growth opportunities
Cutting-edge technology
+3
Lead Databricks Data Engineer
Lead Databricks Data Engineer

Persistent Systems • Bengaluru

Híbrido
INR 3.500.000 - 6.000.000
Hybrid work model
Medical insurance
Learning @ Work
+2
Senior Data Engineer II
Senior Data Engineer II

Ganit • Bengaluru

Presencial
INR 700.000 - 1.100.000
Career growth opportunities
Global exposure
Data Engineer ID89384
Data Engineer ID89384

AgileEngine, LLC. • Ernakulam

Híbrido
INR 1.500.000 - 2.100.000
Remote work
Local presence in India
Competitive INR compensation
+1