Big Data Engineer

Pradeepit Global Consulting Private Limited

Bengaluru

Remote

INR 2,500,000 - 4,500,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

PradeepIT is a leading IT Consulting firm based in Bengaluru, delivering global solutions with a strong focus on data engineering. The Principal Engineer (Big Data Engineer) role targets seasoned professionals to design and implement large-scale data platforms using Spark, PySpark, Scala/Java, and cloud services.

You will lead data pipeline development, manage complex data integrations, and mentor a technical team while collaborating with clients such as Fortune 500 companies.

Qualifications

  • Strong technical background with Big Data, Spark, Python (PySpark) / Scala/Java, AWS/Azure/GCP.
  • Experience with MS SQL and NoSQL databases (HBase, Cassandra).
  • Ability to build Data Pipelines (ETL to consumption) using Apache Stack Tools, GCP, AWS and Azure.

Responsibilities

  • Write and review high quality code and support the client’s business use cases and technical requirements.
  • Translate requirements into technical design and develop mappings and data flows.
  • Lead the technical team and drive system design, performance tuning and architecture reviews.
  • Define guidelines for non-functional requirements and ensure best practices across projects.
  • Troubleshoot complex issues through systematic root cause analysis and resolution.

Skills

Big data
Spark
Python
Scala
Java
AWS
Azure
GCP
MS SQL
NoSQL
Data pipelines

Tools

HBase
Cassandra

Job description

Principal Engineer (Big Data Engineer)
About the job

Accelerate your career with PradeepIT PradeepIT is one of the largest, globally recognized IT Consulting firm to connect India's deeply vetted talent team to global customer. Were headquartered in Bengaluru, Silicon Valley of India. PradeepITs customers include SAP Lab, Bosch, Rolls-Royce, Daikin, Daimler and J&J and hundreds of other Fortune 500 companies and fast-growing startups. With continuous hard work and working remotely by choice, PradeepIT is certified as a Great Place to Work! Trusted by leading brands and fortune 500 companies from around the world, we have achieved: 6+ Years of Experience 580+ Open source technology Consultant 120+ SAP Consultant 40+ Salesforce Consultant 60+ Adobe Consultant 100+ Mobility Consultant 890+ Clients in APAC, EMEA & USA Our Beliefs PradeepIT believes in connecting people across the globe and provide them an opportunity work on remotely. Being a people-first organization, PradeepIT constantly strives for individuals who won't just keep up, but break new ground, work with cutting edge technology and ramp-up their skills with course created by our Vertical Heads, Senior Architect for freely with help of PradeepIT Academy.

Experience

Experience: 12 15 years

Requirements

Requirements: Should have a strong technical background with Big data, Spark, Python (PySpark) / Scala/Java, AWS/Azure/GCP ; Experience working with MS SQL and NoSQL databases (Hbase, Cassandra). Abilities to build Data Pipelines (from ETL to consumption) using Apache Stack Tools, GCP, AWS and Azure. Design and develop complex mappings, performance tuning of mappings, Process Flows Hands-on experience in System designing, Event sourcing, and message broker architecture Requirement gathering and analysis Define, understand, and analyze non-functional requirements for the project Lead the technical team Should be able to compare technologies to find the best fit for project requirements Should be able to troubleshoot complex or unusual bugs.

Responsibilities

Responsibilities: Writing and reviewing great quality code/support Understanding the client's business use cases and technical requirements and being able to convert them into a technical design that elegantly meets the requirements. Mapping decisions with requirements and being able to translate the same to developers. Identifying different solutions and being able to narrow down the best option that meets the client's requirements. Defining guidelines and benchmarks for NFR considerations during project implementation Writing and reviewing design documents explaining the overall architecture, framework, and high-level design of the application for the developers Review architecture and design on various aspects like extensibility, scalability, security, design patterns, user experience, NFRs, etc., and ensure that all relevant best practices are followed. Developing and designing the overall solution for defined functional and non-functional requirements; and defining technologies, patterns, and frameworks to materialize it Understanding and relating technology integration scenarios and applying these learnings in projects Resolving issues that are raised during code/review, through exhaustive systematic analysis of the root cause, and being able to justify the decision taken.

Experience Level

Senior Level

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Engineer (Big Data Engineer)
Principal Engineer (Big Data Engineer)

PradeepIt Global Consulting Pvt Ltd. • Bengaluru

Remote
INR 1,500,000 - 2,500,000
Flexible working hours
Continuous learning opportunities
Work with Fortune 500 clients
Data Engineer
Data Engineer

PradeepIt Global Consulting Pvt Ltd. • Bengaluru

Remote
INR 600,000 - 1,000,000
Lead Software Engineer (Big Data and Azure)
Lead Software Engineer (Big Data and Azure)

PradeepIt Global Consulting Pvt Ltd. • Bengaluru

On-site
INR 1,200,000 - 1,800,000
Lead Java Developer
Lead Java Developer

Pradeepit Global Consulting Private Limited • Bengaluru

On-site
INR 1,200,000 - 2,400,000
Python Backend Engineer
Python Backend Engineer

Pradeepit Global Consulting Private Limited • Bengaluru

Remote
INR 1,200,000 - 2,400,000
Big Data & GCP- Senior Software Engineer
Big Data & GCP- Senior Software Engineer

PradeepIt Global Consulting Pvt Ltd. • Bengaluru

On-site
INR 1,200,000 - 1,500,000
Lead Java Developer
Lead Java Developer

PradeepIt Global Consulting Pvt Ltd. • Bengaluru

Remote
INR 1,000,000 - 1,500,000
Site Lead
Site Lead

PradeepIt Global Consulting Pvt Ltd. • Bengaluru

Remote
INR 6,000,000 - 9,000,000
Technology Lead - Java
Technology Lead - Java

Pradeepit Global Consulting Private Limited • Bengaluru

On-site
INR 2,500,000 - 5,500,000
Software Development Engineer I
Software Development Engineer I

PradeepIt Global Consulting Pvt Ltd. • Bengaluru

Remote
INR 600,000 - 1,000,000