Principal Engineer (Big Data Engineer)

PradeepIT Consulting Services Pvt Ltd; Bengaluru; Remote; INR 1,500,000 - 2,500,000; Full time

Benefits offered by this job

Flexible working hours

Continuous learning opportunities

Work with Fortune 500 clients

Job summary

A leading IT consulting firm is seeking a Principal Engineer (Big Data Engineer) with 12-15 years of experience. The role involves developing data pipelines, system design, and leading technical teams using technologies like Spark, Python, and cloud platforms. Ideal candidates will have a strong background in both coding and architecture, and an ability to translate client needs into effective solutions. This is a great opportunity to work with a renowned firm and contribute to exciting projects.

Qualifications

• 12-15 years of experience in relevant technologies.
• Strong technical background in Big Data and cloud platforms.
• Hands-on experience in system designing and architecting solutions.

Responsibilities

• Writing and reviewing high-quality code.
• Understanding client requirements to design technical solutions.
• Leading the technical team and ensuring best practices.

Skills

Big data

Spark

Python (PySpark)

Scala

Java

AWS

Azure

GCP

MS SQL

NoSQL (Hbase, Cassandra)

Tools

Apache Stack Tools

Job description

About the job Principal Engineer (Big Data Engineer)

About the job

Accelerate your career with PradeepIT

PradeepIT is one of the largest, globally recognized IT Consulting firm to connect India's deeply vetted talent team to global customer.

Were headquartered in Bengaluru, Silicon Valley of India. PradeepITs customers include SAP Lab, Bosch, Rolls-Royce, Daikin, Daimler and J&J and hundreds of other Fortune 500 companies and fast-growing startups.

With continuous hard work and working remotely by choice, PradeepIT is certified as a Great Place to Work! Trusted by leading brands and fortune 500 companies from around the world, we have achieved:

6+ Years of Experience

580+ Open source technology Consultant

120+ SAP Consultant

100+ Mobility Consultant

890+ Clients in APAC, EMEA & USA

Our Beliefs

PradeepIT believes in connecting people across the globe and provide them an opportunity work on remotely. Being a people-first organization, PradeepIT constantly strives for individuals who won't just keep up, but break new ground, work with cutting edge technology and ramp-up their skills with course created by our Vertical Heads, Senior Architect for freely with help of PradeepIT Academy.

Experience: 12 15 years

Requirements:

• Should have a strong technical background with Big data, Spark, Python (PySpark) / Scala/Java, AWS/Azure/GCP;
• Experience working with MS SQL and NoSQL databases (Hbase, Cassandra).
• Abilities to build Data Pipelines (from ETL to consumption) using Apache Stack Tools, GCP, AWS and Azure.
• Design and develop complex mappings, performance tuning of mappings, Process Flows
• Hands-on experience in System designing, Event sourcing, and message broker architecture
• Requirement gathering and analysis
• Define, understand, and analyze non-functional requirements for the project
• Lead the technical team
• Should be able to compare technologies to find the best fit for project requirements
• Should be able to troubleshoot complex or unusual bugs

Responsibilities:

• Writing and reviewing great quality code/support
• Understanding the client's business use cases and technical requirements and being able to convert them into a technical design that elegantly meets the requirements.
• Mapping decisions with requirements and being able to translate the same to developers.
• Identifying different solutions and being able to narrow down the best option that meets the client's requirements.
• Defining guidelines and benchmarks for NFR considerations during project implementation
• Writing and reviewing design documents explaining the overall architecture, framework, and high-level design of the application for the developers
• Review architecture and design on various aspects like extensibility, scalability, security, design patterns, user experience, NFRs, etc., and ensure that all relevant best practices are followed.
• Developing and designing the overall solution for defined functional and non-functional requirements; and defining technologies, patterns, and frameworks to materialize it
• Understanding and relating technology integration scenarios and applying these learnings in projects
• Resolving issues that are raised during code/review, through exhaustive systematic analysis of the root cause, and being able to justify the decision taken.