Senior Data Engineer

Rearc; Bengaluru; On-site; INR 3,000,000 - 4,500,000; Full time

Job summary

Rearc is seeking a Senior Data Engineer in Bengaluru to lead data architecture and build scalable data pipelines. You will work with cloud-native frameworks to deliver efficient, reliable data solutions and govern data management practices across teams.

Ideal candidates will have extensive Python experience, expertise in Databricks, Spark, and AWS-based data architectures, and a track record of mentoring data engineers in a fast-paced environment.

Qualifications

• 8+ years of experience in data engineering across architectures and use cases.
• Designing data warehouse and data lake architectures, especially in AWS environments.
• Strong Python experience for data engineering tasks and libraries.
• Proven experience with data pipelines using Airflow, Databricks, DBT or AWS Glue.
• Hands-on with data analysis tools like PySpark, NumPy, Pandas, or Dask.
• Proficiency with Spark and Databricks; experience with SQL and NoSQL databases.
• Deep knowledge of data architecture in cloud environments; AWS IaC tools (Terraform, CloudFormation, AWS CDK).
• Excellent communication, able to convey complex concepts to varied stakeholders.

Responsibilities

• Provide strategic data engineering leadership and vision linked to business objectives.
• Architect and implement data pipelines and scalable data architectures using modern tools.
• Drive innovation and adopt new data engineering technologies and methods.
• Apply expertise in ETL, data modelling, and warehousing to optimize workflows and quality.
• Mentor junior team members and collaborate with cross-functional teams.
• Contribute to thought leadership via technical writing and community engagement.

Skills

Data engineering

Data warehouse

Data lake

Python

Databricks

PySpark

Delta Lake

Airflow

DBT

AWS

Terraform

CloudFormation

AWS CDK

Spark

Kubernetes

SQL

NoSQL

PostgreSQL

Redshift

DynamoDB

Tools

Databricks

Airflow

DBT

AWS Glue

Terraform

CloudFormation

AWS CDK

Kafka

Kubernetes

Job description

At Rearc, we're committed to empowering engineers to build awesome products and experiences. Success as a business hinges on our people's ability to think freely, challenge the status quo, and speak up about alternative problem-solving approaches. If you're an engineer driven by the desire to solve problems and make a difference, you're in the right place!

Our approach is simple — empower engineers with the best tools possible to make an impact within their industry.

We're on the lookout for engineers who thrive on ownership and freedom, possessing not just technical prowess, but also exceptional leadership skills. Our ideal candidates are hands‑on leaders who don't just talk the talk but also walk the walk, designing and building solutions that push the boundaries of cloud computing.

As a Senior Data Engineer at Rearc, you will be at the forefront of driving technical excellence within our data engineering team. Your expertise in data architecture, cloud-native solutions, and modern data processing frameworks will be essential in designing workflows that are optimized for efficiency, scalability, and reliability. You'll leverage tools like Databricks, PySpark, and Delta Lake to deliver cutting‑edge data solutions that align with business objectives. Collaborating with cross‑functional teams, you will design and implement scalable architectures while adhering to best practices in data management and governance. Building strong relationships with both technical teams and stakeholders will be crucial as you lead data‑driven initiatives and ensure their seamless execution.

What You Bring

• 8+ years of experience in data engineering, showcasing expertise in diverse architectu res, technology stacks, and use cases.
• Strong expertise in designing and implementing data warehouse and data lake architectures, particularly in AWS environments.
• Extensive experience with Python for data engineering tasks, including familiarity with libraries and frameworks commonly used in Python-based data engineering workflows.
• Proven experience with data pipeline orchestration using platforms such as Airflow, Databricks, DBT or AWS Glue.
• Hands‑on experience with data analysis tools and libraries like Pyspark, NumPy, Pandas, or Dask.
• Proficiency with Spark and Databricks is highly desirable.
• Experience with SQL and NoSQL databases, including PostgreSQL, Amazon Redshift, Delta Lake, Iceberg and DynamoDB.
• In-depth knowledge of data architecture principles and best practices, especially in cloud environments.
• Proven experience with AWS services, including expertise in using AWS CLI, SDK, and Infrastructure as Code (IaC) tools such as Terraform, CloudFormation, or AWS CDK.
• Exceptional communication skills, capable of clearly articulating complex technical concepts to both technical and non‑technical stakeholders.
• Demonstrated ability to quickly adapt to new tasks and roles in a dynamic environment.

What You'll Do

• Strategic Data Engineering Leadership : Provide strategic vision and technical leadership in data engineering, guiding the development and execution of advanced data strategies that align with business objectives.
• Architect Data Solutions : Design and architect complex data pipelines and scalable architectures, leveraging advanced tools and frameworks (e.g., Apache Kafka, Kubernetes) to ensure optimal performance and reliability.
• Drive Innovation : Lead the exploration and adoption of new technologies and methodologies in data engineering, driving innovation and continuous improvement across data processes.
• Technical Expertise : Apply deep expertise in ETL processes, data modelling, and data warehousing to optimize data workflows and ensure data integrity and quality.
• Collaboration and Mentorship : Collaborate closely with cross‑functional teams to understand requirements and deliver impactful data solutions—mentor and coach junior team members, fostering their growth and development in data engineering practices.
• Thought Leadership : Contribute to thought leadership in the data engineering domain through technical articles, conference presentations, and participation in industry forums.

Some More About Us

Founded in 2016, we pride ourselves on fostering an environment where creativity flourishes, bureaucracy is non‑existent, and individuals are encouraged to challenge the status quo. We're not just a company; we're a community of problem‑solvers dedicated to improving the lives of fellow software engineers.

Our commitment is simple - finding the right fit for our team and cultivating a desire to make things better. If you're a cloud professional intrigued by our problem space and eager to make a difference, you've come to the right place. Join us, and let's solve problems together!