Senior Cloud Data Engineer — Python/Java, Kubernetes

Compunnel Inc.; Wakefield (MA); On-site; USD 120,000 - 180,000; Full time

Job summary

Compunnel, Inc. in the United States is seeking a Senior Software Engineer to design, develop, and support scalable cloud-native applications and data platforms using AWS, microservices, big data architectures, and modern data technologies.

The role requires strong backend skills in Python and Java and hands-on experience with Kubernetes to build reliable, high‑performance cloud solutions. You will implement REST APIs and event-driven patterns, design large-scale data processing pipelines with

Qualifications

• Strong hands-on experience with AWS cloud-native engineering.
• Experience with AWS S3, Lambda, API Gateway, and EventBridge.
• Experience building scalable cloud data platforms.
• Strong experience designing and developing microservices architectures.
• Strong experience developing REST APIs and event-driven applications.
• Hands-on experience with Apache Spark and PySpark.
• Experience developing large-scale data processing pipelines.
• Strong backend programming experience with Python and Java.
• Experience with data lakes, data warehousing, and modern lakehouse architectures.
• Experience with Delta Lake or similar lakehouse technologies.
• Strong hands-on experience with Kubernetes; Kubernetes is critical for this role.
• Experience with containerized application development and deployment.
• Strong understanding of distributed systems, scalability, performance, and reliability.

Responsibilities

• Design, develop, and maintain scalable cloud-native applications and data platforms on AWS.
• Build and integrate solutions using AWS services including S3, Lambda, API Gateway, and EventBridge.
• Design and implement microservices-based architectures for scalable and resilient applications.
• Develop and maintain REST APIs and event-driven architectures.
• Design and implement large-scale data processing pipelines using Apache Spark and PySpark.
• Develop solutions for processing and transforming large volumes of data.
• Build and maintain data lakes, data warehouses, and modern lakehouse architectures.
• Implement and support Delta Lake or similar modern data platform technologies.
• Develop backend services and applications using Python and Java.
• Design and optimize distributed systems for scalability, performance, reliability, and availability.
• Containerize applications and deploy, manage, and troubleshoot workloads using Kubernetes.
• Collaborate with engineering and data teams to develop scalable and reliable cloud solutions.
• Troubleshoot application, data processing, infrastructure, and distributed system issues.
• Apply software engineering best practices across development, testing, deployment, and production support.

Skills

AWS cloud-native engineering

Backend development

Distributed systems

REST APIs

Microservices

Python

Java

Kubernetes

Data processing

Tools

AWS S3

Lambda

API Gateway

EventBridge

Apache Spark

PySpark

Delta Lake

Job description

Job Summary
The Senior Software Engineer will design, develop, and support scalable cloud-native applications and data platforms using AWS, distributed systems, microservices, big data technologies, and modern data architectures. The role requires strong backend programming skills in Python and Java, hands-on experience with Kubernetes, and the ability to build reliable, high-performance cloud solutions.

Key Responsibilities
• Design, develop, and maintain scalable cloud-native applications and data platforms on AWS.
• Build and integrate solutions using AWS services including S3, Lambda, API Gateway, and EventBridge.
• Design and implement microservices-based architectures for scalable and resilient applications.
• Develop and maintain REST APIs and event-driven architectures.
• Design and implement large-scale data processing pipelines using Apache Spark and PySpark.
• Develop solutions for processing and transforming large volumes of data.
• Build and maintain data lakes, data warehouses, and modern lakehouse architectures.
• Implement and support Delta Lake or similar modern data platform technologies.
• Develop backend services and applications using Python and Java.
• Design and optimize distributed systems for scalability, performance, reliability, and availability.
• Containerize applications and deploy, manage, and troubleshoot workloads using Kubernetes.
• Collaborate with engineering and data teams to develop scalable and reliable cloud solutions.
• Troubleshoot application, data processing, infrastructure, and distributed system issues.
• Apply software engineering best practices across development, testing, deployment, and production support.

Required Qualifications
• Strong hands-on experience with AWS cloud-native engineering.
• Experience with AWS S3, Lambda, API Gateway, and EventBridge.
• Experience building scalable cloud data platforms.
• Strong experience designing and developing microservices architectures.
• Strong experience developing REST APIs and event-driven applications.
• Hands-on experience with Apache Spark and PySpark.
• Experience developing large-scale data processing pipelines.
• Strong backend programming experience with Python and Java.
• Experience with data lakes, data warehousing, and modern lakehouse architectures.
• Experience with Delta Lake or similar lakehouse technologies.
• Strong hands-on experience with Kubernetes; Kubernetes is critical for this role.
• Experience with containerized application development and deployment.
• Strong understanding of distributed systems, scalability, performance, and reliability.