Senior Data Engineer

Cloudndata; United States; Hybrid; USD 140,000 - 180,000; Full time

Job summary

Cloudndata is seeking world-class talent to join our data engineering team in the United States. You will design and implement data warehouse architectures, ETL/ELT processes, and analytics tooling across modern cloud environments.

Proficiency in Python, Scala or Java, Spark, Hive, Delta Lake, and Kafka is expected, with strong SQL skills and a passion for scalable, real-time data solutions. This role emphasizes collaboration, innovation, and robust CI/CD practices.

Qualifications

• Bachelor’s degree or higher in a technical field or equivalent education/experience.
• 5+ years in data warehouse architectures, ETL/ELT and analytics tools.
• 3+ years in Big Data stacks (Hadoop, Spark, Hive, Delta Lake).
• 3+ years with multiple file formats and APIs (Parquet, Avro, Delta Lake).
• Experience with AWS cloud environment and serverless tech.
• Strong SQL in relational and NoSQL contexts.
• Proficient in Python, Scala and Java.
• Experience building realtime streaming data pipelines and using Kafka.
• Familiar with CI/CD and GitHub workflows.

Responsibilities

• Build and maintain data pipelines with Lambda/Kappa architecture.
• Develop streaming solutions for real-time analytics.
• Collaborate with teams to solve complex data problems.
• Leverage CI/CD and source control for reliable deployments.

Skills

Big Data

ETL/ELT

Python

Scala

Java

SQL

CI/CD

GitHub

Kafka

Streaming pipelines

Lambda/Serverless

Delta Lake

Spark

Hive

NoSQL

AWS

APIs

Data structures & algorithms

Education

Bachelor’s degree in Computer Science or related

Tools

Hadoop

Spark

Hive

Delta Lake

AWS

Kafka

Job description

We are looking for world class talent that brings the following key skills and experience to this role:

• Bachelor’s degree or higher or Combination of relevant education, experience and training.
• 5+ years of relevant experience with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.
• MS/BS in Computer Science, or related technical subject area
• 3+ years’ experience in Big Data stack environments (Hadoop, SPARK, Hive & Delta Lake)
• 3+ year’s working with multiple file formats (Parque, Avro, Delta Lake) & API
• 3+ year’s experience in cloud environments like AWS (Serverless technologies like AWS Lambda, API Gateway, NoSQL like Dynamo, EMR & S3)
• Experience with relational and non relational SQL
• Strong experience in coding languages like Python, Scala & Java
• Have experience in building realtime streaming data pipelines
• Experience in pub/sub modes like Kafka
• Strong understanding of data structures and algorithms
• Experience in building lambda, kappa, microservice and batch architecture
• Experience working on CI/CD processes and source control tools such as GitHub and related dev processes.
• Has a passion for data solutions and willing to pick up new programming languages, technologies, and frameworks
• Proven track record to partner with teams in solving complex problems by taking a broad perspective to identify innovative solutions
• Effective communication skills (with team members, the business, and in code)