Senior Data Engineer (AWS & Confluent Data/AI Projects) | Remote - PH

TASQ Staffing Solutions; Philippines; Remote; PHP 1,200,000 - 1,800,000; Full time

Job summary

A staffing solutions company seeks a Senior Data Engineer to architect and design scalable data solutions using AWS and Confluent. This remote role requires expertise in data streaming and cloud solutions, along with proficiency in programming languages like Python or Java. The ideal candidate will have 3 to 5 years of experience, strong SQL skills, and familiarity with CI/CD practices. Join us to lead innovative data projects in a flexible work environment.

Qualifications

• 3 to 5 years of experience in data engineering focusing on cloud solutions.
• Hands-on experience with Confluent Platform/Apache Kafka.
• Experience with CI/CD pipelines and DevOps practices.

Responsibilities

• Architect and design scalable data pipelines on AWS.
• Implement and optimize data streaming solutions using Confluent.
• Develop data solutions leveraging AWS services.

Skills

Data streaming expertise

AWS services

Programming languages (Python, PySpark, Scala, Java)

SQL proficiency

Data modeling concepts

Education

Bachelor's or master's degree in computer science or related field

Tools

Confluent Platform

Docker

Kubernetes

Job description

About the job Senior Data Engineer (AWS & Confluent Data/AI Projects) | Remote - PH

About the job: Senior Data Engineer (AWS & Confluent Data/AI Projects) | Remote

Work Set-up: Remote

Schedule: 10am-6pm SGT

Responsibilities:

• Architect and Design Data Solutions: Lead the design and architecture of scalable, secure, and efficient data pipelines for both batch and real-time data processing on AWS. This includes data ingestion, transformation, storage, and consumption layers.
• Confluent Kafka Expertise: Design, implement, and optimize highly performant and reliable data streaming solutions using Confluent Platform (Kafka, ksqlDB, Kafka Connect, Schema Registry). Ensure efficient data flow for real-time analytics and AI applications.
• AWS Cloud Native Development: Develop and deploy data solutions leveraging a wide range of AWS services, including but not limited to:
• Data Storage: S3 (Data Lake), RDS, DynamoDB, Redshift, Lake Formation.
• Data Processing: Glue, EMR (Spark), Lambda, Kinesis, MSK (for Kafka integration).
• Orchestration: AWS Step Functions, Airflow (on EC2 or MWAA)
• Analytics & ML: Athena, QuickSight, SageMaker (for MLOps integration).

Required Skills and Qualifications:

• Bachelor's or master's degree in computer science, software engineering, or a related quantitative field.
• 3 to 5 years of experience in data engineering, with a significant focus on cloud-based solutions.
• Extensive hands-on experience with Confluent Platform/Apache Kafka for building real-time data streaming applications.
• Proficiency in programming languages such as Python, PySpark, Scala, or Java.
• Expertise in SQL and experience with various database systems (relational and NoSQL).
• Solid understanding of data warehousing, data lakes, and data modeling concepts (star schema, snowflake schema, etc.).
• Experience with CI/CD pipelines and DevOps practices (Git, Terraform, Jenkins, Azure DevOps, or similar).
• AWS Certifications (e.g., AWS Certified Data Analytics - Specialty, AWS Certified)

Preferred Qualifications (Nice to Have):

• Solutions Architect - Associate/Professional).
• Experience with other streaming technologies (e.g., Flink).
• Knowledge of containerization technologies (Docker, Kubernetes).
• Familiarity with data mesh or data fabric concepts.
• Experience with data visualization tools (e.g., Tableau, Power BI, and QuickSight).
• Understanding of MLOps principles and tools.
• The candidate must have a working laptop