Data Engineer

Westcon-Comstor; Daerah Khusus Ibukota Jakarta; On-site; USD 40,000 - 90,000; Full time

Job summary

An established industry player is seeking a skilled Data Engineer to design and implement scalable data pipelines in cloud environments. This role involves collaborating with cross-functional teams to meet client needs and developing data models for structured and unstructured sources. The ideal candidate will have a strong background in data engineering, proficiency in programming languages like Python and Java, and experience with cloud services. Join a dynamic team focused on innovation and excellence in technology distribution, where your contributions will drive impactful solutions for clients.

Qualifications

• 2-5 years of experience in data engineering with a focus on cloud environments.
• Hands-on experience with cloud data and analytics services.

Responsibilities

• Design and maintain scalable data pipelines and ETL processes in AWS.
• Collaborate with teams to develop solutions that meet technical and business objectives.

Skills

Python

SQL/RDBMS

Git/Version Control

AWS

Agile/Scrum

Data skills (granularity, meaning, standardization)

Notebooks

Usage of GenAI in daily tasks

Client-facing skills (demos, workshops)

Education

Bachelor's or Master's degree in Computer Science

Tools

AWS Glue

Azure Data Factory

Google Cloud Dataflow

PySpark

Databricks

Snowflake

CloudFormation

Terraform

Data visualization tools (e.g., Quicksight, Superset, Tableau, Power BI)

Job description

Responsibilities:

1. Design, implement, and maintain scalable data pipelines and ETL processes in cloud environments (AWS) to support client business needs.
2. Collaborate with cross-functional teams to develop solutions that meet both technical and business objectives.
3. Work with clients to understand data requirements, existing systems, and opportunities for modernisation.
4. Develop and maintain data models, schemas, databases, and data lakes for structured and unstructured data sources.
5. Integrate data with GenAI solutions, including implementing architectures such as RAG in SageMaker.
6. Troubleshoot and resolve data quality, consistency, and availability issues promptly.
7. Stay updated on emerging technologies, tools, and trends in cloud computing and data engineering to foster continuous improvement and innovation.

Qualifications / Requirements:

1. Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
2. 2-5 years of experience in data engineering, focusing on building and optimizing data pipelines in cloud environments.
3. Proficiency in Python, Java, or Scala, with experience in SQL and NoSQL databases (e.g., Amazon RDS, DynamoDB).
4. Hands-on experience with cloud data and analytics services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow).
5. Strong understanding of data modeling, data warehousing, and distributed computing systems.
6. Practical experience delivering ML/AI use cases.
7. Excellent problem-solving skills, attention to detail, and ability to work independently and collaboratively.
8. Effective communication skills to articulate technical concepts to non-technical stakeholders.
9. Self-motivated learner eager to advance their career.
10. Experience with agile methodologies such as Scrum or Kanban.

Skills must have:

• Python
• SQL/RDBMS
• Git/Version Control
• AWS
• Agile/Scrum
• Data skills (granularity, meaning, standardization)
• Notebooks
• Usage of GenAI in daily tasks
• Client-facing skills (demos, workshops)

Nice to have:

• IAC (e.g., CloudFormation or Terraform)
• Cloud Data Engineering (e.g., PySpark, Databricks, Snowflake)
• Linux
• Cloud Security
• Documentation
• AWS Data Engineer Associate Certification
• Data visualization tools (e.g., Quicksight, Superset, Tableau, Power BI)
• Compute services (EC2, Lambda, Batch)
• Data formats (JSON, XML, flat files)
• Data lake/mesh architectures
• Kinesis/Streaming experience

About Us:

Westcon-Comstor is a global technology distributor specializing in Cloud, Deployment, Security, UC, Networking, and Data Center solutions. With teams in over 110 offices across more than 70 countries, we are committed to transforming technology distribution and delivering exceptional partner experiences.