Data Engineering

Diensten Tech Ltd; United States; À distance; USD 120 000 - 150 000; Plein temps

Résumé du poste

Dienste Tech Ltd. is seeking a Data Engineer to design, build, and maintain scalable data infrastructure for large-scale data processing in the cloud.

You will collaborate with business and tech teams to capture requirements and deliver reliable data solutions. The role requires hands-on experience with GCP services (BigQuery, Dataflow, DataProc), SQL, Java, and Python, plus familiarity with data governance and microservices.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Information Systems, Data Analytics, or a related field.
• 4+ years of Data Engineering or Software Engineering experience, with cloud-based data platforms.
• Strong proficiency in SQL, Java, and Python, with cloud data pipelines on GCP (BigQuery, Dataflow, DataProc).
• SOA and microservices understanding within cloud data platforms.
• Experience with relational and NoSQL databases and data governance concepts.
• Familiarity with CI/CD, IaC tools (Terraform, Tekton).
• Excellent analytical and problem-solving skills for data platforms and microservices.

Responsabilités

• Collaborate with stakeholders to define data requirements.
• Design and build scalable data infrastructure for collection, storage, transformation, and analysis.
• Develop data pipelines, data models, and applications for reliable data workflows.
• Design and maintain data platforms like data warehouses and data lakes.
• Develop analytical tools and scripts to support data engineering tasks.
• Identify optimization opportunities and monitor cost and performance.

Connaissances

Python

BigQuery

AI

Airflow

GCP

Formation

Bachelor's Degree

Master's Degree

Outils

Terraform

Tekton

Description du poste

Position Description

Employees in this job function are responsible for designing, building, and maintaining data solutions including data infrastructure, pipelines, etc. for collecting, storing, processing and analyzing large volumes of data efficiently and accurately Key Responsibilities:

1) Collaborate with business and technology stakeholders to understand current and future data requirements

2) Design, build and maintain reliable, efficient and scalable data infrastructure for data collection, storage, transformation, and analysis

3) Plan, design, build and maintain scalable data solutions including data pipelines, data models, and applications for efficient and reliable data workflow

4) Design, implement and maintain existing and future data platforms like data warehouses, data lakes, data lakehouse etc. for structured and unstructured data 5) Design and develop analytical tools, algorithms, and programs to support data engineering activities like writing scripts and automating tasks 6) Ensure optimum performance and identify improvement opportunities

Skills Required:Python, Big Query, AI, AIRFLOW, GCPSkills Preferred:N/AExperience Required:Engineer 2 Exp: 4+ years Data Engineering work experienceExperience Preferred:0Education Required:Bachelor's DegreeEducation Preferred:Master's DegreeAdditional Safety Training/Licensing/Personal Protection Requirements:

Additional Information :Bachelor's degree in Computer Science, Information Technology, Information Systems, Data Analytics, or a related field (or equivalent combination of education and experience). - 4+ years of experience in Data Engineering or Software Engineering, with at least 2 years of hands‑on experience building and deploying cloud-based data platforms (GCP preferred). -Strong proficiency in SQL, Java, and Python, with practical experience in designing and deploying cloud-based data pipelines using GCP services like BigQuery, Dataflow, and DataProc. -Solid understanding of Service-Oriented Architecture (SOA) and microservices, and their application within a cloud data platform. - Experience with relational databases (e.g., PostgreSQL, MySQL), NoSQL databases, and columnar databases (e.g., BigQuery). - Knowledge of data governance frameworks, data encryption, and data masking techniques in cloud environments. - Familiarity with CI/CD pipelines, Infrastructure as Code (IaC) tools like Terraform and Tekton, and other automation frameworks. - Excellent analytical and problem-solving skills, with the ability to troubleshoot complex data platform and microservices issues. - Experience in monitoring and optimizing cost and compute resources for processes in GCP technologies (e.g., BigQuery, Dataflow, Cloud Run, DataProc).