Senior Data Engineer

Dikshatek; United States; A distancia; USD 28.000 - 44.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Competitive salary

13th-month salary guarantee

Performance bonus

Premium health insurance

Extensive annual leave

Descripción de la vacante

GFT is seeking a Senior Data Engineer with 8+ years of experience to design, maintain, and optimize data infrastructure and pipelines. You will work with data science and product teams to implement robust data models and scalable architectures.

The role emphasizes batch and real-time processing, Airflow workflows, and cloud services. You will ensure data quality, performance, and reliability while pursuing continuous improvement of the data platform.

Formación

• Minimum of 8 years of experience in data engineering.
• Experience building data lakes, pipelines, and metadata repositories.
• Collaborate with data science and product teams on data models.
• Experience with AWS services like Glue, EMR, EC2, S3, Lambda.
• CI/CD knowledge with Jenkins, CircleCI, or CodePipeline.
• Strong testing and quality assurance practices.

Responsabilidades

• Design, maintain, and optimize data infrastructure and pipelines.
• Collaborate with data science teams on schemas and models.
• Develop batch and real-time data processing systems.
• Use Airflow and AWS Batch for workflow management.
• Ensure reliability with tests and monitoring.
• Drive continuous improvement of data platforms.

Conocimientos

Unix environments

Cloud computing

Python (pandas, pyspark)

Git

Apache Airflow

Herramientas

Athena

Redshift

Vertica

Hive/Hadoop

AWS

Docker

Kubernetes

Descripción del empleo

Location: Vietnam

Role Summary:
We are seeking a highly motivated Senior Data Engineer with minimum of 8 years of experience to join our dynamic data engineering team. As a Senior Data Engineer at GFT, you will play a pivotal role in designing, maintaining, and enhancing various analytical and operational services and infrastructure crucial for the organization's functions. You'll collaborate closely with cross-functional teams to ensure the seamless flow of data for critical decision-making processes.

Key Activities:

• Data Infrastructure Design and Maintenance: Architect, maintain, and enhance analytical and operational services and infrastructure, including data lakes, databases, data pipelines, and metadata repositories, to ensure accurate and timely delivery of actionable insights.
• Collaboration: Work closely with data science teams to design and implement data schemas and models, integrate new data sources with product teams, and collaborate with other data engineers to implement cutting-edge technologies in the data space.
• Data Processing: Develop and optimize large-scale batch and real-time data processing systems to support the organization's growth and improvement initiatives.
• Workflow Management: Utilize workflow scheduling and monitoring tools like Apache Airflow and AWS Batch to ensure efficient data processing and management.
• Quality Assurance: Implement robust testing strategies to ensure the reliability and usability of data processing systems.
• Continuous Improvement: Stay abreast of emerging technologies and best practices in data engineering, and propose and implement optimizations to enhance development efficiency.

Required Skills:

• Technical Expertise: Proficient in Unix environments, distributed and cloud computing, Python frameworks (e.g., pandas, pyspark), version control systems (e.g., git), and workflow scheduling tools (e.g., Apache Airflow).
• Database Proficiency: Experience with columnar and big data databases like Athena, Redshift, Vertica, and Hive/Hadoop.
• Cloud Services: Familiarity with AWS or other cloud services like Glue, EMR, EC2, S3, Lambda, etc.
• Containerization: Experience with container management and orchestration tools like Docker, ECS, and Kubernetes.
• CI/CD: Knowledge of CI/CD tools such as Jenkins, CircleCI, or AWS CodePipeline.

Nice-to-have requirements:

• Programming Languages: Familiarity with JVM languages like Java or Scala.
• Database Technologies: Experience with RDBMS (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., DynamoDB, Redis).
• BI Tools: Exposure to enterprise BI tools like Tableau, Looker, or PowerBI.
• Data Science Environments: Understanding of data science environments like AWS Sagemaker or Databricks.
• Monitoring and Logging: Knowledge of log ingestion and monitoring tools like ELK stack or Datadog.
• Data Privacy and Security: Understanding of data privacy and security tools and concepts.
• Messaging Systems: Familiarity with distributed messaging and event streaming systems like Kafka or RabbitMQ.

What can we offer you?

• Competitive salary
• 13th-month salary guarantee
• Performance bonus
• Professional English course for employees
• Premium health insurance
• Extensive annual leave