Data Engineer: Scalable NLP/ML Pipelines on Cloud

Andersen; Town of Poland (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Andersen, an international IT company, is seeking an experienced Data Engineer to join its team and work with a US company on new SaaS solutions that analyze data to solve enterprise problems.

The role focuses on building scalable pipelines, ETL for NLP/ML/DL projects, and collaborating with Data Scientists and ML Engineers to enable intelligent capabilities across products. English level is intermediate, with strong Python and cloud experience preferred.

Qualifications

• 3+ years of Data Engineer experience.
• Hands-on experience with Kafka for 1+ year.
• Experience with data formats JSON, Protocol Buffers, Avro.
• Experience with cloud platforms (AWS, GCP).
• Experience with data processing automation tools: Airflow, Luigi, Oozie, NiFi, Beam, Make.
• Experience with Linux Bash/CL tools.
• Strong coding and software engineering skills.
• 2+ years Python experience.
• English level: Intermediate.

Responsibilities

• Designing and building scalable data processing pipelines and ETL for NLP/ML/DL and other projects.
• Owning Data Engineering in Deep Learning, NLP, Data Capturing, Information Retrieval, and related areas.
• Collaborating with Data Scientists, ML Engineers, and developers to build intelligent capabilities for the company’s products.
• Creating robust data engineering design and best practices.

Skills

Data Engineering

Kafka

Python

AWS

GCP

JSON/Avro/Protobuf

ETL design

Linux Bash

English communication

Education

MS degree in CS/Data Science

Tools

Airflow

Luigi

Oozie

NiFi

Beam

Make

Job description

Andersen, an international IT company, is seeking an experienced Data Engineer to join its team and work with a US company on new SaaS solutions that analyze data to solve enterprise problems.

The role focuses on building scalable pipelines, ETL for NLP/ML/DL projects, and collaborating with Data Scientists and ML Engineers to enable intelligent capabilities across products. English level is intermediate, with strong Python and cloud experience preferred.