Data Ops/Engineer (with Capital markets exp)

Luxoft; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

A leading company seeks an experienced Data Engineer to develop a scalable data collection platform. The role involves web scraping, enhancing data pipelines, and building a greenfield data platform on Snowflake and AWS. Ideal candidates will have extensive experience in data engineering, strong skills in SQL and Python, and a willingness to work in the EU time zone.

Qualifications

• 10+ years of experience as a Data Engineer.
• Proficiency in SQL and Python.
• Experience with AWS and strong DevOps skills.

Responsibilities

• Web scraping using scripts, APIs, and tools.
• Help build and maintain a greenfield data platform on Snowflake and AWS.
• Onboard new data providers and manage data migration projects.

Skills

SQL

Python

Linux

Containerization (Docker, Kubernetes)

Communication

AWS

DevOps (K8s, Docker, Jenkins)

Job description

Project description

Develop a scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics.

Responsibilities

• Web scraping using scripts, APIs, and tools.
• Help build and maintain a greenfield data platform running on Snowflake and AWS.
• Understand existing pipelines and enhance them for new requirements.
• Onboard new data providers.
• Data migration projects.

Skills

Must have

• 10+ years of experience as a Data Engineer.
• Proficiency in SQL.
• Proficiency in Python.
• Experience with Linux.
• Containerization skills (Docker, Kubernetes).
• Good communication skills.
• Experience with AWS.
• Strong DevOps skills (K8s, Docker, Jenkins).
• Willing to work in the EU time zone.

Nice to have

• Market Data Projects / Capital markets experience.
• Snowflake expertise.
• Experience with Airflow.