Senior Data Engineer - AI & Data Pipelines (Hybrid NYC)

CloudIngest INC; New York (NY); Hybrid; USD 76,000 - 83,000; Part time

Job summary

CloudIngest INC is seeking an experienced Data Engineer to design and maintain reliable data pipelines on-site in Manhattan, NY, with a hybrid work arrangement (3 days in-office). You will collaborate with the Data Architect and AI team to deliver high-quality data for analytics and AI solutions.

The role demands 8–12 years of experience, strong SQL/Python, and hands-on work with Airflow/ADF, Spark, and cloud data services.

Qualifications

• 5+ years in data engineering (hands-on).
• Strong SQL and Python (or Scala) skills.
• Experience with Airflow or Azure Data Factory for orchestration.
• Hands-on experience with cloud data services and Spark.
• Understanding of data warehousing and lakehouse concepts.
• Familiarity with Git, CI/CD, and infrastructure-as-code.
• Experience supporting ML/AI data needs is a plus.

Responsibilities

• Design, build, and optimize batch and streaming data pipelines.
• Implement data models and transformations defined by the Data Architect.
• Ingest and integrate data from multiple source systems and APIs.
• Implement data quality checks, monitoring and alerting.
• Optimize performance and cost of data workloads.
• Collaborate with AI Developers and analysts to provision datasets.
• Maintain documentation, version control and CI/CD for data code.
• Work with offshore data engineers on knowledge transfer and delivery.

Skills

SQL

Python

Spark

Airflow

Azure Data Factory

CI/CD

Tools

Git

CI/CD

Job description

CloudIngest INC is seeking an experienced Data Engineer to design and maintain reliable data pipelines on-site in Manhattan, NY, with a hybrid work arrangement (3 days in-office). You will collaborate with the Data Architect and AI team to deliver high-quality data for analytics and AI solutions.

The role demands 8–12 years of experience, strong SQL/Python, and hands-on work with Airflow/ADF, Spark, and cloud data services.