AI Data Engineer — Build Scalable Cloud Data Pipelines for ML

AVP VIGILANT TECHNOLOGY PVT LTD; New York (NY); On-site; USD 115,000 - 195,000; Full time

Benefits offered by this job

Competitive salary

Health benefits

401(k)

Paid time off

Professional development

Hybrid work

AI/ML cloud tech

Job summary

AVP Vigilant Technology Pvt Ltd. is seeking an AI Data Engineer to design, build, and optimize scalable data pipelines and cloud data platforms for AI/ML workloads. The role emphasizes Python and SQL expertise, data modeling, and ML data workflows.

The candidate will collaborate with data scientists, ML engineers, software engineers, and business teams to ensure data quality, governance, and secure data processing across environments.

Qualifications

• 2–7 years of experience in Data Engineering, AI/ML Engineering, or a related field.
• Strong hands-on experience with Python and SQL.
• Solid understanding of data engineering, data modeling, ETL/ELT, and data warehousing.
• Working knowledge of machine learning concepts and AI data workflows.
• Experience with at least one major cloud platform: AWS, Azure, or Google Cloud.
• Experience working with relational and analytical databases.
• Strong understanding of data integration, APIs, and distributed data processing.
• Ability to work effectively in a collaborative, fast-paced environment.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT and data pipelines.
• Build and optimize data infrastructure supporting AI/ML applications.
• Develop reliable data processing solutions using Python and SQL.
• Integrate data from databases, APIs, applications, and other sources.
• Design data models, data warehouses, and analytics-ready datasets.
• Prepare, transform, validate, and optimize datasets for machine learning.
• Collaborate with Data Scientists, ML Engineers, Software Engineers, and business teams.
• Implement data quality, monitoring, governance, and security best practices.
• Optimize data pipelines and cloud infrastructure for performance, scalability, and reliability.
• Support ML model training, deployment, and production data workflows.
• Troubleshoot data and pipeline issues and continuously improve system performance.

Skills

Python

SQL

Data Engineering

ETL/ELT

Data Warehousing

Cloud platforms

Tools

Apache Spark

Databricks

Airflow

Kafka

Snowflake

BigQuery

Redshift

Docker

Kubernetes

CI/CD

Job description

AVP Vigilant Technology Pvt Ltd. is seeking an AI Data Engineer to design, build, and optimize scalable data pipelines and cloud data platforms for AI/ML workloads. The role emphasizes Python and SQL expertise, data modeling, and ML data workflows.

The candidate will collaborate with data scientists, ML engineers, software engineers, and business teams to ensure data quality, governance, and secure data processing across environments.