AI Data Engineer (US)

AVP VIGILANT TECHNOLOGY PVT LTD; San Francisco (CA); On-site; USD 115,000 - 195,000; Full time

Benefits offered by this job

Health, dental, and vision benefits

401(k) retirement benefits

Paid time off and holidays

Professional development opportunities

Flexible/hybrid work options

Job summary

AVP Vigilant Technology Pvt Ltd is seeking an AI Data Engineer to design and build scalable data pipelines, cloud-based data platforms, and high-quality datasets for modern AI applications. Strong Python and SQL skills and hands-on cloud experience are required.

You will work with Data Scientists, ML Engineers, Software Engineers, and business teams to deliver reliable data infrastructure, ETL processes, data warehouses, and analytics-ready datasets across multiple locations including San

Qualifications

• 2–7 years of experience in Data Engineering, AI/ML Engineering, or a related field.
• Strong hands-on experience with Python and SQL.
• Solid understanding of data engineering, data modeling, ETL/ELT, and data warehousing.
• Working knowledge of machine learning concepts and AI data workflows.
• Experience with at least one major cloud platform: AWS, Azure, or Google Cloud.
• Experience working with relational and analytical databases.
• Strong understanding of data integration, APIs, and distributed data processing.
• Ability to work effectively in a collaborative, fast-paced environment.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT and data pipelines.
• Build and optimize data infrastructure supporting AI/ML applications.
• Develop reliable data processing solutions using Python and SQL.
• Integrate data from databases, APIs, applications, and other sources.
• Design data models, data warehouses, and analytics-ready datasets.
• Prepare, transform, validate, and optimize datasets for machine learning.
• Collaborate with Data Scientists, ML Engineers, Software Engineers, and business teams.
• Implement data quality, monitoring, governance, and security best practices.
• Optimize data pipelines and cloud infrastructure for performance, scalability, and reliability.
• Support ML model training, deployment, and production data workflows.
• Troubleshoot data and pipeline issues and continuously improve system performance.

Skills

Python

SQL

Data modeling

ETL/ELT

Data warehousing

APIs

Distributed data processing

Tools

AWS

Azure

Google Cloud

Apache Spark

Databricks

Airflow

Kafka

Snowflake

BigQuery

Redshift

Docker

Kubernetes

CI/CD

Job description

AI Data Engineer

Experience: 2–7 Years

Location: San Francisco, CA | New York, NY | Seattle, WA | Dallas, TX

Salary: $115,000–$195,000+ per year

Job Type: Full-Time

About The Role

We are looking for an experienced AI Data Engineer to build scalable data pipelines, cloud-based data platforms, and high-quality datasets that power modern AI and machine learning applications. The ideal candidate has strong expertise in Python and SQL, solid data engineering fundamentals, and hands‑on experience with cloud technologies and ML workflows.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT and data pipelines.
• Build and optimize data infrastructure supporting AI/ML applications.
• Develop reliable data processing solutions using Python and SQL.
• Integrate data from databases, APIs, applications, and other sources.
• Design data models, data warehouses, and analytics‑ready datasets.
• Prepare, transform, validate, and optimize datasets for machine learning.
• Collaborate with Data Scientists, ML Engineers, Software Engineers, and business teams.
• Implement data quality, monitoring, governance, and security best practices.
• Optimize data pipelines and cloud infrastructure for performance, scalability, and reliability.
• Support ML model training, deployment, and production data workflows.
• Troubleshoot data and pipeline issues and continuously improve system performance.

Required Qualifications

• 2–7 years of experience in Data Engineering, AI/ML Engineering, or a related field.
• Strong hands‑on experience with Python and SQL.
• Solid understanding of data engineering, data modeling, ETL/ELT, and data warehousing.
• Working knowledge of machine learning concepts and AI data workflows.
• Experience with at least one major cloud platform: AWS, Azure, or Google Cloud.
• Experience working with relational and analytical databases.
• Strong understanding of data integration, APIs, and distributed data processing.
• Ability to work effectively in a collaborative, fast‑paced environment.

Preferred Qualifications

• Experience with Apache Spark, Databricks, Airflow, Kafka, or similar technologies.
• Experience with Snowflake, BigQuery, Redshift, or other modern data warehouses.
• Knowledge of GenAI, LLM, RAG, vector databases, or AI application pipelines.
• Experience with Docker, Kubernetes, CI/CD, and cloud infrastructure.
• Familiarity with MLOps and production machine learning systems.
• Experience building highly scalable and reliable cloud data platforms.

What We Offer

• Competitive salary: $115K–$195K+
• Health, dental, and vision benefits
• 401(k) and retirement benefits
• Paid time off and holidays
• Professional development opportunities
• Flexible/hybrid work options, depending on role and location
• Opportunity to work with modern AI, ML, data, and cloud technologies

Skills: cloud,data engineering,ml,python