Senior Engineer

Tata Consultancy Services; Dallas (TX); On-site; USD 120,000 - 130,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage (Health

Dental & Vision

Disability Planning & Insurance

Commuter Benefits

Certification & Training Reimbursement

Vacation & Holidays

Job summary

Tata Consultancy Services in Dallas, TX is seeking a Senior ETL & AI/ML Test Automation Engineer to validate end-to-end data pipelines, ensure data quality, and test AI/ML integrations across enterprise platforms. The role covers Databricks testing, API validation, and robust test automation within CI/CD workflows.

You will write complex SQL, validate data lineage, and ensure model scoring and feature pipelines meet business outcomes.

Qualifications

• 6+ years of experience in ETL/Data Testing.
• 3+ years of experience testing Databricks or Spark-based platforms.
• Hands-on experience in AI/ML Model Testing or MLOps validation.
• Strong coding experience in Python and SQL.

Responsibilities

• Validate end-to-end ETL/ELT pipelines across enterprise data platforms.
• Perform Source-to-Target validation and reconciliation testing.
• Validate data transformations, aggregations, business rules, and lineage.
• Execute data quality checks on large-scale datasets.
• Write complex SQL queries for backend validation and data integrity testing.
• Test Databricks notebooks, workflows, Delta Live Tables, and Unity Catalog.
• Verify model scoring results and prediction outputs.
• Perform data drift and feature validation testing.
• Design and develop automation frameworks for ETL, API, and ML testing.
• Integrate automated tests into CI/CD pipelines.

Skills

ETL testing

Data testing

Databricks

Python

SQL

AI/ML testing

MLOps validation

CI/CD

APIs

Test automation

Education

Bachelor's degree

Tools

Databricks

Spark

MLflow

Azure ML

Job description

Senior ETL & AI/ML Test Automation Engineer

Must Have Technical/Functional Skills

ETL & Data Testing

• Validate end-to-end ETL/ELT pipelines across enterprise data platforms.
• Perform Source-to-Target validation and reconciliation testing.
• Validate data transformations, aggregations, business rules, and lineage.
• Execute data quality checks on large-scale structured and semi-structured datasets.
• Perform schema, metadata, and Delta Lake validations.
• Write complex SQL queries for backend validation and data integrity testing.
• Validate CDC (Change Data Capture) and incremental data loading processes.
• Test batch and streaming data pipelines.

Databricks Testing

• Validate Databricks notebooks, Workflows, Jobs, Delta Live Tables, and Unity Catalog implementations.
• Test Bronze, Silver, and Gold layer data transformations.
• Verify workflow orchestration and dependency handling across Databricks jobs.
• Perform performance validation of Spark and PySpark transformations.
• Validate data quality frameworks implemented in Databricks.

AI/ML Testing

• Validate AI/ML model integrations within enterprise applications.
• Verify model scoring results and prediction outputs.
• Perform data drift, schema drift, and feature validation testing.
• Validate feature engineering pipelines and training datasets.
• Execute model input/output verification against expected business outcomes.
• Validate explainability, traceability, and auditability requirements.
• Test model deployment workflows integrated through MLflow, Databricks ML, Azure ML, or similar platforms.
• Validate GenAI/RAG workflows, prompt-response accuracy, and grounding mechanisms where applicable.

Integration Testing

• Perform end-to-end integration testing across:
• Databricks
• Data warehouses
• APIs
• AI/ML services
• Cloud platforms
• Downstream reporting systems
• Validate data flow between source systems, data lakes, ML models, and consuming applications.
• Perform contract testing and API validation.

Test Automation

• Design and develop automation frameworks for:
• ETL testing
• API testing
• Data validation
• ML model validation
• Build reusable test utilities using Python.
• Integrate automated tests into CI/CD pipelines.
• Develop automated quality gates for deployment validation.
• Generate automated execution reports and test metrics.

Required

• 6+ years of experience in ETL/Data Testing.
• 3+ years of experience testing Databricks or Spark-based platforms.
• Hands-on experience in AI/ML Model Testing or MLOps validation.
• Strong coding experience in Python and SQL.

Education

Bachelor's degree

TCS Employee Benefits Summary

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range: $120000 - $130000 a year

Qualifications

BACHELOR OF COMPUTER SCIENCE