Batch Automation

Tata Consultancy Services; Irving (TX); On-site; USD 100,000 - 135,000; Full time

Benefits offered by this job

Restaurant d'entreprise

Indemnités de stage/alternance

Job summary

Tata Consultancy Services in Irving, TX seeks a data engineering professional to automate and optimize batch workflows with Shell scripting, manage secure file transfers with NDM and SFTP, and develop scalable PySpark ETL pipelines for large-scale data processing.

The role emphasizes automation via Ansible, infrastructure as code, and collaboration with cross-functional teams to ensure reliable, compliant data movement. Salary range $100,000-$135,000 annually; on-site position.

Qualifications

• Strong scripting and file-processing expertise for batch automation and workflow optimization.
• Experience developing scalable PySpark ETL pipelines for large-scale datasets.
• Experience managing secure enterprise file transfers using NDM and SFTP.
• Familiarity with infrastructure as code practices and Ansible.
• Collaborative work with cross-functional teams to ensure secure, reliable data movement.

Responsibilities

• Automate and optimize batch workflows using robust Shell scripting and file-processing techniques.
• Manage secure file transfers using NDM (Connect: Direct) and SFTP, including monitoring, troubleshooting, and operational support.
• Develop scalable PySpark ETL pipelines, ensuring high performance and reliability for large-scale data processing.
• Implement automation and configuration management using Ansible, following Infrastructure as best practices.
• Collaborate with cross-functional teams to ensure secure, reliable, and compliant data-movement operations.

Skills

Shell Scripting

PySpark

Operational Excellence

Education

Bachelor of Computer Science

Tools

NDM (Connect: Direct)

SFTP

Ansible

Job description

• Automate and optimize batch workflows using robust Shell scripting and file-processing techniques.
• Manage secure file transfers using NDM (Connect: Direct) and SFTP, including monitoring, troubleshooting, and operational support.
• Develop scalable PySpark ETL pipelines, ensuring high performance and reliability for large-scale data processing.
• Implement automation and configuration management using Ansible, following Infrastructure as best practices.
• Collaborate with cross-functional teams to ensure secure, reliable, and compliant data-movement operations.

Job Description

• Automate and optimize batch workflows using robust Shell scripting and file-processing techniques.
• Manage secure file transfers using NDM (Connect: Direct) and SFTP, including monitoring, troubleshooting, and operational support.
• Develop scalable PySpark ETL pipelines, ensuring high performance and reliability for large-scale data processing.
• Implement automation and configuration management using Ansible, following Infrastructure as best practices.
• Collaborate with cross-functional teams to ensure secure, reliable, and compliant data-movement operations.

Roles & Responsibilities

• Automate and optimize batch workflows using robust Shell scripting and file-processing techniques.
• Manage secure file transfers using NDM (Connect: Direct) and SFTP, including monitoring, troubleshooting, and operational support.
• Develop scalable PySpark ETL pipelines, ensuring high performance and reliability for large-scale data processing.
• Implement automation and configuration management using Ansible, following Infrastructure as best practices.
• Collaborate with cross-functional teams to ensure secure, reliable, and compliant data-movement operations.

Required Skills

Shell Scripting: Strong scripting and file-processing expertise for batch automation and workflow optimization.

PySpark: Hands-on experience developing scalable, high-performance, and reliable ETL pipelines for large-scale datasets.

NDM (Connect: Direct): Experience managing secure enterprise file transfers, monitoring transfer jobs, and troubleshooting failures.

SFTP: Strong knowledge of secure file-transfer operations, controls, monitoring, and issue resolution, Experience implementing automation and configuration management using Infrastructure as Code practices.

Operational Excellence: Ability to support secure, reliable, and compliant data movement in collaboration with cross-functional teams.

Salary Range: $100,000 to $135,000 per year

Qualifications:

BACHELOR OF COMPUTER SCIENCE