Data Engineer III/ IV - IN (Operation/ Supports)
Work Location: Remote Work From Home
Shift: Rotation Shifts (24/7)
Experience: 7-12 years
Summary
The Data Engineer is responsible for managing and operating Databricks, Dbt, SSRS, SSIS, AWS DWS, AWS APP Flow, and PowerBI/Tableau. The engineer will work closely with the customer and team to manage and operate a cloud data platform.
Responsibilities
- Leads Level 4 operational coverage: Resolving pipeline issues, proactive monitoring for sensitive batches, RCA and retrospection of issues, and documenting defects.
- Design, build, test and deploy fixes to non‑production environment for Customer testing. Work with Customer to deploy fixes on production upon receiving Customer acceptance of fix.
- Cost/performance optimization and audit/security including any associated infrastructure changes.
- Troubleshooting incidents/problems, collecting logs, cross‑checking against known issues, and investigating common root causes (e.g., failed batches, infrastructure items such as connectivity to source, network issues).
- Knowledge Management: Create/update runbooks as needed.
- Governance: Monitor configuration changes to batches and infrastructure (cloud platform) and map them with proper documentation.
- Communication: Lead and act as a point of contact for the customer from off‑site, handling communication, escalation, issue isolation, and coordination with off‑site resources while setting expectations across stakeholders.
- Change Management: Align resources for on‑demand changes and coordinate with stakeholders as required.
- Request Management: Handle user requests—if the request is not runbook‑based, create a new knowledge base entry or update the runbook accordingly.
- Incident and problem management, root cause analysis, and proposing preventive measures such as enhancing monitoring or systematic changes as needed.
Skills
- Hands‑on experience with Databricks, Dbt, SSRS, SSIS, AWS DWS, AWS APP Flow, PowerBI/Tableau.
- Ability to read and write SQL and stored procedures.
- Experience in configuring, managing, and troubleshooting along with strong analytical and problem‑solving skills.
- Excellent written and verbal communication skills.
- Ability to communicate technical information clearly and promote inclusiveness in small groups.
Experience / Education
- Requires a Bachelor’s degree in computer science or a related field plus 10+ years of hands‑on experience in configuring and managing AWS/Tableau and Databricks solutions.
- Experience with Databricks and Tableau environments is desired.
Job Complexity
- Requires extensive problem‑solving skills and the ability to research an issue, determine its root cause, and implement a resolution.
- Researches various sources such as Databricks/AWS/Tableau documentation to identify and resolve issues.
- Must prioritize issues and multi‑task effectively.
Seniority level
Not Applicable
Employment type
Full‑time
Job function
Information Technology
Industries
IT Services and IT Consulting