Data Engineer II

Jobtailor; Bengaluru; On-site; INR 1,200,000 - 1,900,000; Full time

Job summary

Jobtailor in Bengaluru, India is seeking a data engineer to design and maintain scalable ELT pipelines on GCP, collaborating with Finance and analytics teams to deliver reliable datasets for reporting.

You will implement data workflows with BigQuery, Dataform, Airflow, Terraform, and Looker integration, ensure performance, cost efficiency, and document runbooks for cross‑functional teams.

Qualifications

• 3–5 years of hands-on data engineering experience with modern cloud data platforms.
• Strong understanding of transactional vs analytical data warehouses.
• Proficiency with Python and SQL for data pipelines and analytics.
• Experience building ETL/ELT jobs and maintaining data quality at scale.
• Hands-on with GCP services: BigQuery, Dataform, Pub/Sub, Cloud Run Functions.

Responsibilities

• Collaborate with Finance to translate business requirements into data sets and dashboards.
• Design, develop, and maintain scalable ELT pipelines in GCP for structured/unstructured data.
• Build BI/data workflows using Looker or similar tools; support ML models.
• Manage IaC with Terraform and CI/CD with GitLab across pipelines.
• Maintain production jobs and month-end data loads; ensure reliability and cost efficiency.

Skills

GCP proficiency

Data engineering

Python

SQL

ETL/ELT

Looker/BI

Airflow/Cloud Composer

Terraform

GitLab/CI-CD

API data ingestion

Data visualization

AI/automation tooling

Education

Bachelor's degree in CS/CE or related

Master's degree (beneficial)

Tools

BigQuery

Dataform

Cloud Run Functions

Workflows

Pub/Sub

Airflow/Cloud Composer

Terraform

GitLab CI/CD

Looker

Google Data Studio

Cloud Storage

Job description

Responsibilities

• Collaborate with stakeholders across the Finance team to understand business requirements and translate them into well‑defined, actionable data sets for analysis and reporting.
• Design, develop, and maintain scalable data extraction and ELT pipelines in Google Cloud Platform (GCP) to process structured and unstructured data from diverse sources including databases, REST/SOAP APIs, and cloud storage systems.
• Leverage a suite of GCP services such as Big Query, Dataform, Cloud Run Functions, Workflows, Airflow, Pub/Sub, and Cloud Storage to build efficient, secure, and high‑performing data workflows.
• Manage infrastructure as code using Terraform and maintain code and CI/CD pipelines in GitLab following branching, review, and deployment best practices.
• Build, test, and maintain AI agents and internal AI tooling to automate repetitive data engineering and finance workflows and improve productivity across the team.
• Continuously monitor and optimize data pipelines for performance, scalability, reliability, and cost‑efficiency.
• Manage and monitor scheduled production jobs (Daily, Weekly, Bi‑Weekly, and Monthly), ensuring timely and accurate data processing across all cycles during PST business hours.
• Provide production support during critical month‑end data load windows (Day 1 to Day 5), ensuring data availability and resolving issues swiftly to meet business reporting deadlines.
• Maintain clear and thorough technical documentation and runbooks for data pipelines, workflows, integrations, and system architecture to support ongoing development and cross‑functional collaboration.
• Support Looker dashboards and ML models that serve finance and business stakeholders.
• Identify opportunities for improvement in existing applications and workflows, recommending and implementing scalable solutions that enhance system functionality and user experience.
• Provide insights by collecting, analyzing, and summarizing data‑related development and operational issues to support troubleshooting and continuous improvement.
• Manage multiple tasks and projects simultaneously, effectively prioritizing work across the full lifecycle of data engineering initiatives.
• Respond to and fulfill ad‑hoc data requests from business stakeholders, delivering timely and accurate datasets or insights to support decision‑making and operational needs.

Requirements

• Bachelor's and/or master’s degree in computer science, Computer Engineering, or a related technical discipline.
• 3–5 years of hands‑on data engineering experience, with a strong understanding of the architectural differences between transactional systems and analytical data warehouses.
• Hands‑on experience with Google Cloud Platform (GCP) services, including BigQuery, Dataform, Cloud Run Functions, Workflows, and Pub/Sub.
• Strong experience reading data from REST and SOAP APIs and building data applications using Python (JavaScript and Java a plus).
• Proven expertise in building, deploying, and maintaining data pipelines using tools such as Apache Airflow / Cloud Composer and Dataform.
• Experience managing infrastructure as code with Terraform.
• Proficiency with GitLab, CI/CD tools and methodologies.
• Experience designing and writing efficient ETL/ELT jobs to ingest and transform data into Google Cloud Storage and BigQuery, ensuring scalability, performance, and data integrity.
• Advanced SQL proficiency, with the ability to write, optimize, and manage complex queries in high‑volume data environments.
• Strong skills in data visualization, with experience creating dashboards using Looker, Google Data Studio or similar BI tools; exposure to ML models is a plus.
• Experience building or working with AI agents / AI‑powered automation tools to improve productivity is highly desirable.
• Experience supporting scheduled production jobs during critical finance month‑end data load cycles, ensuring data accuracy, timeliness, and system reliability throughout the Day 1 to Day 5 close period.
• Availability to work in the PST time zone to support production jobs and ad‑hoc requirements.
• Solid understanding of cloud architecture design principles, including performance and cost optimization.
• Exceptional attention to detail, with a commitment to delivering high‑quality, reliable work.
• Self‑starter with the ability to thrive in unstructured environments, manage ambiguity, and independently drive initiatives forward.
• GCP certifications (e.g., Professional Cloud Developer, Professional Cloud Database Engineer) are a strong plus.