Remote Senior Data Engineer — Cloud Data Platforms
Unavailable
Consigue una respuesta de este empleador
Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.
Ventajas ofrecidas por este puesto de trabajo
Descripción de la vacante
Endava is seeking a Senior Data Engineer to operate and improve a Snowflake-based data platform in a regulated healthcare environment. You will own data pipelines end-to-end, ensure data quality, and collaborate with Cloud/DevOps and Analytics teams to deliver reliable, scalable data solutions.
You will design, build, and maintain reliable pipelines across RAW, CURATED, and CONSUMPTION layers, with dbt models, Airflow DAGs, and CI/CD practices.
Formación
- 5+ years of professional experience in data engineering or a related software engineering role.
- Strong hands-on experience with Snowflake and cloud data warehouses.
- Proficient in Python and SQL for data processing and automation.
- Experience with DBT, Airflow, and CI/CD practices.
- Familiarity with AWS services and data ingestion tools.
Responsabilidades
- Provide senior technical ownership for the Data Platform service tower.
- Operate and improve Snowflake production and non-production environments.
- Administer data access with least-privilege controls and audit readiness.
- Develop and maintain data ingestion pipelines (Fivetran/HVR as applicable).
Conocimientos
Herramientas
Descripción del empleo
Our data team has expertise across engineering, analysis, architecture, modeling, machine learning, artificial intelligence, and data science. This discipline is responsible for transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced analytics and predictive modeling.
As a Data Engineer at Endava you will:
- Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, validation, and delivery.
- Develop reusable, modular, and configuration-driven data processing solutions.
- Contribute to technical architecture and evaluate solutions based on scalability, performance, reliability, and maintainability.
- Design and optimize data models, SQL queries, stored procedures, and data processing workflows.
- Design and maintain Apache Airflow workflows, including scheduling, dependencies, retries, monitoring, and error handling.
- Develop automated data validation and quality checks using DBT and SQL .
- Implement appropriate monitoring, alerting, and observability for critical data pipelines.
- Troubleshoot and resolve complex data processing and quality issues.
- Develop data solutions using AWS and cloud-based data warehouse technologies such as Snowflake.
- Work with cloud services for data storage, processing, compute, messaging, and serverless workloads.
- Support data platforms across development, QA, staging, and production environments.
- Optimize data workloads for performance, scalability, reliability, and cost.
- Develop and maintain CI/CD pipelines for data engineering applications and infrastructure.
- Apply Infrastructure-as-Code practices using Terraform or equivalent technologies.
- Support production deployments, incident resolution, root cause analysis, and continuous improvement.
- Maintain technical documentation, operational runbooks, and troubleshooting guides.
- Take ownership of complex engineering initiatives from design through production.
- Conduct code reviews and promote engineering best practices.
- Mentor junior and mid-level engineers and provide technical guidance.
- Collaborate with architecture, DevOps, product, and other technology teams.
Job Description
Our data team has expertise across engineering, analysis, architecture, modeling, machine learning, artificial intelligence, and data science. This discipline is responsible for transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced analytics and predictive modeling.
As a Data Engineer at Endava you will:
- Data Engineering & Architecture
- Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, validation, and delivery.
- Develop reusable, modular, and configuration-driven data processing solutions.
- Contribute to technical architecture and evaluate solutions based on scalability, performance, reliability, and maintainability.
- Design and optimize data models, SQL queries, stored procedures, and data processing workflows.
- Workflow Orchestration & Data Quality
- Design and maintain Apache Airflow workflows, including scheduling, dependencies, retries, monitoring, and error handling.
- Develop automated data validation and quality checks using DBT and SQL .
- Implement appropriate monitoring, alerting, and observability for critical data pipelines.
- Troubleshoot and resolve complex data processing and quality issues.
- Cloud & Platform Engineering
- Develop data solutions using AWS and cloud-based data warehouse technologies such as Snowflake.
- Work with cloud services for data storage, processing, compute, messaging, and serverless workloads.
- Support data platforms across development, QA, staging, and production environments.
- Optimize data workloads for performance, scalability, reliability, and cost.
- CI/CD & Production Support
- Develop and maintain CI/CD pipelines for data engineering applications and infrastructure.
- Apply Infrastructure-as-Code practices using Terraform or equivalent technologies.
- Support production deployments, incident resolution, root cause analysis, and continuous improvement.
- Maintain technical documentation, operational runbooks, and troubleshooting guides.
- Technical Leadership
- Take ownership of complex engineering initiatives from design through production.
- Conduct code reviews and promote engineering best practices.
- Mentor junior and mid-level engineers and provide technical guidance.
- Collaborate with architecture, DevOps, product, and other technology teams.
- Programming: Python, SQL
- Orchestration: Apache Airflow or equivalent
- Data Transformation: DBT, SQL, PySpark
- Data Warehouse: Snowflake or equivalent cloud data warehouse
- Cloud: AWS or equivalent cloud platform
- AWS Services: S3, EMR, Lambda, SNS/SQS or equivalent
- CI/CD: Jenkins, GitHub, GitHub Actions, or equivalent
Mandatory Qualifications
- 5+ years of professional experience in data engineering or a related software engineering role.
- Strong hands-on experience building and supporting production-grade data pipelines.
- Strong knowledge of Apache Airflow, including DAG development, scheduling, dependencies, retries, and monitoring.
- Strong experience with Snowflake or equivalent cloud data warehouse technologies.
- Proven experience with DBT.
- Solid experience with AWS or another major cloud platform.
- Experience with CI/CD and Infrastructure-as-Code practices.
- Strong programming skills in Python and SQL.
- Demonstrated experience troubleshooting complex production issues and performing root cause analysis.
- Ability to independently own complex technical initiatives from design through implementation and production.
- Strong communication, problem-solving, and technical documentation skills.
Preferred Qualifications
- Experience with large-scale data processing.
- Experience with PySpark and EMR.
- Experience mentoring engineers and contributing to technical standards and architecture.
Discover some of the global benefits that empower our people to become the best version of themselves:
- Finance: Competitive salary package, value-based recognition awards, referral bonus;
- Career Development : Career coaching, global career opportunities, non-linear career paths, internal development programmes for management and technical leadership;
- Learning Opportunities: Complex projects, rotations, internal tech communities, training, certifications, coaching, online learning platforms subscriptions, pass-it-on sessions, workshops, conferences;
- Work-Life Balance: Hybrid work and flexible working hours, employee assistance programme;
- Health: Global internal wellbeing programme, access to wellbeing apps;
- Community: Global internal tech communities, hobby clubs and interest groups, inclusion and diversity programmes, events and celebrations.
At Endava, we’re committed to creating an open, inclusive, and respectful environment where everyone feels safe, valued, and empowered to be their best. We welcome applications from people of all backgrounds, experiences, and perspectives—because we know that inclusive teams help us deliver smarter, more innovative solutions for our customers. Hiring decisions are based on merit, skills, qualifications, and potential. If you need adjustments or support during the recruitment process, please let us know.
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
We are looking for a Senior Data Engineer to operate and improve a Snowflake-based enterprise data platform in a regulated healthcare environment.
- 5+ years of professional experience in Data Engineering or Data Platform Engineering.
- Strong hands-on experience operating and developing solutions on Snowflake , including data-layer design, warehouse performance, access patterns, and production troubleshooting.
- Advanced SQL skills and strong experience with dbt for transformation, testing, documentation, lineage, and controlled deployment.
- Experience operating managed ingestion tools such as Fivetran or HVR and supporting custom data-ingestion pipelines.
- Hands-on experience with AWS data services , particularly S3 and event-driven or file-based ingestion patterns.
- Experience orchestrating and troubleshooting data workloads with Argo Workflows on Kubernetes , or comparable workflow-orchestration technologies.
- Proficiency in Python or a comparable language for data engineering, automation, and operational tooling.
- Strong understanding of data modeling, pipeline dependencies, schema evolution, backfills, data validation, and production data quality.
- Experience with observability, logging, alerting, and incident-management practices for production data platforms.
- Demonstrated ability to lead complex incident resolution, perform root-cause analysis, and convert findings into preventive improvements.
- Ability to make well-reasoned technical decisions, identify tradeoffs, estimate work, and guide improvements across a complex platform.
- Experience mentoring engineers and collaborating effectively with Cloud / DevOps, Analytics, Security, Governance, and business stakeholders.
- Strong written and verbal English communication skills, with the ability to work directly with client stakeholders.
- Full availability to work from 9:00 AM to 6:00 PM Pacific Time and participate in an agreed on-call rotation.
- Hands-on experience with a data-specific observability platform.
- Experience supporting Tableau Cloud administration, data-source connectivity, extracts, scheduled refreshes, or production dashboard dependencies.
- Familiarity with Snowflake cost optimization, audit logging, tasks, stored procedures, and environment rationalization.
- Experience migrating legacy ingestion or orchestration patterns such as Boomi or AWS Data Pipeline to modern managed or Kubernetes-based solutions.
- Experience with Terraform, CI/CD pipelines, and Infrastructure as Code practices supporting data platforms.
- Experience with service-management and change-control tools such as Freshservice and Jira.
- Familiarity with HIPAA, GDPR, FDA-related controls, least-privilege access, separation of duties, and audit-ready operational practices.
- Provide senior technical ownership for the Data Platform service tower during the LatAm coverage window, including day-to-day operations, complex troubleshooting, and L2/L3 escalation.
- Operate and improve Snowflake production and non-production environments, including warehouse configuration and sizing, performance and consumption monitoring, object lifecycle, environment hygiene, and support for production changes.
- Administer data access within established controls, including users, roles, service accounts, secrets, and credential rotation, while maintaining least-privilege and audit-ready practices.
- Operate and improve data ingestion across Fivetran, HVR where applicable, and custom pipelines, including connector configuration, scheduling, source onboarding, schema-change coordination, failure recovery, backfills, and dependency management.
- Design, build, and maintain reliable pipelines and dbt models across RAW, CURATED, and CONSUMPTION layers, with appropriate testing, documentation, lineage, version control, and CI/CD practices.
- Support AWS S3 data-lake operations, including raw and landing-zone workflows, lifecycle and retention controls, access patterns, logging, ingestion failures, and coordination with downstream Snowflake workloads.
- Support Argo Workflows and Kubernetes-hosted data workloads in close coordination with the Cloud / DevOps team, including scheduling, troubleshooting, deployment, recovery, and capacity dependencies.
- Define and improve data quality and observability standards, including freshness, zero-row, row-count growth, null, duplicate, schema-drift, and referential-integrity checks.
- Expand end-to-end monitoring and lineage using tools such as SYNQ, dbt, Snowflake audit data, Splunk, and the agreed alerting stack, linking actionable alerts to evidence and runbooks.
- Lead or support major data incidents, root-cause analysis, post-incident reviews, and preventive actions across ingestion, orchestration, Snowflake, and downstream Tableau dependencies.
- Support Tableau Cloud operations where upstream data, connectivity, permissions, extracts, or refresh failures require Data Platform investigation.
- Identify and deliver standardization, automation, reliability, performance, and cost improvements, including migration of suitable legacy or custom extraction patterns toward agreed golden paths.
- Execute work through controlled incident, request, access, change, and release processes using established service-management workflows.
- Create and maintain runbooks, operating procedures, architecture context, ownership information, recovery procedures, and knowledge-transfer materials.
- Mentor Middle-level engineers, review technical work, improve team practices, and ensure effective handoffs across the distributed service team.
- Participate in the Data Platform on-call rotation for critical incidents outside staffed service hours.
- Professional growth : Mentorship, TechTalks, and personalized growth roadmaps.
- Competitive compensation : USD-based pay with education, fitness, and team activity budgets.
- Exciting projects : Modern solutions with Fortune 500 and top product companies.
- Flextime : Flexible schedule with remote and office options.
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
We are looking for a Senior Data Engineer to operate and improve a Snowflake-based enterprise data platform in a regulated healthcare environment.
- 5+ years of professional experience in Data Engineering or Data Platform Engineering.
- Strong hands-on experience operating and developing solutions on Snowflake , ...
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
