Senior Data Engineer: Scale Data Pipelines & Architect Solutions

Atlassian; New York (NY); Hybrid; USD 167,000 - 219,000; Full time

Job summary

Atlassian is hiring a Senior Data Engineer to design premier data architecture and ELT/ETL pipelines, partnering with product and data scientists to translate business needs into scalable data solutions.

You will mentor engineers, own data quality and models, and collaborate with software teams to enable self-service data consumption across the organization. The role supports on-call rotation for platform reliability.

Qualifications

• BE in Computer Science or equivalent with 6+ years of professional experience as a Data Engineer or in a similar role.
• Experience building scalable data pipelines in Spark using Airflow scheduler/executor framework or similar scheduling tools.
• Experience with Databricks and its APIs.
• Experience with modern databases (Redshift, DynamoDB, MongoDB, PostgreSQL or similar) and data lakes.
• Proficient in one or more programming languages such as Python/Scala and rock-solid SQL skills.
• Champion automated builds and deployments using CICD tools like Bitbucket, Git
• Experience working with large-scale, high-performance data processing systems (batch and streaming)
• Experience working for SAAS companies
• Experience with Machine Learning
• Committed code to open source projects

Responsibilities

• Partner across engineering organizations to spearhead and execute high-impact, company-wide initiatives.
• Mentor and upskill junior engineers, fostering a culture of technical excellence and continuous growth.
• Collaborate with leadership, product engineers, program managers, and data scientists to deeply understand and prioritize business data requirements.
• Leverage deep technical expertise to design and architect scalable, high-performance data solutions.
• Develop, launch, and optimize highly efficient, reliable ELT/ETL pipelines capable of processing both large-scale and specialized datasets.
• Intelligently design robust data models optimized for high-speed storage, accessibility, and downstream retrieval.
• Implement and deploy rigorous data quality frameworks to guarantee data accuracy, freshness, and trust.
• Maintain full technical ownership of the end-to-end data engineering lifecycle for your team's solutions.
• Participate in the team’s on-call rotation to ensure platform stability and support operational health.
• Partner with software engineering teams to design and build next-generation data systems that enable rapid, self-service data consumption.

Skills

Python

Scala

SQL

Spark

Airflow

CI/CD

Team Leadership

Education

BE in Computer Science or equivalent

Tools

Apache Spark

Airflow

Databricks

Redshift

DynamoDB

MongoDB

PostgreSQL

Bitbucket

Git

Job description

Atlassian is hiring a Senior Data Engineer to design premier data architecture and ELT/ETL pipelines, partnering with product and data scientists to translate business needs into scalable data solutions.

You will mentor engineers, own data quality and models, and collaborate with software teams to enable self-service data consumption across the organization. The role supports on-call rotation for platform reliability.