Senior Data Engineer: Data Pipelines & Platform Architect

Referrals Only; Raleigh (NC); On-site; USD 120,000 - 180,000; Full time

Job summary

Thoughtworks in the United States seeks a Senior Data Engineer to design, build and operate data architectures for scalable data applications. You will own end-to-end data pipelines, collaborate with data scientists, and apply modern data engineering practices across distributed storage and processing platforms.

You will work with Python, SQL, ETL pipelines, Databricks and cloud services (AWS/Azure/GCP), emphasize data governance and security, and mentor teammates to uphold technical excellence

Qualifications

• Experience building and operating data pipelines.
• Hands-on data modeling with modern tools.
• Proficient in Python, SQL and ETL pipelines.
• Experience with cloud data platforms (AWS/Azure/GCP).
• Ability to communicate insights to stakeholders.

Responsibilities

• Develop and operate modern data architectures to meet business objectives and provide end-to-end data solutions.
• Develop intricate data processing pipelines addressing clients' most challenging problems.
• Collaborate with data scientists to design scalable models.
• Write clean, iterative code using TDD and continuous delivery practices to deploy, support and operate data pipelines.
• Use distributed storage and computing technologies from the plethora of options available.
• Develop data models by selecting from various modeling techniques and implementing the chosen data model.
• Collaborate on data governance, data security and data privacy.
• Incorporate data quality into your day-to-day work.

Skills

Data pipelines

Data modeling

Python

SQL

Databricks

Cloud platforms

Data warehousing

Big data

Tools

Databricks

Snowflake

Redshift

BigQuery

Job description

Thoughtworks in the United States seeks a Senior Data Engineer to design, build and operate data architectures for scalable data applications. You will own end-to-end data pipelines, collaborate with data scientists, and apply modern data engineering practices across distributed storage and processing platforms.

You will work with Python, SQL, ETL pipelines, Databricks and cloud services (AWS/Azure/GCP), emphasize data governance and security, and mentor teammates to uphold technical excellence