Senior Data Engineer: Build & Scale Data Platforms

Thoughtworks; United States; On-site; USD 130,000 - 190,000; Full time

Job summary

Thoughtworks seeks Senior Data Engineer to design and operate data architectures and pipelines for client data platforms, using modern big data tools and data mesh approaches to enable scalable, reliable solutions. You will develop data models, ensure data quality, governance and security, and collaborate with data scientists and product teams.

The role requires hands-on coding in Python, Spark, and cloud environments (AWS/Azure/GCP), with Databricks experience preferred, and a passion for

Qualifications

• Hands-on experience with Python, Spark, and big data processing.
• Experience with at least one cloud platform (AWS, Azure, or GCP).
• Databricks experience is preferred.
• Ability to build and operate data pipelines in distributed systems.
• Experience with data modeling and modern data engineering tools.
• Experience writing clean, high-quality code.
• Experience with data-driven approaches and data governance and privacy practices.
• Experience with different types of databases (SQL, NoSQL, data lake).

Responsibilities

• Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions.
• Develop intricate data processing pipelines addressing clients' most challenging problems.
• Collaborate with data scientists to design scalable implementations of models.
• Write clean, iterative code using TDD and continuous delivery practices to deploy and operate data pipelines.
• Use various distributed storage and computing technologies from a wide range of options.
• Develop data models by selecting techniques and implementing in the chosen tech stack.
• Collaborate on data governance, data security and data privacy areas.
• Incorporate data quality into day-to-day work.

Skills

Python

Spark

Cloud platform

Databricks

Data modeling

Data pipelines

Distributed systems

Data visualization

Data security & privacy

Job description

Thoughtworks seeks Senior Data Engineer to design and operate data architectures and pipelines for client data platforms, using modern big data tools and data mesh approaches to enable scalable, reliable solutions. You will develop data models, ensure data quality, governance and security, and collaborate with data scientists and product teams.

The role requires hands-on coding in Python, Spark, and cloud environments (AWS/Azure/GCP), with Databricks experience preferred, and a passion for