Cloud Data Engineer II: Scalable ETL Pipelines

Brooksource

Dallas (TX)

On-site

USD 110,000 - 150,000

Full time

10 days ago

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

Brooksource seeks a Data Engineer II to help build and migrate data pipelines for large volumes of data in a cloud-first environment. You will contribute to ETL processing, scale data platforms on AWS and Snowflake, and collaborate with scientists, analysts, and stakeholders across divisions.

You will write maintainable Python code, optimize SQL, and participate in governance, security, and observability initiatives while mentoring junior engineers and contributing to infrastructure-as-code with

Qualifications

Bachelor’s degree in Computer Science, Engineering, Information Technology or related field.
2–4 years of experience in Data Engineering or a related role, with demonstrated success in delivering data solutions.
Proficient in SQL, with the ability to write complex queries, perform query optimization, and conduct performance tuning.
Experience with NoSQL databases, such as MongoDB, Cassandra, or DynamoDB, and an understanding of their appropriate use cases.
Strong programming skills in Python, Java, or Scala, with experience in data processing frameworks (e.g. Glue, Lambda, S3, dbt, Qlik, Apache Spark, Hadoop).
Experience with cloud platforms (AWS, Snowflake, Azure), and data services, such as Redshift, Data Sync, Appflow, Databricks.
Knowledge of big data technologies, including Hadoop, Spark, Kafka, and HBase, with experience in distributed data processing.
Familiarity with data orchestration tools, such as Apache Airflow or Luigi, for scheduling and managing data workflows.
Experience with data versioning and testing tools, such as DVC (Data Version Control) and dbt (data build tool).
Understanding of data security practices, including encryption, access controls, and data masking.
Proven ability to collaborate across technical and non-technical teams—communicating clearly, translating business requirements into engineering decisions, and building trust with stakeholders at multiple levels.

Responsibilities

Design, build, and maintain efficient and scalable data pipelines and ETL processing large volumes of structured and unstructured data.
Optimize data storage and retrieval methods to ensure performance, scalability, and cost-efficiency.
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
Implement data governance, data security, and compliance measures to protect sensitive data and ensure regulatory compliance.
Develop and manage data integrations between internal systems and third-party data providers.
Conduct performance tuning of SQL queries and data processing workflows to improve system efficiency.
Create and maintain comprehensive documentation for data pipelines, data models, and related processes.
Provide technical support and mentorship to junior data engineers, guiding them in best practices and problem-solving.
Write clean, modular Python for data ingestion, transformation logic, and orchestration—applying software engineering best practices including testing, versioning, and code review.
Build and maintain data quality frameworks, pipeline observability, and alerting systems that give the team confidence in production data across critical domains including pricing, supply chain, and sales operations.
Contribute to infrastructure-as-code practices using Terraform to provision and manage cloud resources in a repeatable, auditable way (experience a plus; willingness to learn required).