Technical Lead

Brooksource; Charlotte (NC); On-site; USD 140,000 - 200,000; Full time

Benefits offered by this job

401(k) Matching Plan

6 Paid Holidays

Regular engagement meetings

Job summary

Brooksource seeks a Technical Lead – Data Engineering to provide hands-on technical leadership across our data engineering platform. You will partner with architects and senior engineers to design scalable data solutions and drive best practices.

The role is highly hands-on with AWS Glue, PySpark, Kafka, DMS, Lambda, Redshift, and Aurora PostgreSQL, and you will mentor engineers while guiding platform reliability, performance, and data quality.

Qualifications

• 5+ years in data engineering or backend data platform work.
• 3+ years in a senior or lead role with architectural ownership.
• Senior experience with AWS Glue (PySpark), Python, and distributed processing.
• Experience with realtime streaming architectures and data modeling.

Responsibilities

• Provide hands-on technical leadership across the data engineering platform.
• Design scalable data solutions with architects and senior engineers.
• Lead ETL/ELT pipelines using AWS Glue and Python.
• Develop AWS Lambda orchestration and CDC/integration workflows.
• Ensure platform reliability, performance, and cost optimization.

Skills

Data Eng exp

Leadership

SQL

Data Modeling

ETL Pipelines

Streaming

Cloud AWS

Tools

AWS Glue

PySpark

Kafka

DMS

Lambda

Redshift

Aurora PostgreSQL

Spark

Job description

We are seeking a Technical Lead – Data Engineering to provide hands-on technical leadership across our data engineering platform. This role combines deep technical expertise with collaborative leadership, architectural stewardship, and delivery accountability.

In this role, you will partner closely with existing technical leadership within the Data & Analytics organization, including other technical leads, architects, and senior engineers, to design and evolve scalable data solutions. You will help drive consistency, share best practices, and collectively shape the technical direction of the platform.

The role remains highly hands-on, with significant contribution in AWS Glue (PySpark), Python, Kafka, AWS DMS, AWS Lambda, AWS Redshift, and Aurora PostgreSQL, while acting as a key technical voice within the broader data engineering leadership group.

Key Responsibilities

Technical Leadership & Collaboration

• Act as a hands-on technical leader for the Data Engineering area, collaborating with existing technical leads and architects to define and align on platform standards and architectural direction.
• Partner with peer technical leaders to ensure consistency across data ingestion, transformation, streaming, and consumption patterns.
• Contribute to shared technical roadmaps, design reviews, and architecture forums.
• Provide technical guidance and escalation support while respecting shared ownership and collective decision-making.

Architecture & Engineering Standards

• Help design, evolve, and maintain scalable, fault-tolerant architectures for batch and streaming data pipelines.
• Establish and reinforce engineering standards for:
• Pipeline design and reliability
• Code quality and testing
• Performance and cost optimization
• Security and operational best practices
• Drive alignment on ingestion strategies, CDC patterns, streaming vs. batch tradeoffs, and data modeling approaches.
• Lead development of complex ETL/ELT pipelines using AWS Glue (PySpark) and Python.
• Implement and tune AWS DMS pipelines for full-load and CDC ingestion.
• Develop AWS Lambda functions for orchestration, automation, monitoring, and event-driven workflows.
• Design and optimize schemas, queries, and performance for Amazon Aurora PostgreSQL.

Platform Reliability, Performance & Cost

• Share ownership of platform reliability, scalability, and performance with other technical leaders.
• Identify and resolve bottlenecks in Spark jobs, streaming consumers, and database workloads.
• Ensure robust error handling, retries, idempotency, and recovery strategies.
• Partner with cloud/platform teams on cost optimization, capacity planning, and infrastructure improvements.

Data Quality & Trust

• Partner closely with Data Quality Engineers and peer technical leaders to ensure quality validations are embedded throughout pipelines.
• Ensure pipelines support reconciliation, auditability, and observability.
• Enforce data freshness, completeness, and accuracy SLAs for downstream consumers, including Qlik.
• Mentor data engineers through code reviews, design sessions, and technical coaching.
• Collaborate with other technical leaders to raise the overall engineering maturity of the team.
• Contribute to hiring, onboarding, and skills development efforts.
• Promote knowledge sharing through documentation, demos, and internal forums.

Stakeholder & Cross-Team Engagement

• Work closely with analytics, BI, and business partners to translate requirements into scalable data solutions.
• Partner with BI teams to ensure curated datasets are analytics-ready for Qlik.
• Communicate technical decisions, risks, and tradeoffs clearly and collaboratively.

Required Qualifications

• 5+ years of experience in data engineering or backend data platform development.
• 3+ years in a senior or technical leadership role with shared architectural ownership.
• Senior-level experience with AWS Glue (PySpark), Python, and distributed data processing.
• Experience with Realtime streaming architectures.
• Advanced SQL skills and data warehousing practices to facilitate data analysis and design work
• Experience with serverless/event-driven architectures.
• Strong understanding of data modeling and batch vs. streaming design patterns.
• Demonstrated ability to collaborate effectively with peer technical leaders.
• Excellent communication and documentation skills.

Preferred Qualifications

• Familiarity with orchestration tools (Step Functions, Airflow).
• Knowledge of data governance, lineage, and cataloging practices.
• Experience optimizing Spark workloads for performance and cost.
• Experience in enterprise or regulated environments.
• Experience with RedShift

What Success Looks Like

• Strong partnership and alignment with existing technical leadership.
• Data engineering standards are well-defined, shared, and consistently applied.
• Pipelines are scalable, reliable, and meet performance and quality SLAs.
• Engineers are well-supported, mentored, and productive.
• Ramp up on product development and implementation within a 30-45 day timeframe

Why Join Us?

• 401(k) Matching Plan
• Relationship Driven Process to Find Your Best Fit
• 6 Paid Holidays
• Regular Meetings to Ensure Quality in Your Engagement

About Us:

At Brooksource, relationships are the foundation of everything we do. Since 2000, we’ve built lasting partnerships with clients, consultants, and internal teams to deliver an exceptional experience across every engagement. As a trusted IT and Engineering services provider, Brooksource supports Fortune 500 organizations through Experience-Driven Staffing, Professional Services, and Elevate, our proprietary Workforce Transformation program. Whether you're hiring for software development, cloud computing, cybersecurity, data analytics, or enterprise IT, our customized staffing solutions are designed to align with your company’s unique goals, culture, and technology stack. We offer flexible hiring models, including contract, contract-to-hire, and direct placement to meet your evolving business needs.

We are a certified partner of leading platforms, including Salesforce, AWS, Microsoft, and Google Cloud, enabling us to deliver scalable, end-to-end technology solutions. With a growing national footprint, Brooksource is redefining expectations in IT consulting, engineering services, and technology workforce solutions.

Brooksource is a top consulting and professional services provider specializing in IT and Engineering solutions for Fortune 500 companies and the public sector. It utilizes a national network of over 25,000 consultants, local and national delivery engines, as well as strategic partnerships with industry-leading technology providers to support its customers’ high-impact initiatives.

EEO Statement:

Brooksource is an equal opportunity employer that does not discriminate on the basis of actual or perceived race, color, creed, religion, national origin, ancestry, citizenship status, age, sex or gender (including pregnancy, childbirth, lactation and related medical conditions), gender identity or gender expression, sexual orientation, marital status, military service and veteran status, physical or mental disability, protected medical condition as defined by applicable state or local law, genetic information, or any other characteristic protected by applicable federal, state, or local laws and ordinances.

Benefits & Perks: Brooksource offers competitive medical, dental, vision, Health Savings Account, Dependent Care FSA, and supplemental coverage with plans that can fit each employee’s needs. We offer a 401k plan that includes a company match and is fully vested after you become eligible, paid time off, sick time, and paid company holidays. We also offer an Employee Assistance Program (EAP) that provides services like virtual counseling, financial services, legal services, life coaching, etc.

Pay Disclaimer:

The pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.