Data Engineer 4: Python/PySpark/AWS — Cloud Pipelines

capitalone; McLean (VA); On-site; USD 197,000 - 225,000; Full time

Job summary

Capital One in McLean, VA is seeking an experienced Data Engineer to join a broad team delivering cloud-first data solutions. You will collaborate with agile teams to design, build, test, and support data pipelines and platforms using Python, Spark, and modern data warehousing tools.

You will mentor others, enforce data engineering design patterns, and communicate complex data concepts to stakeholders while driving scalable, secure, and observable data workloads across the enterprise.

Qualifications

• Bachelor's Degree or higher in Computer Science or a related quantitative field.
• At least 4 years of experience in application development.
• At least 2 years of experience in distributed data.
• At least 2 years of experience with SQL.
• At least 2 years of experience with one programming language (Python, Java, or Scala).
• At least 2 years of experience in data pipeline design and development.
• At least 1 year of experience in data modeling and designing end-to-end data solutions.

Responsibilities

• Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions.
• Influence a team of developers, data analysts and data scientists with experience in ML and lakehouse architecture.
• Utilize Python and Spark with relational and NoSQL databases and cloud data warehousing.
• Share passion for trends in data and mentor others in the data community.
• Collaborate with product managers and software engineers to deliver cloud-first data solutions.
• Independently design, build and deliver cloud data solutions with minimal supervision.
• Architect and enforce common data engineering design patterns for code quality and reusability.
• Serve as ambassador for the data engineering team, communicating concepts to stakeholders.
• Design and build data pipelines focusing on scalability, resilience, and performance.
• Implement data security standards and regulatory compliance.

Skills

Python

Spark

SQL

Databricks

Snowflake

Cloud

Education

Bachelor's degree in Computer Science or related quantitative field

Tools

Databricks

Snowflake

SQL

Python

Spark

AWS/Azure/GCP

Job description

Capital One in McLean, VA is seeking an experienced Data Engineer to join a broad team delivering cloud-first data solutions. You will collaborate with agile teams to design, build, test, and support data pipelines and platforms using Python, Spark, and modern data warehousing tools.

You will mentor others, enforce data engineering design patterns, and communicate complex data concepts to stakeholders while driving scalable, secure, and observable data workloads across the enterprise.