Data Engineer II: AI-Driven Pipelines & Insights

CoStar Group, Inc.; Arlington (VA); On-site; USD 103,000 - 153,000; Full time

Benefits offered by this job

Healthcare coverage

401(k) plan

Tuition reimbursement

Parking benefits

Employee stock purchase plan

Job summary

CoStar Group is seeking a Data Engineer II for Homes.com in Arlington, VA to design and implement scalable data architectures. You will develop data pipelines, manage data platforms, and deliver insights through KPI dashboards across multiple teams.

The role emphasizes building secure, scalable ETL processes, integrating AI-enabled tooling, and partnering with DBA/DevOps/SecOps to ensure data integrity. A strong background in Python, PySpark, SQL, and cloud data services is required.

Qualifications

• Bachelor's degree from an accredited college/university.
• 3+ years of hands-on software development experience.
• Strong proficiency in Python, PySpark, and SQL.
• Solid foundation in Object-Oriented Programming principles.
• Experience building data-driven products at terabyte scale.
• Experience designing and implementing enterprise-grade data platforms.
• Hands-on experience with cloud-based databases (relational and non-relational).
• Knowledge of BI software (Power BI or similar).
• Familiarity with AI technologies and tooling (LLMs, AI-assisted development).

Responsibilities

• Design and oversee dimensional modeling, database design, and cloud data platform structures (Databricks, Snowflake, BigQuery).
• Develop data platform framework for secure data warehouses and data lakes, building scalable ETL pipelines using Databricks, Snowflake, AWS.
• Implement and optimize Spark jobs and data transformations in Databricks.
• Build and deployed AI/ML models within data pipelines using Databricks ML and AWS ML.
• Define and apply AI-assisted software development practices to improve productivity and code quality.
• Collaborate with multiple teams to translate data domains into robust data models and pipelines.
• Oversee automated data infrastructure and streamline data tracking across products.
• Ensure data governance, lineage tracking, testing, and monitoring for data integrity.
• Mentor engineers and analysts in data engineering practices.
• Leverage AWS DevOps and CI/CD to automate deployment of data pipelines and infrastructure.

Skills

Python

PySpark

SQL

Data Modeling

Databricks

Snowflake

BigQuery

AWS

Power BI

AI tooling

ML

Education

Bachelor's degree

Tools

Databricks

Snowflake

BigQuery

AWS

DynamoDB

Kafka

Kubernetes

GitHub Copilot

Job description

CoStar Group is seeking a Data Engineer II for Homes.com in Arlington, VA to design and implement scalable data architectures. You will develop data pipelines, manage data platforms, and deliver insights through KPI dashboards across multiple teams.

The role emphasizes building secure, scalable ETL processes, integrating AI-enabled tooling, and partnering with DBA/DevOps/SecOps to ensure data integrity. A strong background in Python, PySpark, SQL, and cloud data services is required.