Databricks Engineer

Digerati Systems Inc.; Austin (TX); Hybrid; USD 140,000 - 200,000; Full time

Job summary

Digerati Systems Inc. is seeking a seasoned Databricks Engineer to design, develop, and optimize scalable data solutions. The role focuses on PySpark/Scala for large-scale processing and building robust ingestion pipelines using Delta Lake and Lakeflow.

You will implement data governance, quality controls, and security while delivering dashboards and applications within Databricks to support enterprise analytics and reporting. Remote and onsite work options apply.

Qualifications

• 8+ years of IT experience delivering technology solutions.
• Experience with Databricks and Apache Spark for large-scale data processing.
• Deep knowledge of data warehousing and dimensional models.
• Proficient in SQL and Python (or Scala) for analytics.
• Experience building dashboards and applications within Databricks.
• Experience implementing data governance, data quality, and security.
• Experience with Lakeflow Declarative Pipelines and Delta Live Tables.
• Experience with Delta Lake and medallion architecture.
• Nice to have experience with orchestration tools (e.g., Airflow).
• Excellent communication skills with technical and business stakeholders.

Responsibilities

• Design, develop, and optimize scalable data solutions on Databricks.
• Build and maintain ingestion pipelines and DL/DT pipelines.
• Develop data models and implement data quality and governance.
• Create dynamic dashboards and Databricks Apps for analytics.
• Ensure scalability, security, and cost efficiency of workloads.

Skills

Databricks

ETL/ELT pipelines

SQL

Python

Scala

Databricks dashboards

Data governance

Data quality

Data security

Lakeflow Declarative Pipelines

Delta Lake

Medallion architecture

Airflow or orchestration

Data lakehouse design

Communication with stakeholders

Job description

The working position is Hybrid - On Site and Telework.

Position will be 3 days remote with 2 days (Mondays and Thursdays) required to be onsite at the location listed above.

OTHER SPECIAL REQUIREMENTS

Program will only accept LOCAL ONLY candidates for this position. ***Days are subject to change per manager***

Interviews will be conducted: (check all that apply)

• Through Microsoft Teams

8 or more years of experience, relies on experience and judgment to plan and accomplish goals, independently performs a variety of complicated tasks, a wide degree of creativity and latitude is expected.

Understands business objectives and problems, identifies alternative solutions, performs studies and cost/benefit analysis of alternatives. Analyzes user requirements, procedures, and problems to automate processing or to improve existing computer system: Confers with personnel of organizational units involved to analyze current operational procedures, identify problems, and learn specific input and output requirements, such as forms of data input, how data is to be; summarized, and formats for reports. Writes detailed description of user needs, program functions, and steps required to develop or modify computer program. Reviews computer system capabilities, specifications, and scheduling limitations to determine if requested program or program change is possible within existing system.

The Databricks Engineer should design, develop, and optimize scalable data solutions on Databricks, leveraging PySpark or Scala for large-scale data processing. Build and maintain ingestion pipelines, Declarative Pipelines (DLT), and Medallion Architecture (Bronze, Silver, Gold) to support enterprise analytics and reporting. Develop robust data models and implement data quality, validation, and governance frameworks. Create dynamic dashboards, Databricks Apps, and analytical solutions to deliver actionable business insights. Optimize workloads, monitoring, and operational processes to ensure scalability, security, and cost efficiency.

II. CANDIDATE SKILLS AND QUALIFICATIONS

Candidates that do not meet or exceed the minimum stated requirements (skills/experience) will be displayed to customers but may not be chosen for this opportunity.

Years Required/Preferred Experience

• 8 Required Experience in IT, supporting the design, development, deployment, or delivery of technology solutions.
• 8 Required Experience with Databricks, including building and optimizing ETL/ELT data pipelines using Apache Spark.
• 8 Required Experience in data warehousing and dimensional data modeling (star/snowflake schemas).
• 8 Required Proficiency in SQL and Python (or Scala) for large-scale data processing.
• 8 Required Experience designing and developing dashboards and applications natively within Databricks (e.g., Databricks SQL dashboards, Databricks Apps).
• 8 Required Experience implementing data governance, data quality, and data security practices.
• 8 Required Experience implementing Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT) for building and managing production data pipelines.
• 8 Required Experience with Delta Lake, medallion architecture (bronze/silver/gold layers), data lakehouse design, and creating and scheduling offline jobs using Lakeflow Jobs (formerly Databricks Workflows) or similar orchestration tools (e.g., Airflow).
• 8 Required Excellent communication skills, both verbal and written, including presenting insights to technical and business stakeholders.
• 1 Preferred Experience working in public sector or state government environments.
• 1 Preferred Databricks certification (e.g., Databricks Certified Data Engineer Associate/Professional).
• 1 Preferred Experience with CI/CD practices for data pipelines (DevOps, Git-based workflows).

III. TERMS OF SERVICE

Services are expected to start 09/01/2026 and are expected to complete by 08/31/2027. Total estimated hours per Candidate shall not exceed 2008 hours.