Databricks Engineer

Position Overview

At RELI Group, our work is grounded in purpose. We partner with government agencies to solve complex challenges, improve public health, strengthen national security, and make government services more effective and efficient. Our team of over 500 professionals brings deep expertise and a shared commitment to delivering meaningful outcomes. RELI Group is seeking a highly skilled Data Engineer to support the Centers for Medicare & Medicaid Services (CMS) Multidimensional Information Data Analytics System (MIDAS) Program. This role will play a critical part in the modernization of the MIDAS data platform, building and maintaining scalable, high-performing data pipelines that drive enterprise analytics, reporting, and operational decision-making. The Data Engineer will focus on developing data ingestion, transformation, and processing pipelines across AWS and Databricks-based environments, with an emphasis on Delta Lake lakehouse architectures.

Key Responsibilities

• Design, build, and maintain robust ETL/ELT pipelines that ingest, transform, and curate data for analytics and reporting solutions.
• Develop efficient data processing workflows using Python, SQL, and PySpark within Databricks and Delta Lake environments.
• Implement scalable data lakehouse architectures supporting complex healthcare datasets and enterprise analytics.
• Collaborate with Data Architects, Automation Test Engineers, QA Analysts, and Business Analysts to define data transformation logic and data integration standards.
• Build reusable code and frameworks for data ingestion, data quality validation, and exception handling.
• Optimize data processing jobs for performance, reliability, scalability, and cost efficiency in AWS and Databricks cloud platforms.
• Participate in schema design, data modeling, and version-controlled data pipeline development.
• Contribute to data governance, metadata management, and data lineage documentation to support audit, compliance, and FISMA reporting requirements.
• Partner with DevOps teams to integrate data pipelines into CI/CD workflows using GitHub, Databricks Repos, and related tools.
• Perform root cause analysis and resolution of data issues across staging, integration, and production environments.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related technical field.
• 6+ years of experience in data engineering or data pipeline development roles.
• Strong hands-on experience developing data pipelines using Python, SQL, and PySpark.
• Experience in Java is a plus to understand existing code and convert it to Python and Databricks Notebooks.
• Extensive experience with Databricks, including Databricks Workflows, Delta Lake, notebooks, and distributed computing.
• Proven experience with ETL/ELT design, development, and optimization for large-scale data processing.
• Solid understanding of data lakehouse architecture, data partitioning, and Delta Lake versioning.
• Experience with AWS data services such as S3, Redshift, Glue, IAM, and RDS.
• Familiarity with CI/CD processes for data pipelines (GitHub Actions, Jenkins, Databricks Repos).
• Experience with data quality validation, data profiling, and debugging data pipeline failures.
• Strong collaboration and communication skills with the ability to translate business needs into technical requirements.

Preferred Qualifications

• Experience supporting CMS programs and understanding of the CMS Technical Reference Architecture (TRA) and Target Lifecycle (TLC).
• Exposure to BI platforms such as QuickSight, Tableau, or Power BI for data consumption and reporting validation.
• Familiarity with Unity Catalog, data governance, and access control within Databricks.
• Experience supporting regulated environments requiring audit trails, FISMA audits, or CMS Acceptable Risk Safeguards (ARS 5.0).
• Exposure to healthcare datasets such as ACA, QHP, HICS, or HIOS.

Benefits & Perks

• Compensation: $100,000 to $150,000 annually. Actual compensation will depend on factors such as skills, experience, and qualifications.
• Additional benefits include a comprehensive benefits package. For details on the benefits offered, click here.