Databricks Data Engineer - Senior Consultant

Position Overview

As a Senior Data Engineer, you will oversee the end-to-end design, deployment, and optimization of enterprise-scale data engineering solutions using Databricks on AWS, Azure, or GCP. In this highly strategic role you will lead innovation in big data architecture and analytics, shape best practices, advise senior stakeholders, and ensure data solutions align with business objectives and drive measurable results. Recruiting for this role ends on 2/13/2026.

Key Responsibilities

• Architect and Deliver Solutions: Lead the development, implementation, and scaling of advanced data engineering solutions using Databricks across AWS, Azure, or GCP environments.
• Champion Best Practices: Establish, document, and promote best-in-class approaches for data architecture, integration, and modelling.
• Pipeline Ownership: Oversee the design, development, and maintenance of robust data pipelines and architectures that support large-scale enterprise data needs.
• Drive Excellence: Initiate and manage efforts to improve data quality, operational efficiency, and process scalability.
• Technology Leadership: Evaluate, pilot, and integrate new big data and analytics technologies to keep the organization at the cutting edge.
• Strategic Data Governance: Consult on, design, and implement governance, security, and compliance strategies for modern cloud data ecosystems.
• Team Leadership and Mentoring: Lead, coach, and develop data engineering teams, fostering technical growth and effective project delivery.
• Stakeholder Engagement: Communicate technical concepts and business value to executives, business leads, and technology teams.
• DevOps and Automation: Oversee CI/CD practices with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell for streamlined deployments and operations.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field (Master's preferred).
• 5+ years of hands-on experience in data engineering with a strong focus on Databricks deployed on AWS, Azure, or GCP.
• Expertise with cloud-native databases, storage solutions, and distributed compute platforms.
• Deep understanding of Lakehouse architecture, Apache Spark, Delta Lake, and related big data technologies.
• Advanced skills in data warehousing, 3NF, dimensional modeling, and enterprise-level data lakes.
• Experience with Databricks components including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, and orchestration tools (e.g., Apache Airflow).
• Expertise in designing and supporting incremental data loads and building metadata-driven ingestion/data quality frameworks using PySpark.
• Hands-on experience with Databricks Unity Catalog and implementing fine-grained security and access control.
• Proven track record in deploying code and solutions via automated CI/CD pipelines.
• At least 1 year of leadership experience managing complex, cross-functional data projects and technical teams.
• Experience with performance optimization of data engineering pipelines, code, and compute resources.
• Ability to travel up to 50% based on project and client needs.
• Limited immigration sponsorship may be available.

Preferred Qualifications

• Comprehensive knowledge of AWS, Azure, and GCP cloud ecosystems and associated big data stacks.
• Demonstrated skill in performance tuning and optimization within Databricks/Apache Spark environments.
• Up-to-date with the latest Databricks feature releases and platform enhancements.
• Exceptional communication and stakeholder management abilities, including interfacing with executive leadership.
• Experience with Databricks Lakeflow is a plus.
• Experience in AI/ML is a plus.

Benefits & Perks

• Compensation: A wage range of $136,700 - $188,900, determined by skill sets, experience, certifications, and other organizational factors. (Note: It is not typical for an individual to be hired at or near the top of this range.)
• You may also be eligible to participate in a discretionary annual incentive program based on individual and organizational performance.
• For applicants requiring accommodation, please visit this link.