Lead Azure Databricks Platform Engineer/Architect Hands-on Platform Engineering | Serverless | FinOps | POSIT/RStudio Migration6 Month contractInside IR35 - 500 a dayLondon/HybridRole PurposeWe are seeking a highly experienced, hands-on Azure Databricks Platform Engineer/Architect to enhance and optimise an enterprise Data Platform. The role combines architecture with direct implementation: the successful candidate must be able to configure, develop, troubleshoot and optimise Azure Databricks rather than operate only at design or governance level. The role is centred on three outcomes: enabling and optimising Databricks Serverless, strengthening FinOps and platform controls, and enhancing the Databricks Discovery Zone to support workloads currently delivered through POSIT/RStudio.Key Responsibilities1. Databricks Serverless Enablement and OptimisationAssess existing workloads and determine suitability for Serverless, classic, job or interactive compute based on duration, utilisation, SLA, concurrency, performance and cost.Enable and configure Serverless for appropriate jobs, SQL workloads, notebooks, analytical processing and data pipelines.Establish workload-placement guidance, including when Serverless is not economical for predictable, heavy or continuously running workloads.Implement compute policies, autoscaling, quotas, budget controls and operational guardrails.Measure cost and performance outcomes, identify idle or oversized compute, and recommend optimisation ..... full job details .....