img
Permanent

Databricks Data Engineer

London
money-bag 75000.00-75000.00 Annual
3219934563
Posted Today

We are seeking an experienced Senior Databricks Data Engineer with extensive hands-on experience designing, developing and implementing enterprise-scale data engineering solutions using Azure Databricks. The successful candidate will have strong experience delivering complex data platforms within UK Public Sector environments, working with large datasets, sensitive information and secure cloud-based data architectures. Essential Technical Skills 5+ years of hands-on experience with Databricks. Expert-level knowledge of Azure Databricks. Advanced PySpark, Spark SQL and Python programming. Strong experience developing complex ETL/ELT pipelines. Deep understanding of Delta Lake, Delta Tables and Delta Live Tables / Lakeflow Declarative Pipelines. Experience implementing Medallion Architecture (Bronze, Silver and Gold). Strong knowledge of Apache Spark architecture, optimisation and performance tuning. Hands-on experience with Unity Catalog, access controls and data governance. Experience implementing incremental data loads, CDC and SCD Type 1/Type 2. Strong SQL development and query optimisation skills. Experience integrating Azure Data Factory with Databricks. Knowledge of Azure Data Lake Storage Gen2. Experience with CI/CD pipelines, Git, Azure DevOps and automated deployments. Experience implementing data quality, validation and reconciliation frameworks. Public Sector Experience Previous experience delivering data engineering projects within UK Government, Central ..... full job details .....

Other jobs of interest...

Adecco
South CroydonToday
money-bag£590/day Umbrella, PAYE, Holiday, Pensions Scheme
Qualient Technology Solutions UK Limited
LondonToday
money-bag10000-500000 Annual
Computappoint
LondonToday
money-bag40000.00-40000.00 Annual
AFBI - Agri-Food and Biosciences Institute
CityToday
money-bag37694.00-37694.00 Annual
Fdo Consulting Limited
MarlowToday
money-bag60000.00-60000.00 Annual
CBSbutler Holdings Limited
CityToday
money-bag80000.00-80000.00 Annual
GIS Recruiter Ltd
LondonYesterday
money-bag£75000 - £95000/annum Hybrid Working
CBSbutler Holdings Limited
City of LondonYesterday
money-bag£80,000
Fdo Consulting Limited
MarlowYesterday
money-bag£60,000
Intelligent Talent Solutions Ltd
City Of WestminsterYesterday
money-bag65000.00-65000.00 Annual

Perform a fresh search...

  • Create your ideal job search criteria by
    completing our quick and simple form and
    receive daily job alerts tailored to you!

Jobs. Straight to your inbox!