Pyspark Developer
PySpark + Fabric Developer (Contract) London Office-Based Location: London (Office-based) Contract: 6 months (potential extension) Start: ASAP Rate: Market rate - Inside IR35 We re looking for experienced PySpark + Fabric Developers to join a major transformation programme with a leading global financial data and infrastructure organisation. This is an exciting opportunity to work on cutting-edge data engineering solutions, driving innovation and performance at scale. Key Responsibilities Design, build, and optimise data pipelines for both batch and streaming workloads. Develop and manage dataflows and semantic models to support analytics and reporting. Implement complex data transformations, aggregations, and joins with a focus on performance and reliability. Apply robust data validation, cleansing, and profiling techniques to maintain accuracy. Enforce role-based access, data masking, and compliance standards. Tune and optimise workloads to minimise latency and enhance throughput. Collaborate with analysts and stakeholders to translate business needs into technical solutions. Maintain clear documentation and contribute to internal best practices. Requirements Strong hands-on experience with PySpark (RDDs, DataFrames, Spark SQL). Proven ability to build and optimise ETL pipelines and dataflows. Familiar with Microsoft Fabric or similar lakehouse/data platform environments. Experience with Git, CI/CD pipelines, and automated deployment. Knowledge of market data, transactio
Perform a fresh search...
-
Create your ideal job search criteria by
completing our quick and simple form and
receive daily job alerts tailored to you!