Databricks SME and AWS Data Engineer
Databricks SME and AWS Data Engineer Location: UK - Northampton (Hybrid)6 months UMBRELLA ONLY Project Objective: A key initiative involves migrating from Aerospike to Postgres and leveraging Databricks for back-testing fraud detection models. This role will contribute to the development and integration of Proof of Concepts (PoCs) from the detection backlog.Key Responsibilities:Collaborate with cross-functional teams to architect, design, develop and deliver PoCs related to fraud detection.Lead the ETL and Data manipulation/engineering work on AWS.Integrate Databricks-based back-testing into the fraud detection pipeline.Work closely with architects and other developers to ensure seamless integration with existing systems.Participate in weekly stand-up calls to demonstrate progress and align on deliverables.Take ownership of tasks from the backlog, ensuring timely and high-quality delivery. Required Skills and Experience:Strong AWS Data Engineering expertise.Proficiency in Kafka for real-time data streaming and integration.Proficiency with Databricks for data processing and analytics.Working knowledge of NodeJS would be an added advantage.Solid programming skills in Python, PySpark, Spark.Candidate must be adept at working with large-scale datasets, S3, Python, and data cataloguing tools. Familiarity with data engineering best practices is essential.Performance and Code Quality - candidate should demonstrate a strong commitment to building high-performance, scalable, and resil
Perform a fresh search...
-
Create your ideal job search criteria by
completing our quick and simple form and
receive daily job alerts tailored to you!