Data Engineer - 14-Week Contract (Outside IR35) Likely to ExtendStart Date: 12th January Rate: 350 per day Location: Remote (UK-based) Interview: Immediate - Offer before ChristmasWe are seeking an experienced Data Engineer to join a 14-week project focused on building robust data pipelines and integrating complex data sources. This is an outside IR35 engagement, offering flexibility and autonomy.Key ResponsibilitiesDesign and implement ETL/ELT pipelines with strong error handling and retry logic.Develop incremental data processing patterns for large-scale datasets.Work with AWS services including Glue, Step Functions, S3, DynamoDB, Redshift, Lambda, and EventBridge.Build and optimise vector database solutions and embedding generation pipelines for semantic search.Implement document processing workflows (PDF parsing, OCR, metadata extraction).Integrate data from REST APIs, PIM systems, and potentially SAP.Ensure data quality, governance, and lineage tracking throughout the project.Required SkillsETL/ELT pipeline design and data validation frameworks.Advanced Python (pandas, numpy, boto3) and SQL (complex queries, optimisation).Experience with AWS Glue, Step Functions, and event-driven architectures.Knowledge of vector databases, embeddings, and semantic search strategies.Familiarity with document parsing libraries (PyPDF2, pdfplumber, Textract) and OCR tools.Understanding of data governance, schema validation, and master data management.Strong grasp of real-time vs batch proc