Senior Data Engineer

London
1 month ago
Applications closed

Related Jobs

View all jobs

Analytics Engineering Manager

Data Scientist | London | AI-Powered SaaS Company

Senior Network Engineer

Senior Network Engineer

Senior Software Engineer

Teleport Senior Engineer

Position: Senior Data Engineer

Salary: £55,000 - £65,00

Location: Hybrid - Bristol, Manchester, or London
Security Requirement: SC Clearance to start and be willing / able to obtain DV Clearance

We are seeking Data Engineers, with a keen interest / experience in AI /ML and strong proficiency in Python Scripting. To join our Consultancy client working on meaningful projects in the Defence and Security sector.

Required Experience:

End-to-End Data Development: Strong experience with data pipelines, ETL processes, and workflow orchestration, demonstrating best practices across tech stacks.
Strong Python Skills
Diverse Data Handling: Familiarity with batch, streaming, real-time, and unstructured data sources.
Architectural Design & Systems Thinking: Ability to design and build scalable, high-performance data solutions.
Data Modelling & Warehouse Design: Proficiency in data modelling, warehouse design, and database optimization, with examples of logical and physical models.
Distributed Data Systems: Experience in deploying, managing, and tuning distributed systems for optimal reliability and performance.
Coding & Development Practices: Demonstrated coding expertise with modular, reusable, and efficient code in various languages.
Development Lifecycle: Understanding of SDLC, CI/CD pipelines, and version control.
Data Governance & Security: Knowledge of data security, governance, metadata management, and master data principles.Key Responsibilities:

As a Senior Data Engineer, you'll bridge the gap between client needs and technical solutions, creating data pipelines that ingest, transform, and enrich large data volumes. You'll have client-facing responsibilities, delivering high-quality data solutions in multi-disciplinary teams across industries.

In this consulting role, your responsibilities will vary depending on client engagement focus and your skillset, but will often include:

Data Engineering & AI Integration: Apply data engineering tools, integration frameworks, and query engines to create high-quality, standardised data for AI applications and reporting.
Data Pipeline Development: Design and implement robust data pipelines and stores in collaboration with other engineers and developers.
Innovative Problem Solving: Bring fresh approaches to challenging data engineering problems.
Architecture for Scale: Design scalable, complex data architectures that provide cross-team value.
Data Modelling & Governance: Establish standards in logical and physical data modelling and data governance.
Distributed Computing: Employ parallel processing, streaming, and batch workflows to manage large data volumes effectively.
ETL & Workflow Automation: Build ETL processes and automated workflows for efficient data movement.
System Optimization: Tune data systems for performance, scalability, and monitoring.
Data Security: Apply best practices for information security, including encryption and data anonymity for sensitive data assets.
Data Governance & Quality: Manage metadata, data lineage, and data quality standards.If you're passionate about using data engineering and AI to solve complex problems in the Defence and Security sector, we'd love to hear from you!

People Source Consulting Ltd is acting as an Employment Agency in relation to this vacancy. People Source specialise in technology recruitment across niche markets including Information Technology, Digital TV, Digital Marketing, Project and Programme Management, SAP, Digital and Consumer Electronics, Air Traffic Management, Management Consultancy, Business Intelligence, Manufacturing, Telecoms, Public Sector, Healthcare, Finance and Oil & Gas

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Quantum-Enhanced AI at the Edge: The Future of Decentralised Intelligence

As the modern world pushes towards instant data processing and real-time analytics, edge computing has emerged as a compelling solution. Instead of funnelling every piece of data to centralised data centres or the cloud, edge computing brings computation closer to the data source—reducing latency, lowering bandwidth costs, and enabling on-the-spot decision-making. From IoT sensors in smart cities to autonomous vehicles and remote industrial sites, the edge has quickly become a linchpin of digital transformation. Simultaneously, Artificial Intelligence (AI) has shown explosive growth, driving breakthroughs in natural language processing, computer vision, and advanced analytics. Cloud-based AI solutions have served organisations well, but in scenarios demanding ultra-low latency or local autonomy, the cloud’s round-trip time becomes a bottleneck. Hence, edge AI—embedding AI models at or near the point of data collection—promises a new wave of hyper-responsive applications and decentralised intelligence. Yet, as we continue pushing the boundaries of data volume, complexity, and speed, even advanced edge solutions sometimes struggle with the exponential computational requirements of AI. This is where quantum computing enters the picture, potentially offering new methods to tackle intractable problems in optimisation, high-dimensional data analysis, and machine learning. While quantum hardware remains in its early stages, the prospect of integrating quantum algorithms into AI workflows at the edge is generating significant excitement. In this article, we’ll explore: The current state and challenges of edge computing. A concise overview of quantum computing and why it matters. The concept of quantum-enhanced AI—especially in distributed or decentralised environments. Potential real-world applications at the intersection of quantum, AI, and edge computing. Key job roles and skill sets emerging in this new frontier. Considerations around security, ethics, and hardware constraints as we move towards quantum solutions at the edge. If you’re a professional in edge computing, an AI enthusiast, or simply curious about what the future of decentralised tech might look like, read on. The fusion of quantum computing and AI at the network edge could redefine how we collect, process, and learn from data in real time.

Edge Computing Jobs at Newly Funded UK Start-ups: Q3 2025 Investment Tracker

Edge computing has emerged as a revolutionary paradigm for processing data closer to where it’s generated—think IoT devices, sensors in remote locations, autonomous vehicles, and more. By reducing latency and bandwidth usage, edge computing enables real-time insights and responsive applications. In the UK, a growing ecosystem of innovators is capitalising on edge technology, buoyed by increased venture capital, academic prowess, and government-backed programmes that stimulate tech development. In this Q3 2025 Investment Tracker, we’ll explore newly funded UK start-ups blazing a trail in edge computing. We’ll also highlight the wealth of job opportunities these investments create for software engineers, DevOps specialists, data scientists, and other tech professionals looking to carve out a career at the cutting edge—pun fully intended.

Portfolio Projects That Get You Hired for Edge Computing Jobs (With Real GitHub Examples)

Edge computing is transforming how data is collected, processed, and acted upon—often in real time and close to where data is generated. From Internet of Things (IoT) devices to 5G networks and industrial automation, edge computing unlocks new possibilities for low-latency analytics, intelligent decision-making, and resource optimisation. With the proliferation of edge devices and the need for distributed computing architectures, demand for skilled edge computing professionals continues to rise. If you want to stand out in this exciting field, you need more than a great CV: you need a well-curated portfolio demonstrating your hands-on capabilities. This guide will show you how to build that portfolio, including: Why a dedicated edge computing portfolio is crucial. How to choose projects aligned with your target edge roles. Real GitHub examples that illustrate best practices. Actionable project ideas for edge deployments and data processing. Tips on presenting your portfolio so recruiters and hiring managers see your value instantly. When you’re ready, don’t forget to upload your CV on EdgeComputingJobs.co.uk so potential employers can find your newly polished portfolio. Let’s dive in!