Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

PySpark Developer

London
1 week ago
Create job alert

We are looking for an experienced PySpark Developer with strong Microsoft Fabric and Azure engineering skills to join a major transformation programme within the financial-markets domain. This role is fully hands-on, focused on building and optimising large-scale data pipelines, dataflows, semantic models, and lakehouse components.
Key Responsibilities

  • Design, build and optimise Spark-based data pipelines for batch and streaming workloads
  • Develop Fabric dataflows, pipelines, and semantic models
  • Implement complex transformations, joins, aggregations and performance tuning
  • Build and optimise Delta Lake / delta tables
  • Develop secure data solutions including role-based access, data masking and compliance controls
  • Implement data validation, cleansing, profiling and documentation
  • Work closely with analysts and stakeholders to translate requirements into scalable technical solutions
  • Troubleshoot and improve reliability, latency and workload performance
    Essential Skills
  • Strong hands-on experience with PySpark, Spark SQL, Spark Streaming, DataFrames
  • Microsoft Fabric (Fabric Spark jobs, dataflows, pipelines, semantic models)
  • Azure: ADLS, cloud data engineering, notebooks
  • Python programming; Java exposure beneficial
  • Delta Lake / Delta table optimisation experience
  • Git / GitLab, CI/CD pipelines, DevOps practices
  • Strong troubleshooting and problem-solving ability
  • Experience with lakehouse architectures, ETL workflows, and distributed computing
  • Familiarity with time-series, market data, transactional data or risk metrics
    Nice to Have
  • Power BI dataset preparation
  • OneLake, Azure Data Lake, Kubernetes, Docker
  • Knowledge of financial regulations (GDPR, SOX)
    Details
  • Location: London (office-based)
  • Type: Contract
  • Duration: 6 months
  • Start: ASAP
  • Rate: Market rates
    If you are a PySpark / Fabric / Azure Data Engineer looking for a high-impact contract role, apply now for immediate consideration

Related Jobs

View all jobs

Pyspark Developer

Databricks Engineer - £400PD - Remote

AI Engineer

Data Scientist / Quant Engineer

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Edge Computing Recruitment Trends 2025 (UK): What Job Seekers Need To Know About Today’s Hiring Process

Summary: UK edge computing hiring has moved from tool‑lists to capability‑driven assessments that emphasise resilient edge architectures, real‑time data pipelines, secure device fleets, container/Kubernetes at the edge, on‑device/near‑edge ML, and measurable business impact (latency, reliability, cost‑to‑serve). This guide explains what’s changed, what to expect in interviews & how to prepare—especially for edge platform engineers, IoT/OT engineers, edge SREs, embedded/firmware engineers, edge AI/ML engineers, network engineers (5G/private LTE), security specialists & product managers. Who this is for: Edge platform/SRE, IoT solution architects, embedded/firmware developers, edge AI/ML engineers, network engineers (5G/SD‑WAN), security engineers (OT/ICS), data/streaming engineers, site deployment/field engineers & edge product managers targeting roles in the UK.

Why Edge Computing Careers in the UK Are Becoming More Multidisciplinary

For years, computing innovation was focused on the cloud. But as demand for real-time analytics, low-latency processing and secure local data handling grows, edge computing has become the next frontier. From autonomous vehicles to healthcare monitoring devices, retail checkout systems to industrial IoT, edge computing is transforming how data is processed and used in the UK. This shift has also changed what it means to work in the field. Edge computing careers are no longer purely technical. They now require knowledge of law, ethics, psychology, linguistics & design, as professionals must consider regulation, human behaviour, communication & usability alongside engineering. In this article, we’ll explore why UK edge computing careers are becoming more multidisciplinary, how these five fields intersect with edge roles, and what job-seekers & employers need to know to thrive in this evolving landscape.

Edge Computing Team Structures Explained: Who Does What in a Modern Edge Computing Department

Edge computing is expanding rapidly in the UK, driven by demands for low latency, on-site processing, IoT proliferation, autonomous systems, 5G, AI inference on devices, and regulatory pressures for data sovereignty. Businesses in sectors such as telecoms, industrial automation, retail, smart cities, autonomous vehicles, and healthcare are pushing computation and intelligence closer to where data is generated. But to design, build, deploy, secure, and maintain edge computing systems requires more than just hardware or software — it requires structured teams with clearly defined roles and responsibilities. If you’re hiring, or applying for roles via EdgeComputingJobs.co.uk, understanding who does what in a mature edge computing department will help you plan better, show relevance in job applications, and build resilient teams. This article covers the key roles in edge computing teams, how they collaborate through the project lifecycle, what skills and qualifications UK employers usually expect, salary benchmarks, challenges and trends, and best practices for structuring effective edge teams.