Job Title: Senior Data Engineer
Location: Remote
Contract and Shift details: Monday to Friday, 37.5 hours a week
Are you a motivated Data Engineer ready to enhance data architecture in a dynamic, cloud-first environment? Do you thrive on building modern, scalable data platforms and forward-thinking analytics solutions?
Our Data & Analytics team is looking for a skilled Data Engineer to help design, develop, and optimise our cloud-based data infrastructure. You’ll play a key role in unlocking data-driven insights across the ABF group, working with the latest technologies like Microsoft Fabric, Databricks, and Spark to support real-time and batch data needs across global operations.
What you’ll be doing:
- Designing, building and maintaining robust, scalable data pipelines using Azure, Databricks, and Microsoft Fabric to support current and future data-driven initiatives across the business.
- Developing and optimising our data architecture, ensuring it is efficient, secure, and aligned with evolving analytics and reporting requirements.
- Partnering closely with data analysts, scientists, and software engineers to understand their data needs and deliver tailored solutions that enable actionable insights.
- Supporting both batch and real-time data processing scenarios by leveraging Spark, Delta Lakes, and Lakehouse architecture to deliver consistent and reliable data flows.
- Building reusable frameworks and internal tools that streamline data ingestion, transformation, and analysis across a variety of structured and unstructured data sources.
- Maintaining data quality, security, and compliance by embedding governance practices into the data engineering lifecycle, particularly in cloud and multi-region environments.
- Diagnosing and resolving complex data issues across platforms, performing root cause analysis, and implementing long-term improvements to ensure system reliability and scalability.
- Identifying and implementing opportunities for automation and infrastructure optimization, contributing to a culture of continuous improvement within the Data & Analytics team.
What you’ll bring:
- Experience in designing and developing cloud-native data solutions, with solid understanding of contemporary data platforms including Azure, Databricks, and Microsoft Fabric.
- Deep practical experience with Spark, along with solid knowledge in batch and real-time data processing frameworks.
- Proficient in SQL and Python programming, with experience developing performant and reusable data pipelines across both structured and unstructured data sources.
- A solid understanding of Lakehouse architecture, Delta Lakes, and Data Lake Storage (Gen2), including designing data models and storage strategies for scalability and performance.
- Familiarity with Azure tools such as Data Factory, Synapse Analytics, Logic Apps, and Event Hub, and experience integrating these into enterprise-grade data ecosystems.
- Strong problem-solving skills, with the ability to troubleshoot complex data issues, perform root cause analysis, and recommend practical technical improvements.
- A collaborative mindset with excellent communication skills — able to translate technical concepts for non-technical stakeholders and work effectively in cross-functional teams.
- A commitment to continuous learning and innovation, ideally demonstrated through certifications (Fabric Associate, Azure Data Engineer) or personal projects in cloud data engineering.
Location
This role is predominantly home-based and can be performed remotely - occasional visits to our recently refurbished Peterborough office on Lynchwood Business Park may be required. The office offers a modern working environment, excellent public transport links, free onsite parking, complimentary drinks, a daily free lunch option, and a welcoming team culture.
#LI-Remote