Leading UK Logistics Company require a Senior Data Engineer to play a pivotal role in leading the design, development and maintenance of robust data pipelines and infrastructure within their Microsoft Azure environment. Your expertise will ensure the availability, reliability and scalability of their data platform, empowering the organisation with timely and accurate information to recognise the value of their data.
Client Details
Leading UK Logistics Company
Description
Leading UK Logistics Company require a Senior Data Engineer to play a pivotal role in leading the design, development and maintenance of robust data pipelines and infrastructure within their Microsoft Azure environment. Your expertise will ensure the availability, reliability and scalability of their data platform, empowering the organisation with timely and accurate information to recognise the value of their data.
You will collaborate with the Data Science and Analytics teams to implement data models, create data integration pipelines, and optimise data workflows. This role requires a strong background in data engineering, with a solid understanding of data management principles and techniques. You will support the development of junior data engineers, providing guidance and mentorship to help them build their skills.
This is a 12 month Fixed Term Contract / Fully remote position.
Key responsibilities:
*Lead the support of the Strategic Data Platform (SDP), ensuring it is robust, reliable, and cost-effective in providing high-quality data.
*Design, develop, and uphold data integration processes, including the creation and execution of scalable and effective data pipelines for managing and transforming large datasets.
*Optimise data storage and retrieval processes to meet performance and scalability requirements.
*Ensure data quality and consistency by implementing data validation and cleansing techniques.
*Collaborate with stakeholders to define requirements and integrate new data sources.
*Monitor and troubleshoot data pipeline issues, identify and resolve bottlenecks, and implement performance optimisations.
*Collaborate with the Business Intelligence (BI) team to define and implement data models for analysis and reporting purposes.
*Maintain best practices, governance, and security in data projects.
*Stay up-to-date with the latest trends and technologies in the field of data engineering, and make recommendations for improvement.
*Provide guidance and mentorship to junior data engineers, supporting their development and skill-building; to ensure they become self-sufficient.
*Lead the technical aspects of our data strategy, ensuring alignment with organisational goals and objectives.
Experience, Skills, and Attributes:
*Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field (or equivalent experience).
*Strong programming skills in Python, Java, or Scala, with proficiency in database management systems and excellent SQL skills.
*Proven experience in designing and implementing data pipelines, ETL processes, data warehousing, and scaled data consumption patterns.
*Knowledge of data integration, data modelling concepts, and familiarity with cloud data platforms and storage technologies, ideally within Azure.
*Proficient in Azure data services (e.g., Synapse Analytics, SQL, Data Factory, Data Lake, Databricks, Cosmos DB) - Synapse and Databricks is essential
*Experience in Agile development, branch-based source control/DevOps concepts, and gathering and analysing system requirements.
*Excellent problem-solving, troubleshooting, communication, and collaboration skills.
*Attention to detail and a commitment to delivering high-quality work.
*Azure Data Engineer Associate/DP-203 certification or equivalent practical experience preferred.
*Aptitude for learning new technologies and analytical methodologies, with experience in data visualisation tools (e.g., Power BI) and familiarity with machine learning/statistical analysis techniques being a plus.
*Demonstrated ability to lead and mentor junior team members, fostering a collaborative and supportive work environment.
Profile
*Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field (or equivalent experience).
*Strong programming skills in Python, Java, or Scala, with proficiency in database management systems and excellent SQL skills.
*Proven experience in designing and implementing data pipelines, ETL processes, data warehousing, and scaled data consumption patterns.
*Knowledge of data integration, data modelling concepts, and familiarity with cloud data platforms and storage technologies, ideally within Azure.
*Proficient in Azure data services (e.g., Synapse Analytics, SQL, Data Factory, Data Lake, Databricks, Cosmos DB) - Synapse and Databricks is essential
*Experience in Agile development, branch-based source control/DevOps concepts, and gathering and analysing system requirements.
*Excellent problem-solving, troubleshooting, communication, and collaboration skills.
*Attention to detail and a commitment to delivering high-quality work.
*Azure Data Engineer Associate/DP-203 certification or equivalent practical experience preferred.
*Aptitude for learning new technologies and analytical methodologies, with experience in data visualisation tools (e.g., Power BI) and familiarity with machine learning/statistical analysis techniques being a plus.
*Demonstrated ability to lead and mentor junior team members, fostering a collaborative and supportive work environment.
Job Offer
Opportunity to work in a complex data environment
Opportunity to work on a large scale Data & Digital modernisation programme