Full job description The Opportunity
We are working on a unique search for a Private Markets Investment Firm in London looking to appoint an experienced Lead Data Engineer to take ownership of its enterprise data platform in a completely new greenfield build out.
This is a high impact opportunity to play a central role in the development of a modern, cloud based data infrastructure within a specialised investment environment.
The successful candidate will be responsible for designing, building and managing a Snowflake-based data platform that brings together investment, portfolio, financial and operational data into a single, trusted source of information.
Working closely with senior technology and business stakeholders, you will combine hands-on engineering with architectural ownership, helping establish the foundations for advanced analytics, investment reporting, automation and AI-driven capabilities.
This role would suit a technically strong Data Engineer who enjoys building platforms, solving complex integration challenges and taking genuine ownership of the technology they deliver.
Key Responsibilities
Data Platform Architecture & Engineering
*Lead the design, development and ongoing management of a Snowflake-based enterprise data platform.
*Develop scalable data architectures using modern data modelling principles, including medallion architecture (bronze, silver and gold layers).
*Build and maintain robust ingestion pipelines integrating multiple internal applications, external data providers and third-party administrators.
*Establish data governance, quality, security and lineage frameworks to ensure the integrity and reliability of investment data.
*Implement infrastructure as code, automated deployment processes and CI/CD best practices.
Data Integration & Investment Data Management
*Lead the integration of portfolio management, fund administration, market data and financial reporting systems.
*Design and maintain security master and reference data models covering financial instruments, issuers and counterparties.
*Develop data reconciliation, validation and exception management processes across multiple data sources.
*Build governed datasets supporting investment positions, exposures, cash flows, liquidity, counterparties and fund accounting.
*Work closely with investment, finance, risk and technology teams to translate business requirements into scalable technical solutions.
Platform Ownership & Innovation
*Take end-to-end responsibility for platform performance, reliability, security and cost optimisation.
*Develop data models and semantic layers supporting Power BI, management reporting and investment analytics.
*Support the adoption of AI and automation by making high-quality, governed data accessible to emerging applications and intelligent workflows.
*Establish best practices around data engineering, documentation, monitoring and operational resilience.
*Collaborate with external technology partners while maintaining internal ownership of architecture and technical decisions.
Skills & Experience
Essential
*Extensive hands-on experience designing, implementing and managing Snowflake data platforms in production environments.
*Strong SQL and Python skills, with experience developing complex data pipelines and transformation frameworks.
*Proven experience in financial services, ideally within investment management, asset management, private markets or fund administration.
*Strong understanding of financial datasets, including positions, transactions, cash flows, valuations and reference data.
*Expertise in data modelling, data warehousing, master data management and modern cloud-based architectures.
*Experience with Azure cloud technologies and associated data engineering services.
*Strong understanding of APIs, data ingestion, third-party integrations and data reconciliation.
*Experience with Git, CI/CD, infrastructure as code, automated testing and platform monitoring.
*Ability to engage confidently with senior stakeholders and independently own technical architecture and delivery.
Desirable
*Experience within private credit, direct lending, alternative investments or broader private markets.
*Familiarity with loan administration, portfolio accounting or investment management platforms.
*Exposure to market data providers such as ICE, Bloomberg or equivalent.
*Experience with Power BI, dbt and data catalogue or lineage tools.
*Knowledge of Snowflake Cortex, AI/LLM data integration or advanced analytics use cases.
*Understanding of financial services regulatory requirements, including DORA and GDPR.
*Previous experience building a data platform from the ground up or leading a significant data transformation programme.
Why Consider This Opportunity?
*Significant ownership: Take responsibility for a strategically important enterprise data platform, from architecture through to delivery and ongoing development.
*Modern technology stack: Work extensively with Snowflake, Azure, Python and modern data engineering tools.
*Direct senior exposure: Collaborate closely with technology leadership and key investment, finance and risk stakeholders.
*Greenfield engineering: Shape the technical standards, architecture and engineering practices rather than simply maintaining an established environment.
*Private markets exposure: Work with complex investment datasets in a specialist financial services environment.
*AI and automation: Contribute to the development of a data foundation supporting next-generation investment analytics and intelligent automation.
The Ideal Candidate
We are particularly in