Full job description3 CDI Data Engineering vacancies, differing mainly in where they sit within the customer data lifecycle.
The first is the most data-centric, the second is the most DevOps/MLOps-oriented, and the third is the most transformation-focused.


Remote working.
Client Details
Our client operates within the Oil and Gas sector. British Petroleum.
Description
Role 1: CDI Data Engineer


*Implement and maintain SCI base, fact and curated pipelines.
*Deliver Elavon/PAR/WRE/payment and loyalty matching.
*Support ICE-to-EV data engineering and dashboard datasets.
*Implement DO-site and market-specific logic.
*Investigate volume drops, match-rate changes and data completeness defects.
*Execute controlled reloads, production validation, DQ/freshness controls and BAU support.
*Prepare PRs, tests, technical designs, CAB/release evidence and stakeholder




Role 2: BNA Production-Readiness, MLOps & Integration Engineer:
*Automate end-to-end BNA orchestration
*Reliable retry/recovery and idempotent processing
*Production monitoring and alerting automated activation and acknowledgement handling
*Reducing manual intervention Stable QA production deployment process
*Runbooks and operational handover
*Reducing runtime and production failures




Role 3: CIS/SCI Integration & Post-Migration Stabilisation Engineer:
*Repointing remaining CIS/SCI dependencies
*Stabilising migrated pipelines
*Reusable error-handling patterns
*Reducing legacy technical debt
*Removing dual-platform monitoring
*Resolving SFMC/Comarch integration issues
*Reducing production support burden
*Clear handover into BAU
Profile
Role 1: CDI Data Engineer
Must Have worked with:
*Strong SQL, Python/Spark and Databricks pipeline engineering background.
*Large-scale transaction, payment, loyalty, EPOS/WRE and identity-matching work history
*Historical/full/incremental loads, high-water marks and reconciliation.
*Azure DevOps PRs, unit testing, CI/CD and production release support.
Nice to have: Reporting/Power BI dataset exposure desirable. SFMC/CIS integration and multi-market SCI knowledge desirable.


Role 2: BNA Production-Readiness, MLOps & Integration Engineer
Must Have:
*Strong hands-on background in Databricks, PySpark, Python and SQL Production pipeline engineering and troubleshooting
*Workflow orchestration and dependency management Idempotency, retry/restart and failure-recovery patterns CI/CD, version control and automated deployment
*Worked across monitoring, logging and operational alerting
*Completed data-quality and reconciliation controls API/file/SFTP integration patterns


Nice to have: Delta Lake / Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment work history SFMC or Comarch integration Customer decisioning / personalisation platforms




Role 3: CIS/SCI Integration & Post-Migration Stabilisation Engineer
Must Have worked with:
*Strong hands-on background with Databricks, SQL, Python/PySpark ETL/ELT Data-pipeline migration
*Legacy-to-cloud modernisation
*Production troubleshooting
*Dependency analysis
*Pipeline repointing
*Error handling and recovery
*Data validation/reconciliation
*Integration testing
*Production support / L2-L3 troubleshooting


Nice to have: AWS-to-Databricks migration Informatica / SSIS / legacy ETL SFMC Comarch SFTP/file interfaces Azure/AWS CI/CD










Job Offer
£400-£500 day rate
Remote working
12 months contract
Inside IR35 only.