Salary: £? – ? per year
Requirements:
- Hands-on Kafka experience, including topics, partitions, consumer groups, offsets, replay and schema registry.
- Experience testing high-volume ETL or ELT data pipelines at scale, with millions of rows per day or more.
- Experience writing a test strategy, including identifying what to test first, assessing risk in streaming data and sequencing evidence.
- Experience with modern architecture, including REST, contract testing and schema validation.
- Experience with automated regression testing in CI, with repeatable suites that run on every release.
- Comfort working closely with data engineers and building tooling that enables testing at the pipeline level.
- Strong Python and SQL skills.
- Structured, evidence-led approach with defined criteria, traceability and thorough documentation.
- Ability to work in a small team with loose boundaries and high standards.
- SC clearance, or genuine eligibility for it; active clearance is strongly preferred on contract.
Responsibilities:
- Test large-scale Kafka streaming and batch data pipelines, including ingestion, transformation, reconciliation and output.
- Build test harnesses with data engineers to make the system testable at scale.
- Prove correctness, completeness and integrity across very large data volumes.
- Design and build an automated regression suite to run on every release and verify accuracy, stability and response times.
- Write the test strategy for the programme from scratch.
- Prepare formal test documentation for defence factory assurance, including what was tested, how, against which criteria and milestone sign-off.
- Work hands-on as part of a small, multidisciplinary team, including supporting engineering leadership.
Technologies:
- AI
- Cloud
- ETL
- Kafka
- Python
- REST
- SQL
- Security
- API
- IoT
- UX UI Design
More:
We are supporting a UK-headquartered AI company building sovereign AI capability for defence and national security. Our platform is an AI operating system built entirely in the UK for high-security, high-assurance environments, and it deploys across cloud, edge, on-premise and fully air-gapped systems. The business has been established for around a decade, recently closed a substantial Series A round and holds a significant government framework position. We are scaling engineering and delivery across the UK, and this role sits on a new twelve-person programme team from day one. The position is remote across the UK, with occasional travel to a client site if required, and the contract rate is around £550 per day, negotiable. Two contract seats are available on the programme.
last updated 38 week of 2026
#J-18808-Ljbffr…