We are an academic research group focusing on Contextual Artificial Intelligence for Computer Assisted Interventions.
CAI4CAI is embedded in the School of Biomedical Engineering & Imaging Sciences at King’s College London, UK
Our engineering research aims at improving surgical & interventional sciences
We take a multidisciplinary, collaborative approach to solve clinical challenges (📖)
Our labs are located in St Thomas’ hospital, a prominent London landmark
We design learning-based approaches for multi-modal reasoning
Medical imaging is a core source of information in our research
We design intelligent systems exploiting information captured by safe light
We thrive at providing the right information at the right time to the surgical team and embrace human/AI interactions (📖)
Strong industrial links are key to accelerate translation of cutting-edge research into clinical impact
We support open source, open access and involve patients in our research (👋)
The 25th of September saw the latest meeting of “Science for Tomorrow’s Neurosurgery,” our now well established PPI group. As always, lots of exciting and valuable discussion with updates from Oscar on the (nearly complete!) NeuroHSI recruitment as well as Matt announcing the official opening of NeuroPPEYE phase 2!
Applications are invited for the fully funded 1+3 years MRes+PhD or 4 years PhD MRC DTP studentship (including home tuition fees, annual stipend and consumables) starting in October 2025.
This project focuses on machine learning approaches to synthetise 3D stereoscopic display from a monocular video feed captured through a standard neuroendoscope. The 3D vision capability will be tested in realistic surgical exercises to evaluate the benefit of computationally inferred depth cues.
The main MICCAI 2024 outcomes are now out and it’s a pleasure to announce that our research will be prominently featured. Congrats to the main authors from the group and all co-authors!
A prospective observational study to evaluate the use of an intraoperative hyperspectral imaging system in neurosurgery.
A prospective observational study to evaluate intraoperative hyperspectral imaging for real-time quantitative fluorescence-guided surgery of low-grade glioma.
The EPSRC Centre for Doctoral Training in Advanced Engineering in Personalised Surgery & Intervention
(CDT AE-PSI) is an innovative three-and-a-half year PhD training program aiming to deliver translational research and transform patient pathways.
Through a comprehensive, integrated training programme, the Centre for Doctoral Training in Smart Medical Imaging
trains the next generation of medical imaging researchers.
The Functionally Accurate RObotic Surgery
(FAROS) H2020 project aims at improving functional accuracy through embedding physical intelligence in surgical robotics.
The GIFT-Surg
project is an international research effort developing the technology, tools and training necessary to make fetal surgery a viable possibility.
The icovid
project focuses on AI-based lung CT analysis providing accurate quantification of disease and prognostic information in patients with suspected COVID-19 disease.
Up to 100 King’s-China Scholarship Council PhD Scholarship programme
(K-CSC) joint scholarship awards are available per year to support students from China who are seeking to start an MPhil/PhD degree at King’s College London.
The integrated and multi-disciplinary approach of the MRC Doctoral Training Partnership in Biomedical Sciences
(MRC DTP BiomedSci) to medical research offers a wealth of cutting-edge PhD training training opportunities in fundamental discovery science, translational research and experimental medicine.
The Translational Brain Imaging Training Network
(TRABIT) is an interdisciplinary and intersectoral joint PhD training effort of computational scientists, clinicians, and the industry in the field of neuroimaging.
The Wellcome / EPSRC Centre for Medical Engineering
combines fundamental research in engineering, physics, mathematics, computing, and chemistry with medicine and biomedical research.
Pathways to clinical impact
Moon Surgical
has partnered with us to develop machine learning for computer-assisted surgery. More information on our press release.
Following successful in-patient clinical studies of CAI4CAI’s translational research on computational hyperspectral imaging system for intraoperative surgical guidance, Hypervision Surgical Ltd
was founded by Michael Ebner, Tom Vercauteren, Jonathan Shapey, and Sébastien Ourselin.
In collaboration with CAI4CAI, Hypervision Surgical
’s goal is to convert the AI-powered imaging prototype system into a commercial medical device to equip clinicians with advanced computer-assisted tissue analysis for improved surgical precision and patient safety.
Intel
is the industrial sponsor of Theo Barfoot’s’s PhD on Active and continual learning strategies for deep learning assisted interactive segmentation of new databases.
Tom Vercauteren worked for 10 years with Mauna Kea Technologies
(MKT) before resuming his academic career.
Medtronic
is the industrial sponsor of Tom Vercauteren’s Medtronic / Royal Academy of Engineering Research Chair in Machine Learning for Computer-Assisted Neurosurgery.
Exemplar outputs of our research
We support open source and typically use github to disseminate our research. Repositories can be found on our group organisation (CAI4CAI) or individual member github profiles.
This work makes significant strides in the field of monoocular endoscopic depth perception, drastically improving on previous methods. The task of monocular depth perception demonstrates a nuanced understanding of the surgical scene, and could act as a vital building block in future technologies. To acheive our results, we leverage large vision transformers trained on huge natural image datasets and fine tuned to our ensembled meta dataset of sugical videos. Read more about it in our pre-print, or get our model here.
FastGeodis is an open-source package that provides efficient implementations for computing Geodesic and Euclidean distance transforms (or a mixture of both), targetting efficient utilisation of CPU and GPU hardware.
We make publicly available a spatio-temporal fetal brain MRI atlas for SBA. This atlas can support future research on automatic segmentation methods for brain 3D MRI of fetuses with SBA.
We have recently published the paper “Garcia-Peraza-Herrera, L. C., Fidon, L., DEttorre, C. Stoyanov, D., Vercauteren, T., Ourselin, S. (2021). Image Compositing for Segmentation of Surgical Tools without Manual Annotations. Transactions in Medical Imaging (📖)”. Inspired by special effects, we introduce a novel deep-learning method to segment surgical instruments in endoscopic images.
We contribute to MONAI, a PyTorch-based, open-source framework for deep learning in healthcare imaging, part of PyTorch Ecosystem.
Open source PyTorch implementation of “Dorent, R., Booth, T., Li, W., Sudre, C. H., Kafiabadi, S., Cardoso, J., … & Vercauteren, T. (2020). Learning joint segmentation of tissues and brain lesions from task-specific hetero-modal domain-shifted datasets. Medical Image Analysis, 67, 101862 (📖).”
DeepReg is a freely available, community-supported open-source toolkit for research and education in medical image registration using deep learning (📖).
We provide open source code and open access data for our paper “García-Peraza-Herrera, L. C., Everson, M., Lovat, L., Wang, H. P., Wang, W. L., Haidry, R., … & Vercauteren, T. (2020). Intrapapillary capillary loop classification in magnification endoscopy: Open dataset and baseline methodology. International journal of computer assisted radiology and surgery, 1-9 (📖).”
NiftyMIC is a Python-based open-source toolkit for research developed within the GIFT-Surg project to reconstruct an isotropic, high-resolution volume from multiple, possibly motion-corrupted, stacks of low-resolution 2D slices. Read “Ebner, M., Wang, G., Li, W., Aertsen, M., Patel, P. A., Aughwane, R., … & David, A. L. (2020). An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain MRI. NeuroImage, 206, 116324 (📖).”
PUMA provides a simultaneous multi-tasking framework that takes care of managing the complexities of executing and controlling multiple threads and/or processes.
GIFT-Grab is an open-source C++ and Python API for acquiring, processing and encoding video streams in real time (📖).
You can browse our list of open positions (if any) here, as well as get an insight on the type of positions we typically advertise by browsing through our list of previous openings. We are also supportive of hosting strong PhD candidates and researchers supported by a personal fellowship/grant.
Please note that applications for the listed open positions need to be made through the University portal to be formally taken into acount.
.js-id-open-positionApplications are invited for the fully funded 1+3 years MRes+PhD or 4 years PhD MRC DTP studentship (including home tuition fees, annual stipend and consumables) starting in October 2025.
This project focuses on machine learning approaches to synthetise 3D stereoscopic display from a monocular video feed captured through a standard neuroendoscope. The 3D vision capability will be tested in realistic surgical exercises to evaluate the benefit of computationally inferred depth cues.
Applications are invited for the fully funded 4 years full-time PhD studentship (including home tuition fees, annual stipend and consumables) starting on 1st June 2022.
Semantic segmentation of brain structures from medical images, in particular Magnetic Resonance Imaging (MRI), plays an important role in many neuroimaging applications. Deep learning based segmentation algorithms are now achieving state-of-the-art segmentation results but currently require large amounts of annotated data under predefined segmentation protocols and data inclusion/exclusion criteria. The rigidity of such approaches forbids natural interactions by humans and thus limits the usefulness for non-routine questions.
On 21st September we held our fourth ‘Science for Tomorrow’s Neurosurgery’ PPI group meeting online, with presentations from Oscar, Matt and Silvère. Presentations focused on an update from the NeuroHSI trial, with clear demonstration of improvements in resolution of the HSI images we are now able to acquire; this prompted real praise from our patient representatives, which is extremely reassuring for the trial going forward. We also took this opportunity to announce the completion of the first phase of NeuroPPEYE, in which we aim to use HSI to quantify tumour fluorescence beyond that which the human eye can see. Discussions were centered around the theme of “what is an acceptable number of participants for proof of concept studies,” generating very interesting points of view that ultimately concluded that there was no “hard number” from the patient perspective, as long as a thorough assessment of the technology had been carried out. This is extremely helpful in how we progress with the trials, particularly NeuroPPEYE, which will begin recruitment for its second phase shortly. Once again, the themes and discussions were summarized in picture format by our phenomenal illustrator, Jenny Leonard (see below) and we are already making plans for our next meeting in February 2024!
This video presents work lead by Martin Huber. Deep Homography Prediction for Endoscopic Camera Motion Imitation Learning investigates a fully self-supervised method for learning endoscopic camera motion from readily available datasets of laparoscopic interventions. The work addresses and tries to go beyond the common tool following assumption in endoscopic camera motion automation. This work will be presented at the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023).
This video presents work lead by Mengjie Shi focusing on learning-based sound-speed correction for dual-modal photoacoustic/ultrasound imaging. This work will be presented at the 2023 IEEE International Ultrasonics Symposium (IUS).
You can read the preprint on arXiv: 2306.11034 and get the code from GitHub.
Recently, we organized a Public and Patient Involvement (PPI) group with Vestibular Schwannoma patients to understand their perspectives on an patient-centered automated report. Partnering with the British Acoustic Neuroma Association (BANA), we recruited participants by circulating a form within the BANA community through their social media platforms.
We are seeking an interventional image computing researcher to design and translate the next generation of AI-assisted hyperspectral imaging systems for surgical guidance using quantitative fluorescence. The postholder, based within the Department of Surgical & Interventional Engineering at King’s College London, will play a key role in NeuroPPEye a collaborative project with King’s College Hospital and work closely with the project’s industrial collaborator Hypervision Surgical, a King’s spin-out company. A clinical neurosurgery study underpins this collaboration. The successful candidate will work on the resulting neurosurgical data as well as controlled phantom data. They will also have the opportunity to provide insight on how to best acquire prospective data.
We are seeking a Post-doctoral Research Associate to develop novel trustworthy artificial intelligence (AI) algorithms able to extract actionable information from surgical imaging data.
CAI4CAI members and alumni are leading the organization of the new edition of the cross-modality Domain Adaptation challenge (crossMoDA) for medical image segmentation Challenge, which will runs as an official challenge during the Medical Image Computing and Computer Assisted Interventions (MICCAI) 2023 conference.
The four co-founders of Hypervison Surgical, a King’s spin-out company, have been awarded the Cutlers’ Surgical Prize for outstanding work in the field of instrumentation, innovation and technical development.
The Cutlers’ Surgical Prize is one of the most prestigious annual prizes for original innovation in the design or application of surgical instruments, equipment or practice to improve the health and recovery of surgical patients.
This video presents work lead by Christopher E. Mower. OpTaS is an OPtimization-based TAsk Specification Python library for trajectory optimization and model predictive control. The code can be found at https://github.com/cmower/optas. This work will be presented at the 2023 IEEE International Conference on Robotics and Automation (ICRA).
We are working to develop new technologies that combine a new type of camera system, referred to as hyperspectral, with Artificial Intelligence (AI) systems to reveal to neurosurgeons information that is otherwise not visible to the naked eye during surgery. Two studies are currently bringing this “hyperspectral” technology to operating theatres. The NeuroHSI study uses a hyperspectral camera attached to an external scope to show surgeons critical information on tissue blood flow and distinguishes vulnerable structures which need to be protected. The NeuroPPEye study is developing this technology adapted for surgical microscopes, to guide tumour surgery.
This video presents work lead by Christopher E. Mower. The ROS-PyBullet Interface is a framework between the reliable contact simulator PyBullet and the Robot Operating System (ROS) with additional utilities for Human-Robot Interaction in the simulated environment. This work was presented at the Conference on Robot Learning (CoRL), 2022. The corresponding paper can be found at PMLR.
We are seeking a motivated research nurse/coordinator to support our NeuroHSI and NeuroPPEye project.
The post will be a Band 6 level Neurosurgery affiliated research nurse/coordinator to work within the neuroscience division at KCH. This is a full-time post, initially until end of August 2023 with a view to be extended by 6 - 12 months. The successful applicant will work across several neurosurgery sub-specialities with a particular focus on neuro-oncology and translational healthcare technology in neurosurgery. The applicate will work on research and clinical trials listed in the Department of Heath national portfolio, principally involving the development and evaluation of advanced smart camera technology for use during surgery. The post holder will work under the supervision of Mr Jonathan Shapey (Senior Clinical Lecturer and Consultant Neurosurgeon), Professor Keyoumars Ashkan (Professor of Neurosurgery) and the management of Alexandra Rizos, Neuroscience Research Manager. Some experience in clinical research and knowledge of good clinical practice would be beneficial.
Muhammad led the development of FastGeodis, an open-source package that provides efficient implementations for computing Geodesic and Euclidean distance transforms (or a mixture of both), targetting efficient utilisation of CPU and GPU hardware. This package is able to handle 2D as well as 3D data, where it achieves up to a 20x speedup on a CPU and up to a 74x speedup on a GPU as compared to an existing open-source library that uses a non-parallelisable single-thread CPU implementation. Further in-depth comparison of performance improvements is discussed in the FastGeodis documentation.
Automated segmentation and labelling of brain structures from medical images, in particular Magnetic Resonance Imaging (MRI), plays an important role in many applications ranging from surgical planning to neuroscience studies. For example, in Deep Brain Stimulation (DBS) procedures used to treat some movement disorders, segmentation of the basal ganglia and structures such as the subthalamic nucleus (STN) can help with precise targeting of the neurostimulation electrodes being implanted in the patient’s brain. Going beyond segmentation of a few discrete structures, some applications required a full brain parcellation, i.e., a partition of the entire brain into a set of non-overlapping spatial regions of anatomical or functional significance. Brain parcellation have notably been used to automate the trajectory planning of multiple intracranial electrodes for epilepsy surgery or to support the assessment of brain atrophy patterns for dementia monitoring.
Approximately 25,000 patients are diagnosed with a brain tumour every year in the UK. Brain metastases affect up to 40% of patients with extracranial primary cancer. Furthermore, although there are presently no reliable data, metastatic brain tumours are thought to outnumber primary malignant brain tumours by at least 3:1. Patients with brain metastases require individualized patient management and may include surgery, stereotactic radiosurgery, fractionated radiotherapy and chemotherapy, either alone or in combination.
Back pain presents because of a wide range of conditions in the spine and is often multifactorial. Spine appearances also change after surgery, and postoperative changes depend on the specific interventions offered to patients and human factors such as healing and mechanical adaptations, which are also unique to each patient. When reviewing medical images for diagnostic, monitoring or prognosis purpose, radiologists are required to evaluate multiple structures, including bone, muscles, and nerves, as well as the surrounding soft tissues and any instrumentation used in surgery. They must use their expertise to assess how each structure has evolved over time visually. Such readings are, therefore, both time-consuming and require dedicated expertise, which is limited to large regional spine centres throughout the UK.
Medical Ultrasound (US) is a low-cost imaging method that is long-established and widely used for screening, diagnosis, therapy monitoring, and guidance of interventional procedures. However, the usefulness of conventional US systems is limited by physical constraints mainly imposed by the small size of the handheld probe that lead to low-resolution images with a restricted field of view and view-dependent artefacts.
WiM-WILL is a digital platform that provides MICCAI members to share their career pathways to the outside world in parallel to MICCAI conference. Muhammad Asad (interviewer) and Navodini Wijethilake (interviewee) from our lab group participated in this competition this year and secured the second place. Their interview was focused on overcoming challenges in research as a student. The link to the complete interview is available below and on youtube.
Approximately 25,000 patients are diagnosed with a brain tumour every year in the UK. Meningiomas and pituitary adenomas are the first and third most common primary tumour, accounting for over 50% of all primary brain tumours. Brain metastases affect up to 40% of patients with extracranial primary cancer.
We are working to develop new technologies that combine a new type of camera system, referred to as hyperspectral, with Artificial Intelligence (AI) systems to reveal to neurosurgeons information that is otherwise not visible to the naked eye during surgery. Two studies are currently bringing this “hyperspectral” technology to operating theatres. The NeuroHSI study uses a hyperspectral camera attached to an external scope to show surgeons critical information on tissue blood flow and distinguishes vulnerable structures which need to be protected. The NeuroPPEye study is developing this technology adapted for surgical microscopes, to guide tumour surgery.
Yijing has been awarded the prestigious Royal Academy of Engineering Research Fellowship for her research in the development of tools to help neurosurgeons during surgery.
Dr Xie says that currently there is a lack of effective ways to assess brain functions in real-time, particularly during brain surgery. During such a procedure, the surgeon must remove all cancerous tissue while preserving surrounding brain tissue and regions that serve important functions.
We are seeking a biomedical optics researcher to design and translate the next generation of hyperspectral imaging systems for surgical guidance using quantitative fluorescence. The postholder, based within the Department of Surgical & Interventional Engineering at King’s College London, will play a key role in a collaborative project with King’s College Hospital and work closely with the project’s industrial collaborator Hypervision Surgical, a recently founded King’s spin-out company. A clinical neurosurgery study has been set up to underpin this collaboration.
Recent release of MONAI Label v0.4.0 extends support for multi-label scribbles interactions to enable scribbles-based interactive segmentation methods.
CAI4CAI team member Muhammad Asad contributed to the development, testing and review of features related to scribbles-based interactive segmentation in MONAI Label.
Applications are invited for the fully funded 4 years full-time PhD studentship (including home tuition fees, annual stipend and consumables) starting on 1st June 2022.
This project aims at enabling wide-field and real-time quantitative assessment of tumour-specific fluorescence by designing novel deep-learning-based computational algorithms. The project will leverage a compact hyperspectral imaging (HSI) system developed by Hypervision Surgical Ltd initially designed for contrast-free imaging.
We are actively involving patients and carers to make our research on next generation neurosurgery more relevant and impactful. Early February 2022, our research scientists from King’s College London and King’s College Hospital organised a Patient and Public Involvement (PPI) meeting with support from The Brain Tumour Charity.
We are seeking an interventional image computing researcher to design and translate the next generation of real-time AI-assisted hyperspectral imaging systems for surgical guidance. The postholder, based within the Department of Surgical & Interventional Engineering at King’s College London, will play a key role in a collaborative project with King’s College Hospital and Hypervision Surgical, a recently founded King’s spin-out company. A clinical neurosurgery study has been set up to underpin this collaboration. The successful candidate will work on the resulting neurosurgical data as well as retrospective data. They will also have the opportunity to provide insight on how to best acquire prospective data.
We are seeking an interventional image computing researcher to design and translate the next generation of AI-assisted hyperspectral imaging systems for surgical guidance using quantitative fluorescence. The postholder, based within the Department of Surgical & Interventional Engineering at King’s College London, will play a key role in a collaborative project with King’s College Hospital and work closely with the project’s industrial collaborator Hypervision Surgical, a recently founded King’s spin-out company. A clinical neurosurgery study has been set up to underpin this collaboration. The successful candidate will work on the resulting neurosurgical data as well as controlled phantom data. They will also have the opportunity to provide insight on how to best acquire prospective data.
We are seeking a highly motivated individual to join us and work on FAROS, a European research project dedicated to advancing Functionally Accurate RObotic Surgery, https://h2020faros.eu, in collaboration with KU Leuven, Sorbonne University, Balgrist Hospital and SpineGuard.
Optimal outcomes in oncology surgery are hindered by the difficulty of differentiating between tumour and surrounding tissues during surgery. Real-time hyperspectral imaging (HSI) provides rich high-dimensional intraoperative information that has the potential to significantly improve tissue characterisation and thus benefit patient outcomes. Yet taking full advantage of HSI data in clinical indication performed under binocular guidance (e.g. microsurgery and robotic surgery) poses several methodological challenges which this project aims to address. Real-time HSI sensors are limited in the spatial resolution they can capture. This further impacts the usefulness of such HSI sensors in multi-view capture settings. In this project, we will take advantage of a stereo-vision combination with a high-resolution RGB viewpoint and a HSI viewpoint. The student will develop bespoke learning-based computational approaches to reconstruct high-quality 3D scenes combining the intuitiveness of RGB guidance and the rich semantic information extracted from HSI.
Multi-task learning is common in deep learning, where clear evidence shows that jointly learning correlated tasks can improve on individual performances. Notwithstanding, in reality, many tasks are processed independently. The reasons are manifold:
Join us at the IEEE International Ultrasonics Symposium where CAI4CAI members will present their work.
Christian Baker will be presenting on “Real-Time Ultrasonic Tracking of an Intraoperative Needle Tip with Integrated Fibre-optic Hydrophone” as part of the Tissue Characterization & Real Time Imaging (AM) poster session.
King’s College London, School of Biomedical Engineering & Imaging Sciences and Moon Surgical announced a new strategic partnership to develop Machine Learning applications for Computer-Assisted Surgery, which aims to strengthen surgical artificial intelligence (AI), data and analytics, and accelerate translation from King’s College London research into clinical usage.
Yijing will develop a 3D functional optical imaging system for guiding brain tumour resection.
She will engineer two emerging modalities, light field and multispectral imaging into a compact device, and develop novel image reconstruction algorithm to produce and display high-dimensional images. The CME fellowship will support her to carry out proof-of-concept studies, start critical new collaborations within and outside the centre. She hopes the award will act as a stepping stone to enable future long-term fellowship and grants, thus to establish an independent research programme.
Miguel will collaborate with Fang-Yu Lin and Shu Wang to create activities to engage school students with ultrasound-guidance intervention and fetal medicine. In the FETUS project, they will develop interactive activities with 3D-printed fetus, placenta phantoms as well as the integreation of a simulator that explain principles of needle enhancement of an ultrasound needle tracking system.