Personal tools



Jump to: navigation, search

Title Multimodal Information Retrieval in Medical Imaging Repositories
Reference PTDC/EEI-ESS/6815/2014
PI Carlos Costa
Participants Sérgio Matos, Augusto Silva, José Luís Oliveira
Funded by FCT
Global funding (€) 157285
RU funding (€) 157285
Starts 2016/07/01
Ends 2019/06/30

Digital medical imaging systems are essential tools in clinical practice, both in decision supporting and in treatment management. Research and industry efforts to develop medical imaging equipment, which evolved gradually to a panoply of imaging resources, have been the major driving forces towards wide acceptance of the Picture Archiving and Communication System (PACS) concept. The DICOM standard (Digital Imaging and Communications in Medicine) was a major contribution to facilitate the exchange of structured medical imaging data, and represents a key component for the success of PACS. PACS gives healthcare practitioners the ability to remotely access multimedia patient information and to set up telemedicine, telework and collaborative work environments. One the most important advantages of using digital medical imaging and networked repositories is to allow widespread sharing and remote access to medical data outside institutions, including mobile platforms. However, medical imaging repositories are often maintained in closed silos, accessible only through the DICOM query and retrieve service. Neverthless, the way users currently search for information has been shaped by search engine interfaces, and free searching is now a common feature expected from any information system. To meet this expectation and simplify the burden of acquiring and maintaining expensive platforms, we have developed Dicoogle (, a open source PACS archive that replaces the traditional centralized database with a more agile indexing and retrieval mechanism. This solution can automatically extract, index and store all metadata detected in medical imaging header, including private DICOM attribute tags, without re-engineering or re-configuration requirements. Dicoogle contains several key features to extract meta-imaging information for retrospective assessments, which is useful for statistics, management and reporting tasks. It can be used for wide-ranging clinical studies requiring, for example, dose metrics that are now increasingly available in DICOM persistent objects created by recent models of digital image equipment. By enabling multiple views over the medical data repository, in a flexible and efficient way, and with the possibility of exporting data for further statistical analysis, Dicoogle allows identification of inconsistencies in data and processes. This tool can be used to audit PACS information data and contribute to improvement of radiology department practices. Currently, the system has been in use in several hospitals and more than 22 million of DICOM images metadata have already been indexed, corresponding to a population of 160 thousands patients, and more than 450 thousands studies. Dicoogle have been also extended to support Content Based Medical Image Retrieval (CBMIR), using a profile-based approach. This solution combines a textual metadata representation, where each object has associated an indexed document, with multiple fields of interest, and a visual representation, where the DICOM object is represented by extracted image features. The CBMIR applies a similarity profile and returns a ranked list of objects sorted by similarity. The main objective of this project is to extend the previous described work to support multimodal features and semantic information. We will investigate new solutions for extracting, merging and searching over multimodal data, including text (DICOM metadata and diagnosis reports) and image information. It is aimed to perform multi-source information retrieval, i.e. the queries will include input fields from distinct data sources. Relevance feedback will be also investigated to increase the results quality of the proposed multimodal architecture. It is also our aim to investigate the contribution of semantic information in imaging retrieval and information extraction.