Work Packages

Clinical Focus Groups

Work Package leader: Prof. Borut Gersak

The technology developed in ARIS*ER is user-driven. The end users of the technology will be surgeons and interventional radiologists working to give the best possible care to patients. The clinical experts in this work package plays an important role in specifying details for the targeted clinical applications. They work on the workflow and evaluation aspects of the different treatment strategies involved.

User Interfaces:

Work Package leader: Dr. Adinda Freudenthal

Because the work situation is one of high cognitive and physical workload with high risks when human errors are made, intuitive and easy to learn interfaces are required. New interface design techniques are emerging involving medical end users in system design with the aim to come up with intuitive and error resistant interfaces. Experiences from aviation and process control are used, and are finally brought into the clinical field. There are still few examples in medical interface design, which meet the needs, required in intra operative control. In the medical domain not much actual application of such techniques in interface design has been demonstrated yet. True workflow supportive interface design does not exist yet. There is much general Human Factors research which is useful; however, specific visual strategies and capacities of clinical workers have hardly been investigated yet.

Some important areas in Human Factors research relevant for such a design concern: cognitive workload, decision making, human control, human learning, levels of professional expertise, professional strategies during treatment, visual performance and strategies, auditory and haptic strategies, physical workload, team work, etc.

Trends in ICT will be exploited in ARIS*ER, to meet the advanced demands:

Advanced multi-modal design: matching the natural strengths of the human user by applying less obvious combinations, such as gesture and eye movement tracking, speech control, 3D haptic/visual combinations, etc. Choices will be dictated by what is best for the user easy to learn and easy to use (and also feasible).
Adaptivity context aware: dependent on which type of intervention, the stage within the intervention, the personal preferences of the clinician user, etc.; adaptivity must be provided while maintaining consistency in interaction.
Trust the system: When the user trusts the system he/she will be more likely to delegate tasks to it.

In order to design such a future system, the first issue is to identify and communicate actual surgical needs and to facilitate problem solving in the multidisciplinary team. This is a key role for the human factor specialists: to act as a bridge between the doctors and the technologists. To understand the users needs, and uncover tacit knowledge and demands a structured analysis of the current workflow, observations of interventions, interviews, literature studies and focus groups sessions with surgeons are being conducted and used as input for future workflow development.

Secondly scientific evaluation methods are being applied and developed to safeguard the implementation of a broad spectrum of requirements, including amongst others safety, ergonomics and medical information content.

Image Processing

Work Package leader: Dr. Sergio Casciaro

Automatic segmentation of organs and other structures is vital for intraoperative usage and is used as input to other parts of the system. A combination of both model-based and statistical methods for the segmentation will be used. In this way not only information contained in the images themselves, but also knowledge of anatomy, pathology etc. is incorporated into the process. The models will be designed to fit structures of interest and at the same time have flexibility to handle unusual morphology, as is often the case with pathological organs. Statistical information will be gathered from previously segmented image sets and guide the segmentation process towards probable solutions.

Image Fusion

Work Package leader: Dr. Jerome Declerck

Research in image fusion will address new challenges in the fusion with medical volume data. The development will go beyond current rigid-body assumptions and look at fusion between pre-operative volume data and intra-operative volume data taking deformations into account. An example will be accurate fusion between a high-definition pre-operative MR data set of the brain with an intra-operative 3D ultrasound data set. The intra-operative data set will typically be different from the pre-operative data set since mass has been removed and the anatomy deformed. Fusion algorithms for robust merging of surface descriptive data (example: video) and volume descriptive data will also be investigated. The research will build upon and extend current methods and products developed by the partner Mirada as well as the current state of knowledge in the field.

Interactive 3D Visualization and Navigation

Work Package leader: Prof. Dieter Schmalstieg

In the proposed project, Augmented Reality (AR) will be used as a means of displaying multi-modal image information in an interactive manner. Reliable tools for the application of AR technology in minimally invasive therapy will be developed. This field is relatively new and immature. AR defines a wide spectrum of research topics, having in common that 3D graphics techniques previously used in Virtual Reality (VR) are being used and improved to augment the reality as we see it with digital content. AR thus leverages development from the field of computer graphics.

In this context, an important objective of the proposed project is to develop real-time rendering techniques of complex information merged into a single 3D scenery. Computer graphics for medical imaging until recently has focused on delivering the highest quality (for the sake of the patient), but has not addressed the real-time requirements of AR application. In particular, the necessary 3D volume rendering at interactive frame rates is a challenge that will be addressed in the scope of this project using and extending some recently developed real-time techniques.

Interactivity also means that the user must be allowed to provide feedback to the AR application, to customise it according to the current needs. This requires application and user interface design with intuitiveness of use and usability in mind. Special attention will be given to visualisation providing an intuitive hand-eye co-ordination. To accomplish this navigation tools for exploration the 3D space will be developed in close collaboration with the user interfaces work package. The visualisation techniques will require different navigation devices to be interfaces. These devices will enable tracking of objects such as: patient anatomy, surgical instruments, clinical user (head and eyes) and imaging equipment. Such tracking might be multi-sensorial (using different tracking techniques) and will control different parts of the 3D scenery (e.g. instruments, viewpoint, location of images).

The goal is a robust framework that comprises a scene-graph, extension nodes, 3D interaction, and communication with other work packages as well as the top-level application. Such implementation will be hidden behind a proper abstraction layer, to ensure hardware independence of the core technology.

Robotics and Haptics

Work Package leader: Prof. Jos Vander Sloten

Management

Work Package leader: Dr. Eigil Samset

The management work package will also see that the results of the participating groups are coherent, and can be integrated with the aim to create a workflow oriented surgical and interventional workplace through utilising all results in the other work packages. The deliverable from this integration work will be a demonstration software system, interfacing with intra-operative imaging and robotics, that can be used to demonstrate principles and objectives that ARIS*ER seeks to achieve. The work package will facilitate the work in the other work packages and make sure milestones are met.