TeachingWs06SegmentationRegistrationHauptseminar

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Hauptseminar: Basics of Medical Image Formation, Segmentation, and Registration

by Tobias Lasser, Martin Groher, Andreas Keil, and Prof. Dr. Nassir Navab

2 SWS
4 ECTS
Time & Location: Wednesday, January 17, 14:15-16:15 in room MI 03.13.010
The seminar language is English.

Participation

If you are interested in image formation send an e-mail to .
If you are interested in image segmentation send an e-mail to .
If you are interested in image registration send an e-mail to .
Please include last and first name, matriculation number, current semester, discipline (diploma, master, bachelor) including major, and a desired topic.

Info and Registration meeting: Thursday, July 20th, 14:00 in MI 03.013.055
There are still spots available, for late registrations please e-mail to .

August 31st, 2006 will be the last withdrawal date to withdraw your registration. Not withdrawing before this date will result in a 5.0 grade for unacceptable or incomplete work.

Abstract

In clinical practice medical imaging is a crucial component of a large number of applications. Usually not only one image modality (imaging device) is used but several (like Computed Tomography, Magnetic Resonance Imaging, Angiography, Ultrasound, or Optical Tomography), providing different kinds of information to physicians. Moreover, medical image data acquired during diagnosis is often used to monitor treatment effects, or aid in operations where intraoperative images are revised for follow-up.

Medical image formation will cover the first part of seminar topics. The formation step covers the concepts of tomography, 3D reconstructions, and touches the physical side of image formation for the different modalities.

Medical image segmentation will cover the second part of topics in this seminar. Segmentation extracts features from images of different modalities for instance to visualize anatomical structures of interest for diagnosis or operative planning or to overlay features of different modalities.

The third part of seminar topics is on medical image registration. One of the goals of registration is to merge different image modalities in order to improve visual judgement for treatment or diagnosis. For instance, injected instruments can be visualized in 3D or tumor positions can be detected, both without opening the patient.

For participants, a general interest in Medicine and Computer Graphics/Vision would be a good point to start from.

Please bear in mind that the topics on image formation, segmentation and registration will be on similar levels!

Schein requirements

As usual:
  • attendance of the Hauptseminar meeting
  • active class participation (discussions)
  • handout (8 - 16 pages) in LaTeX
  • presentation (approximately 45 minutes + discussion) in Powerpoint or PDF format

Timeline

You are required to meet the following deadlines for discussing your work with your supervisor:
  • 4-6 weeks before presentation: Outline
  • 2 weeks before presentation: Handout
  • 1 week before presentation: Presentation Slides

Templates

Detailed Overview

Examples for for/seg/reg

Formation

Today diagnosis of diseases like cancer or study of treatment effects (e.g. for monitoring radiation therapy or for drug development) mostly relies on advanced, non-invasive imaging modalities. Techniques like X-Ray Computed Tomography, 3D Ultrasound or MRI allow non-invasive study of anatomy and have found their way to every-day use in hospitals. To highlight functional processes e.g. in blood vessels or tumors, contrast agents or targetting probes are employed together with functional imaging modalities like Angiography, PET or SPECT. Latest developments allow visualization of molecular processes like apoptosis with specifically designed imaging agents and optical tomography techniques like FMT.

The basic physical principles and theoretical methods for medical image formation will be the content of the first part of this seminar.

Segmentation

Medical images or volumes acquired using different modalities (like CT, MR, Angiography, Ultrasound, PET, SPECT) are often not only analyzed visually by a surgeon but also by more or less automated algorithms trying to emulate human perception. The partitioning of the original data into distinct regions is called "segmentation" and has a wide range of applications:

  • Segmentation for Treatment Planning: (Semi-)automatic segmentation of tumors can dramatically reduce a surgeons' workload when a complete labeling of some anatomical strucures is needed. Instead of manually editing each slice of a 3D data set, segmentation methods can be applied to segment the structure after a seed point is placed by the surgeon in the data set. This is especially useful for planning an operational intervention or radiation treatment.
  • Segmentation for Registration: Feature-based registration relies on correspondence of anatomical features which thus have to be extracted beforehand. The registration of data sets by aligning the segmented vascular structure is one example.
  • Atlas-based labeling: When registering a patient's data set with some pre-segmented standard data, anatomical structures can be quickly labeled for further visualization. This is often done for brain atlantes.
  • Segmentation for Visualization: For gaining a 3D understanding of CT or MR data, segmentation of the anatomical structure of interest can lead to much better visualizations. For virtual colonoscopy for example, a 3D CT data set of the patients abdomen is acquired and the colon's surface is reconstructed using segmentation. This segmentation now allows a virtual flight through the patients colon by rendering it's surface only. The standard procedure of looking for polyps e.g. can now be done without an invasive colonoscopy. The term "augmented reality" denotes the overlay of medical image data onto the real (optical) view. Since it doesn't make sense to overlay the original, detailed data, one might want to add only specific anatomy which leads to the same segemtation task.
  • Segmentation for Tracking: Augmenting some data onto a real view requires a pose estimation for the current view. This is usually done by using markers or features which in either case must be detected using a fully automatic segmentation.

The basics and methods for segmentation will be the content of the second part of this seminar.

Registration

Image registration can provide enhanced information from different image modalities. For that, the modalities must be aligned such that anatomical features of one modality can be automatically detected in the other modality (or modalities). Modalities can differ in viewpoint (from where the images were taken), the sensor (e.g. X-ray, Utrasound, or laser scanner), and the time the images were recorded. Depending on which specific anatomy is imaged, or which device has been used for it respectively, registration methods differ in their applicability. Furthermore, if a real patient is to be aligned with preoperative data, one has to evaluate how to acquire data from the real patient that can be registered with the image data. For instance, laser scanners can compute point clouds in space, which are later registered with the preoperative image data. For all this different applications one has to determine the best registration method. For instance, registering a region of the heart is hardly performed on rigid models since the beating heart deforms patient or modalities involved. Thus, non-rigid approaches are used in order to compensate for heart beat. Registration methods are split up into feature-based and intensity-based approaches, the former ones registering on anatomical features (landmarks, markers), the latter ones only on image intensities (e.g. gray values). The above described segmentation methods are often used to extract structure from medical images and register only them in feature-based approaches. Intensity-based registration, for instance, needs data comparison, so filters or other image processing algorithms are applied to the modalities for a suitable sensor comparability. Optimizing an initial guess of the registration parameters is crucial in almost any registration procedure. Again, the most suitable optimization algorithm (e.g. nonlinear methods like Gauss-Newton, Levenberg-Marquardt, or Best Neighbor iteration) must be chosen for a specific application. All these issues of medical image registration will be discussed in the third part of this seminar.

Topics & Schedule (preliminary)

Date Topic Student Handout/Slides Supervisor
Formation
N/A Formation I free    
N/A Formation II free    
N/A Formation III free    
Segmentation
N/A Parametric Deformable Models (Snakes) canceled   Andreas Keil
N/A Level Set and Fast Marching Methods Andreas Spurny   Andreas Keil
N/A Graph Cuts free   Andreas Keil
Registration
N/A Optical Flow Mustapha Megaiz   Martin Groher
N/A Deformable Models for Registration Xingxing Feng   Martin Groher
N/A Registration III free    

Topic Definitions and Material

The literature given below will be provided, but you are of course encouraged to look for more ;-). Google Scholar does an excellent job in finding journal papers and articles from conference proceedings.

X-ray Computed Tomography

Topic Definition:
  • Principles of X-ray CT
  • Reconstruction Techniques (ART, FBP)
  • Principles of PET and SPECT
  • Applications of CT, PET and SPECT

Literature / Material:

  • Albert Macovski: Medical Imaging Systems, 1983.
  • G. T. Herman: Image reconstruction from projections, Academic Press 1980.
  • Henrik Turbell: Cone-Beam Reconstruction Using Filtered Backprojection, Linköping University, Sweden, 2001.
  • F. Natterer: The Mathematics of Computerized Tomography, Teuber-Wiley 1986.
  • A.C. Kak, M. Slaney: Principles of Computerized Tomographic Imaging, Society of Industrial and Applied Mathematics, 2001.

Magnetic Resonance Imaging

Topic Definition:
  • Principles of MRI
  • Weighting schemes (T1, T2, etc.)
  • Reconstruction Techniques for MRI
  • Functional Imaging with fMRI
  • Applications of MRI, fMRI

Literature / Material:

Diffuse Optical Tomography

Topic Definition:
  • Principles of DOT
  • Molecular Imaging with Fluorescence Molecular Tomography (FMT)
  • Modelling of Diffusion
  • Reconstruction Techniques for FMT
  • Applications for DOT and FMT

Literature / Material:

  • A.C. Kak, M. Slaney: Principles of Computerized Tomographic Imaging, Society of Industrial and Applied Mathematics, 2001.
  • D. Boas et al: Imaging the Body with Diffuse Optical Tomography, IEEE Signal Processing Magazine 18(6), pages 57-75, 2001.
  • V. Ntziachristos, R. Weissleder: Experimental three-dimensional fluorescence reconstruction of diffuse media by use of a normalized Born approximation, Optics Letters 26, pages 893-895, 2001.
  • V. Ntziachristos et al: Looking and listening to light: the evolution of whole-body photonic imaging, Nature Biotechnology 23, pages 313-320, 2005.

Parametric Deformable Models (Snakes)

Topic Definition:
  • Introduction to snakes
  • Snake forces
  • Outline of numerical solution

Literature / Material:

  • M. Kass, A. Witkin, and D. Terzopoulos: Snakes: Active contour models. International Journal of Computer Vision, 1(4):312–331, 1988. doi:10.1007/BF00133570
  • S. Osher and N. Paragios: Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer, 2003. (Chapter 2)
  • G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Sections 4.3.1 and 4.3.2)

Level Set and Fast Marching Methods

Topic Definition:
  • Introduction to and illustration of level sets
  • Mathematical derivation of level set equation
  • Outline of numerical solution

Literature / Material:

  • J.A. Sethian: Level Set Methods and Fast Marching Methods. Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. Cambridge University Press, 2nd edition, 1999 (Chapters 1 and 2; download of introduction, chapter 1, and bibliography)
  • J.A. Sethian's excellent web resource
  • S. Osher and N. Paragios: Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer, 2003. (Chapter 1)
  • G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Section 4.3.3-4.3.5)

Graph Cuts

Topic Definition:
  • Idea of representing an image as graph
  • Explanation of different costs and their computation / estimation
  • Outline of numerical solution

Literature / Material:

  • Yuri Boykov and Marie-Pierre Jolly: Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images. Int'l Conference on Computer Vision (ICCV), 2001. Vol. 1, pp. 105-112
  • Yuri Boykov and Marie-Pierre Jolly: Interactive Organ Segmentation Using Graph Cuts. In Scott L. Delp, Anthony M. DiGioia, and Branislav Jaramaz, editors, MICCAI, volume 1935 of Lecture Notes in Computer Science, pages 276--286. Springer, 2000.
  • Y. Boykov and O. Veksler. Graph cuts in vision and graphis: theories and aplpications. In Nikos Paragios, Yunmei Chen, Olivier Faugeras, editors. Handbook of mathematical models in computer vision. Springer, 2006.

Correspondence Search in Data Sets

Topic Definition:
  • Feature Correspondences and their meaning in Registration
  • Correspondence Determination:
    • Feature Descriptors
    • Deterministic Annealing
    • EM-ICP

Literature / Material:

  • S. Belongie, J. Malik, and J. Puzicha. Shape Matching and Object Recognition Using Shape Contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(24):509–522, apr 2002.
  • H. Chui and A. Rangarajan. A new point matching algorithm for non-rigid registration. Comput. Vis. Image Underst., 89(2-3):114–141, 2003.
  • S. Granger and X. Pennec. Multi-scale em-icp: A fast and robust approach for surface registration. In European Conference on Computer Vision, pages 418–432, Copenhagen, Denmark, 2002.

Optical Flow Registration

Topic Definition:
  • Introduction and Definition of Optical Flow
  • Optical Flow for Computer Vision
  • Optical Flow in the Medical Domain

Literature / Material:

  • Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. Seventh International Joint Conference on Artificial Intelligence, Vancouver, Canada, pp. 674-679
  • Horn, B. and Schunck, B. 1981. Determining optical flow. Artificial Intelligence, 17:185-203
  • J.Y. Bouguet. Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm, Microprocessor Research Labs, 2000
  • J.L.Barron and N.A.Thacker. Tutorial: Computing 2D and 3D Optical Flow.

Deformable Models for Registration

Topic Definition:
  • Definition of Deformable Registration
  • Feature-Based Approach: Correspondences and Spline-Interpolation
  • Free-Form Deformations
  • Variational Approach

Literature / Material:

  • G. Wahba: Spline Models for Observational Data, SIAM 1990
  • S. Ilic: Implicit Meshes: Unifying Implicit and Explicit Surface Representations for 3D Reconstruction and Tracking, PhD thesis, EPFL, Lausanne, 2005
  • D. Zikic: Variational Deformable Registration for Medical Applications, Diploma Thesis, TU Muenchen, 2006

TeachingForm
Title: Basics of Medical Image Formation, Segmentation and Registration
Professor: Prof. Dr. Nassir Navab
Tutors: Tobias Lasser, Martin Groher, Andreas Keil
Type: Hauptseminar
Information: 2 SWS, 4 ECTS
Term: 2006WiSe
Abstract: In clinical practice medical imaging is a crucial component of a large number of applications. Usually not only one image modality (imaging device) is used but several (like Computed Tomography, Magnetic Resonance Imaging, Angiography, Ultrasound, or Optical Tomography), providing different kinds of information to physicians. Moreover, medical image data acquired during diagnosis is often used to monitor treatment effects, or aid in operations where intraoperative images are revised for follow-up. Medical image formation will cover the first part of seminar topics. The formation step covers concepts of tomography, 3D reconstructions, and touches the physical side of image formation for the different modalities. Medical image segmentation will cover the second part of topics in this seminar. Segmentation extracts features from images of different modalities for instance to visualize anatomical structures of interest for diagnosis or operative planning or to overlay features of different modalities. The third part of seminar topics is on medical image registration. One of the goals of registration is to merge different image modalities in order to improve visual judgement for treatment or diagnosis. For instance, injected instruments can be visualized in 3D or tumor positions can be detected, both without opening the patient.


Edit | Attach | Refresh | Diffs | More | Revision r1.27 - 25 Jan 2007 - 13:02 - AndreasKeil

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif