Preliminary Work on Detection of RGB Criticality's: Identification and Display:
Previous results showed the construction of 3d isosurfaces through RGB volume data by the construction of a plane through the RGB data space appropriate to the object we were constructing. The main problem was unwanted attachment of the object to other objects at critical attachment points obscured by the surface. Recognizing, visualizing, and displaying these points is key to pruning undesired objects away from our desired object with minimum or no disruption to the object under study.
Current results show the visualization of attachment points (Digital Morse Criticality "Attachment Points" as a system of arrows at every vertex showing the changes in gradient while moving in X,Y,Z and R,G,B. The changes in direction while moving along data edges is our key to recognizing DMT critical points.
The first dump shows detail of the edge difference arrows (analogous to gradient arrows if this were continuous data) in R,G and B. Arrows are also used (but not shown) for movement not in space but solely in R to G, G to B and B to R (three non spatial 'arrows') we may draw in some other fashion.
The second dump shows a planar section of all of the arrows. Here all
features are shown. We can recognize critical features by the opposition
of arrows heads. The arrows point toward the higher value of R, G, and
B, respectively.
If all arrow heads point toward a point, it is a peak. If all arrow
heads point away from a point, it is a pit. If some point in and some point
out, it is a Vertex Saddle. We can organize systems of these criticality's
in a later step to be implemented.
The third dump shows only planar sections where there are only attachment points (two in, two out).
April 9 Report:
Problem Statement:
Our eye sees objects in an 2d image and we would like to isolate that object from the surrounding pixels. If the 2d image is a slice through a 3d volume, we desire to isolate that entire object from the surrounding voxels, including the unseen parts above and below our planar sub-sample.
Limitations of the The State of the Art:
Isosurface methods appear to give us sections of geometry of objects, but are not generally considered a preferred segmentation method because we have not been able to calculate or see where a particular isosurface connects to other objects that we do not wish to include in our intended object (the Segmentation Problem). Indeed, the very surface from an object we wish to extract generally obscures the attachment places where our desired object connects to undesired objects. This has been the major difficulty in 3d isosurface rendering. Other methods, such as volume rendering, seem to be aesthetically pleasing but do not provide geometric insight required for future analysis and use. Unguided isosurface geometry tends to run amuck, covering adjacent objects and obscuring the entire scene. For critical applications, such as surgical anatomic modeling, we need real surface geometry. Hand segmented objects contain segmentation artifacts, and do not nest properly inside superior and surround inferior objects. New thinking is required to overcome these difficulties without resorting to brute force segmentation by hand drawing slice wise rings, no matter how quickly a professional segmentor can draw rings. There are always going to be too many slices, and there will always be artifacts introduced by the illustrators hand.
Our Solution: Digital Morse Theory
We have undertaken a study of the mathematical nature of isosurfaces and have developed a new theory of the topological nature of isosurface defined objects, as found in digital data. We call this Digital Morse Theory, and it explains the behavior of isovalued lines, surfaces and volumes by mapping the particular locations where they connect, disconnect, emerge and disappear. Recent work in extending this to dimensions of colorspace (without registration issues) and to data fusion between divergent imaging modalities (with registration issues) has yielded promising results show in the sample surfaces here. The problem in image analysis and segmentation can be viewed as a technologically driven explosion in the dimensionally of image data. A topologic analysis, free from dimensional dependence, such as DMT, is our solution. The promise in our DMT solution is that we can now map the attachment points (Morse Criticality's) between objects in a scene, and by manipulating either the threshold value or the data pixel values at these attachment points, precisely disconnect our desired object without disturbing, or minimally disturbing the geometry of the object we are segmenting
The partial solution in 2d is we can see the entire object, and with rapid contouring, we can quickly find a suitable 2d contour line. This line represents the intersection of a larger isosurface through the slice pixel plane. We can use this to seed a 3d surface, however we may find that the surface may have attachment points (DMT Critical Points) to other objects we do not wish to include in our desired contour. The sample segmentation's here are done using this method, and we can see that other objects interfere with the visibility of our desired object. However, once we calculate the attachment critical points, we can achieve the separation we desire. In RGB space, we may find that there are attachments in one portion of color space, and a suitable segmentation surface exists in another portion of color space. The goal of a segmentor in our system under development is to navigate a segmentation isosurface through 6 dimensional space, a task that conventional computer graphics or level set algorithms will not provide help with.
Goal of effort:
Our goal is to build a mathematically inspired and geometrically sound software tool set to
We do this by building a graph of all of the available isosurface
families in a n-dimensional scene. This is a topological representation
of the data is unhindered by the problems of hidden attachment points (Morse
Critical Points being obscured in the higher dimensionally of the scene).
From this Digital Morse Theory Graph (DMT Graph) representation we can
calculate (or directly see), unobscured by geometry, if a particular isosurface
will will connect or 'clear' our desired object in space, color, or other
dimensions. The DMT graph provides information that we can use to precisely
and effectively increase the separation between ill separated objects by
the minimally manipulating the data values, or optimally adding an additional
gradient dimension to increase the barrier between objects we wish so segment.
In this way we are segmenting an object without manipulating the data values
at all, or manipulating the data values the minimum amount required to
'dam' an object and prevent unwanted flooding of the isosurface into adjacent
unwanted object regions.
Screen Dumps of work in progress
Preliminary work with RGB isosurfaces in an object with fuzzy borders (the VHP Female Bladder) have been surprisingly successful. By finding a scalar plane through the RGB space, we make a isosurface through RGB as well as XYZ space. Therefore, we have succeeded in finding that while an object may have an ill defined spatial border, it does have a well defined border in color space. Indeed, the bladder smooth muscle has a characteristic red hue that we use as our basis to extract it from the surrounding fat and other smooth muscle. Future work will require anatomists to validate and help in this surface selection and molding process.
Acknowledgments
Preliminary segmentation work originally sponsored
by Dr. Dennis Healy of the DARPA ACMP, then by Dr. Brian Athey at University
of Michigan NGI Visible Human Project. Data subsample of VHP Female bladder
supplied by Dr. Art Wetzel of the Pittsburgh Supercomputing Project. Dr.
Fred Bookstein at University of Michigan provided stimulating discussion
and insight.