Decomposing Images into Layers via RGB-space Geometry
ABSTRACT
This paper presents an algorithm that decomposes an image into layers. These layers represent a single coat of paint that is applied with varying opacity. The image is decomposed into RGB- space geometry to solve a constrained optimization problem to find translucent, spatially coherent opacity for each layer, such that the composition of the layers reproduces the original image. The algorithm demonstrate the utility of the resulting decompositions for recoloring (global and local) and object insertion.
INTRODUCTION
The main features are:
-“A over B” compositing operation
-Computation of Convex hull into RGB-space
-Generation of layer opacities
-Color palette size cannot be too small and must contain valid colors with valid opacity
RELATED WORK
-Original study was to produce editable vector graphics, but want to be able to produce editable layered bitmaps (Single-Image Decomposition)
-Decomposing time-lapse videos of physical paintings into layers (Editing History)
-Separation of translucent foreground object from known backgrounds in a photo or video (Matting and Reflections)
IDENTIFYING PAINT COLORS
A pixel can be represented as a composition of colors and weights. To get these colors a RBG-space hull is computed from the image. This can be very complex due to the amount of vertices that need to be computed. In order to fix this issue the algorithm simplifies the convex hull to more manageable number of vertices. In order to make the hull simplification easier “invalid colors” are not allowed in the hull. This algorithm only allows colors between the values of 1 and 0 with an opacity value between 1 and 0.
DETERMINING LAYER OPACITY
Using the RGB layer colors obtained from the identifying of paint colors, then colors of pixels are obtained using the ideas from the “over” composition. A pixel color is generalized as a path from the background color to the layer that the pixel is on. The opacity values are determined by the length of each arrow that it requires to get to the top level color. There is at least one possible path to a pixel, due to a consequence of the convex hull.
LAYER ORDERING
The layer order does not matter as long as it is consistent throughout the algorithm. The picture’s composition will always end up as the same image. The user selects the layering order in this algorithm.
GENERALIZED BARYCENTRIC COORDINATES
We express any point inside the convex hull as a weighted average of the polyhedrons vertices. (The colors) A pixel can always be represented as a weighted average of at least 4 colors. We use this weighted average to determine the opacity of a color.
OPTIMIZATION+PERFORMANCE + CONCLUSION
The system was tested on a 2.9Ghz PC with a single core. Implementation is optimized in a multi-resolution manner. Larger images are computed progressively as the multi-resolution optimization converges on smaller images; the final optimization can take anywhere from one minute to 15 minutes to converge
The convex hull of this structure can identify a small set of colors of any image. These colors can be interpreted as layers. Then the layers can be converted to a generalizedbarycentric coordinate representation of the input image, yet aresparser.
LIMITATIONS
Assumes that layers are global when in reality colors aren’t as easily layered. Colors within the convex hull are somewhat hard to detect. Outlier colors greatly affect the convex hull.