Comicolorization: Semi-Automatic Manga Colorization

• Manual colorization is time-consuming because artists must edit pages one by one.Source: §1 · p. 2
• Fully automatic methods struggle to colorize an entire manga title because of color ambiguity.The same character must retain the same color across panels despite visual variation, while different characters may look similar.Source: §1 · p. 2
• Automatic methods tend toward faint colors for consistency or vivid colors that differ across panels.The paper illustrates these competing tendencies with automatic colorization examples.Source: §1 · p. 2, Figure 2 · p. 2
• Comicolorization uses reference-image color features to colorize characters vividly and consistently across panels.Selecting reference images for each character requires only a small number of interactions compared with manual colorization.Source: §1 · p. 2
• Users can further refine results with color dots or by controlling the dominant color.Source: §1 · p. 2

2.1 Automatic Panel Segmentation

• Input pages are binarized and automatically segmented into panels before colorization.Panel detection and segmentation are based on.Source: §2.1 · p. 2

2.2 Semi-Automatic Colorization

• The CNN colorization model combines reference color features, character-name classification training, and adversarial loss for vivid colorization.The reference histogram is quantized and binarized into a 216-dimensional palette, described as more robust than the histogram to differences in color ratios.Source: §2.2 · pp. 2–3

2 PROPOSED SYSTEM

• The pipeline takes manga pages and reference images, segments pages into panels, colorizes them, optionally revises results, and restores the original layout.Source: §2 · p. 2, Figure 3 · p. 3

2.3 User Interactive Revision

• Optional revision uses histograms for global panel changes and color dots for local changes.Users can adjust the most frequent histogram color bin to help prevent color bleeding; a dot colors the area around its selected region.Source: §2.3 · p. 3

2.4 Layout Restoration

• Post-processing changes whitish and blackish areas to pure white and black, resizes panels with super-resolution, and recomposes them with original text and contours.Source: §2.4 · p. 3

• Training uses hand-drawn character images because few paired monochrome and colorized manga images are publicly available.Color images are binarized to form monochrome and color training pairs.Source: §3 · pp. 3–4
• The network is trained with mean squared error over L*a*b color space plus adversarial and classification losses.Source: §3 · p. 4
• Synthesized color dots based on ground-truth colors are used to train the network for dot-based revision.Source: §3 · p. 4

• Reference palettes produced vivid results with the same color composition across input characters, and revisions corrected bleeding and incomplete regions.The paper reports that the method is suitable for manga because characters across panels can be colorized with the same color.Source: §4 · p. 4
• The algorithm sometimes fails to colorize a panel when multiple characters appear; editing such cases can require many color dots.Source: §4 · p. 4

5.1 Training Dataset

• 160,000 character illustrations remain after low-saturation images are removed and character images are selected using an animation-face detector.The images come from the niconico-seiga dataset.Source: §5.1 · p. 4

5.2 Network Architecture

• The architecture adds reference color features, character-name classification, and adversarial loss to the four-network model based on.A 256-dimensional color feature is concatenated into the fusion layer; the global network predicts one of 428 character-name labels.Source: §5.2 · pp. 4–5

5.3 Optimization

• Training combines mean squared error over L*a*b color with adversarial and classification losses at a 1:1:0.003 weighting.The model is trained with Adam until the loss converges.Source: §5.3 · p. 5

5.4 Generation of Training Color Dots

• Training synthesizes one-pixel color dots at random positions, with 0–15 dots per example, to teach the network to recognize user controls at test time.At test time, users can provide any number of dots; the method is reported to work beyond 15 except unusual cases such as all points sharing one position.Source: §5.4 · p. 5

6 CONCLUSIONS

• The system colorizes a whole page and is presented as the first system to colorize an entire manga at once.It uses reference-image color features to color the same character consistently across panels.Source: §6 · p. 5
• Adversarial loss emphasizes the effect of the supplied color information in the deep colorization network.Source: §6 · p. 5
• Users can interactively revise results by adjusting the color histogram and adding color dots.Source: §6 · p. 5

A SUPER RESOLUTION MODEL

• Resolution changes during colorization can unintentionally blur manga panel contours and text, motivating a super-resolution stage.Source: A SUPER RESOLUTION MODEL · p. 5
• Figure 6 compares the monochrome input, the colorization-network output, and the super-resolution output.Source: Figure 6 · p. 6
• Super-resolution increases each colorized panel from 224×224 to 448×448 pixels.The architecture is inspired by PaintsChainer and its model is based on [Ronneberger et al. 2015].Source: A SUPER RESOLUTION MODEL · p. 6

B INTERACTIVE REVISION

• Palette-based results show less color bleeding than raw-histogram results because the palette constrains the possible color space.Raw histograms can spread dominant colors beyond boundaries or produce overly whitish results.Source: B INTERACTIVE REVISION · p. 6, Figure 7 · p. 6
• The comparison includes palette, raw-histogram, and adjusted-histogram outputs alongside their input and reference images.Source: Figure 7 · p. 6
• Two color dots added to a histogram-guided result correct the eye colors with fewer interactions than the six-dot result.The paper recommends starting with reference-image color features and adding only a few dots for local regions.Source: B INTERACTIVE REVISION · p. 7, Figure 8 · p. 7

C HISTOGRAM BLENDING

• In a panel with multiple characters, the method can colorize one character's hair unappealingly, and dot-based revision wastes reference-image color information.Source: C HISTOGRAM BLENDING · p. 7, Figure 9 · p. 7
• When character sizes differ, users should adjust the blending ratio of the reference images.Source: C HISTOGRAM BLENDING · p. 7
• Blending histograms from two reference images at an equal ratio produced a suitable colorized result in the example.When character sizes differ, the user should adjust the blending ratio.Source: Figure 10 · p. 7, C HISTOGRAM BLENDING · p. 7

D FURTHER RESULTS

• The same manga pages were colorized with different combinations of reference images used as palettes.Source: D FURTHER RESULTS · p. 7
• All monochrome manga in Manga 109 were colorized using their manga cover images as palettes.Source: D FURTHER RESULTS · p. 7