Image segmentation – the process of dividing images into meaningful regions – forms a critical foundation for computer vision applications ranging from medical diagnostics to autonomous driving.
When working with grayscale images in low-light conditions, a natural assumption might be that enhancing brightness and contrast would improve segmentation results. Our recent research challenges this intuition, revealing why transformer-based models remain remarkably indifferent to these adjustments.
Initial hypothesis
Our study began with a straightforward premise: enhancing grayscale images through brightness and contrast adjustments should improve segmentation by emphasizing edges and textures. To test this, we systematically evaluated both Retinex-based algorithms and 11 manual Adobe Photoshop enhancement tools.
Key findings
Despite creating visually improved images with better object visibility, our experiments showed a surprising result: segmentation metrics remained virtually unchanged.
Transformer models like SegGPT maintained consistent Intersection over Union (IoU) scores of approximately 0.75 regardless of the enhancements applied.
Why transformers remain unaffected
Our analysis revealed three fundamental reasons why transformer-based segmentation models ignore photometric adjustments:
1. Normalization erases global shifts
Input normalization effectively cancels out brightness and contrast adjustments, which are essentially linear operations. When images are standardized during preprocessing, these global shifts become irrelevant.
2. Augmentation-induced invariance
Modern vision transformers are typically trained with extensive data augmentation, including random brightness and contrast variations. This training approach teaches models to treat such variations as noise rather than meaningful features.
3. Structural primacy
Vision Transformers (ViTs) primarily analyze relative relationships between image patches rather than absolute brightness values. Global adjustments like those made with Levels uniformly scale gradients without altering the underlying structural relationships that transformers use for segmentation.
Technical deep dive: enhancement methods tested
Retinex algorithms (e.g., single-scale SSR) improved visibility but introduced artifacts (unnatural contrasts, halo effects).
SegGPT’s IoU remained unchanged (0.75 → 0.75), prompting a shift to controlled manual adjustments.
So we moved on to the Adobe tools, validated against their theoretical formulations:

1. Levels
Effect: Stretches the dynamic range to [0, 255], expanding the histogram endpoints. Dark areas become lighter, bright areas darker, increasing overall contrast.


2. High pass
Effect: Emphasizes sharp transitions (edges) by subtracting the blurred (low-frequency) component. The histogram centers around mid-gray (0.5), enhancing textures and contours.


3. Contrast
Effect: Compresses mid-tones and stretches histogram extremes. Enhances differences between light and dark areas, sharpening object boundaries.


4. Gradient
Effect: Highlights object boundaries by computing the difference between maximum and minimum intensity in a local neighborhood (e.g., 3×3). The histogram concentrates in mid-tones, visualizing object silhouettes.


5. Exposure
Effect: Linear brightness scaling shifts the histogram to the right; gamma correction non-linearly adjusts tones (γ < 1 lightens shadows, γ > 1 darkens). Risk of detail loss in overexposed areas.


6. Curves
Effect: Point-wise brightness correction. An S-shaped curve increases mid-tone contrast; concave/convex curves smooth the histogram. Enables flexible detail visibility control.


7. Brightness
Effect: Adds a constant offset to all pixels. The histogram shifts right (brighter) or left (darker) without altering its shape. Does not create new structural features.


8. Sharpen
Effect: Enhances high frequencies (edges) by adding the difference between the original and blurred image. A Laplacian filter (kernel: [-1, 5, -1]) increases sharpness but may amplify noise.


9. MixerColor (Channel mixer)
Effect: Converts color images to grayscale using human luminance perception (BT.601 standard). Preserves relative object brightness.


10. AutoFunc (Auto-contrast)
Effect: Automatically stretches the histogram by clipping the extreme 0.1-2% of pixels. Boosts contrast, but risks information loss in over-/underexposed regions.


11. Layers
Effect: Blends layers with different brightness/contrast. Lightens shadows or darkens highlights, balancing illumination. The histogram combines distributions from multiple layers.


Conclusion and future directions
We conducted an extensive analysis of the impact of manual brightness/contrast adjustments on transformer-based segmentation quality. Despite visual improvements in object visibility, segmentation metrics (IoU, Dice) showed no enhancement.
Thus, photometric transformations – simple increases in brightness/contrast – fail to improve segmentation quality in the studied approach. Normalization and training augmentations negate the effects of such adjustments.
Based on our findings, we propose shifting focus to:
- Edge features and gradient descriptors: Using algorithms like Canny or HOG to emphasize object shape rather than intensity levels
- Segmentation loss modification: Introducing regularization on object boundaries or adopting weighted metrics to prioritize precise edge delineation
- Contour-based approaches: Extracting structural or contour-based object characteristics rather than relying on global photometric corrections

Leave a Reply