PaperlayerView exact revision on arXiv

Source-linked AI summary

Google's Cloud Vision API Is Not Robust To Noise

Hossein Hosseini, Baicen Xiao, Radha Poovendran

arXiv:1704.05051v2cs.CVcs.LG

TL;DR

This paper tests whether input noise can disrupt Google Cloud Vision API outputs while people still recognize image content. Across image types, noise changes the API's outputs, and filtering restores mostly original outputs.

  • Problem

    The paper asks whether noise can change Cloud Vision API outputs even when a human observer still recognizes the image content.

    Source: §I · pp. 1–2

  • Method

    The authors add impulse or Gaussian noise to natural images, face images, and text images, and test input noise filters on restored images.

    Source: §I · p. 2, §V · p. 4

  • Results

    Noise changed API outputs across image types; ImageNet attacks required an average 14.25% impulse noise density, and filtering yielded mostly original outputs on restored images.

    Source: §I · p. 2, §IV · p. 3, §V · p. 4

  • Takeaways & Limitations

    The findings indicate vulnerability in adversarial environments, while input noise filtering may improve robustness without updating image-analysis algorithms.

    Source: §I · p. 2, §VI · p. 5

Abstract

from arXiv · show

Google has recently introduced the Cloud Vision API for image analysis. According to the demonstration website, the API "quickly classifies images into thousands of categories, detects individual objects and faces within images, and finds and reads printed words contained within images." It can be also used to "detect different types of inappropriate content from adult to violent content." In this paper, we evaluate the robustness of Google Cloud Vision API to input perturbation. In particular, we show that by adding sufficient noise to the image, the API generates completely different outputs for the noisy image, while a human observer would perceive its original content. We show that the attack is consistently successful, by performing extensive experiments on different image types, including natural images, images containing faces and images with texts. For instance, using images from ImageNet dataset, we found that adding an average of 14.25% impulse noise is enough to deceive the API. Our findings indicate the vulnerability of the API in adversarial environments. For example, an adversary can bypass an image filtering system by adding noise to inappropriate images. We then show that when a noise filter is applied on input images, the API generates mostly the same outputs for restored images as for original images. This observation suggests that cloud vision API can readily benefit from noise filtering, without the need for updating image analysis algorithms.

I. INTRODUCTION

The introduction motivates testing whether noise can make Cloud Vision return different outputs while people still recognize the image. It reports successful attacks across image types and evaluates noise filtering as a countermeasure.

I. INTRODUCTION

  • Cloud Vision offers image labels, face detection, text recognition, and estimates of inappropriate content.The introduction places the API among cloud-based machine-learning services that let users access applications without training or hosting models.Source: §I · p. 1
  • The motivation is that real-world sensors have noise and other imperfections, while machine-learning systems have been reported vulnerable in adversarial settings.The paper frames robustness as important for applications such as banking, medical diagnosis, and autonomous driving.Source: §I · p. 1
  • The study tests whether impulse or Gaussian noise can change API outputs without changing the content perceived by a human observer.The authors attribute human recognition of slightly noisy images to the low-pass filtering characteristic of human vision.Source: §I · p. 2
  • Noise caused unrelated API labels across natural images, faces, and text, while people could still recognize the content; filtering restored mostly original outputs.The authors note possible consequences including bypassing image filters and irrelevant search results.Source: Figure 1 · p. 1, §I · p. 2

II. RELATED WORK

Prior work reports degraded deep-network performance on distorted inputs and methods to improve robustness. This paper instead examines perturbations against a real-world image-classification service, using random noise without model or training-data knowledge.

II. RELATED WORK

  • Prior studies report that deep convolutional networks lose performance on noisy or blurred inputs, while denoising, stabilization, and perturbed-data training have been proposed to improve robustness.The cited approaches include joint denoising, deblurring, and classification, and training to stabilize networks against small distortions.Source: §II · p. 2
  • The experiments used the Google Cloud Vision API website interface on Apr. 7, 2017; the authors also propose input noise filtering without fine-tuning.The paper characterizes the noisy images as adversarial examples because the API changes its label while a person recognizes the original content.Source: §II · p. 2
  • The paper presents random input noise as an attack on a real-world image-classification service, without knowledge of its algorithm, training data, or output labels.It contrasts this with existing black-box attacks and adversarial-example generation methods.Source: §II · p. 2

III. IMAGE NOISE

The paper represents color images as RGB pixel arrays and describes impulse and Gaussian noise models. It also discusses restoration filters and PSNR for assessing restored-image quality.

III. IMAGE NOISE

  • A color image is represented as an RGB array with pixel values ranging from 0 to 255.The array indexes image coordinates and color channels; typical image files use 8 bits per channel.Source: §III · p. 2
  • Impulse noise, also called salt-and-pepper noise, is modeled with a noise density, and weighted-average filtering is used to restore corrupted images.The text identifies impulse and Gaussian noise as types that can occur during image acquisition and transmission.Source: §III · p. 2
  • Gaussian-corrupted pixels are modeled by adding zero-mean Gaussian noise, then clipping values to the 0–255 range; low-pass kernels can reduce this noise.The noisy pixel is described as the original pixel plus a Gaussian random variable.Source: §III · p. 2
  • PSNR assesses restored-image quality against the original in decibels, with higher values indicating better quality.Typical PSNR values are considered to fall between 20 and 40 dB.Source: §III · p. 2

IV. THE PROPOSED ATTACK ON CLOUD VISION API

The attack adds impulse or Gaussian noise to images until Cloud Vision returns different outputs or fails to detect faces or text, even when people can still perceive the original content. Experiments cover ImageNet, Faces94, and text images.

IV. THE PROPOSED ATTACK ON CLOUD VISION API

  • The procedure compares API outputs for clean and noisy images, increasing impulse-noise density from 5% in 5% steps until outputs change or detection fails.Experiments included ImageNet natural images, Faces94 face images, and images with text.Source: §IV · p. 3
  • 14.25% average impulse noise deceived the API across 100 ImageNet images, and 35% always changed its output labels on the sampled images.The attack increased impulse-noise density in 5% increments until it succeeded.Source: §IV · p. 3, Figure 2 · p. 3
  • 23.8% average impulse noise deceived the API on the tested Faces94 images; noisy faces went undetected, and readable text went unidentified.The text image was still easily readable despite the API failure.Source: §IV · p. 3, Figure 3 · p. 3, Figure 4 · p. 3
  • Sufficient zero-mean Gaussian noise also changed API outputs while human observers could still perceive the original image content.The paper reports results similar to those for impulse noise.Source: §IV · p. 3

V. COUNTERMEASURES

The countermeasure tests place noise filters before Cloud Vision’s image-analysis algorithms. Across tested datasets and noise types, restored images mostly recover the API outputs for original images, while retraining-based approaches may be impractical.

V. COUNTERMEASURES

  • Regularization or data augmentation can improve robustness [29], while robust-optimization approaches may be impractical because they require retraining.The paper presents input preprocessing as a more viable approach for image recognition algorithms.Source: §V · pp. 3–4
  • Filtering is motivated as projecting noisy inputs toward the space of natural images, which can reverse noise or adversarial perturbation effects.For impulse noise the experiments used weighted-average filtering; for Gaussian noise they used a low-pass filter.Source: §V · p. 4
  • For noisy ImageNet and Faces94 images corrupted by impulse or Gaussian noise, filtering produced mostly the same API outputs as the original images.In an ImageNet example, noisy-image labels were unrelated to the original labels, while restored-image labels mostly matched the original labels.Source: §V · p. 4, Figure 5 · p. 4
  • After weighted-average filtering, the API recovered the original face attributes and correctly read text that it had missed in noisy images.Figure 6 shows restored face and text images; its captions report PSNR values relative to the originals.Source: Figure 6 · p. 4, §V · pp. 4–5

VI. CONCLUSION

The conclusion finds that noise can deceive Google Cloud Vision API without compromising the system or knowing its algorithms. Applying a noise filter to inputs can improve robustness without updating the image-analysis algorithms.

VI. CONCLUSION

  • Noise can deceive Google Cloud Vision API without system compromise or knowledge of its specific algorithms.Source: §VI · p. 5
  • Adding noise can force the API to return irrelevant labels or fail to detect faces or text.Source: §VI · p. 5
  • Restored images produce mostly the same outputs as originals, suggesting input noise filtering can improve robustness without algorithm updates.Source: §VI · p. 5

Related papers

Loading 1704.05051v2…