跳转到内容

Anomaly Detection vTool#

The Anomaly Detection vTool identifies patterns or features in your images that deviate from normal or expected behavior, based on the model you have selected.

Anomaly detection is a form of binary classification in that it distinguishes normal (desired) from abnormal (undesired) data allowing you to sort your images into good and bad ones. In contrast to standard classification, it can handle highly imbalanced data, where anomalies are rare, not well-represented, or not existent at all in the training data set.

The Anomaly Detection vTool can classify images it receives via the Image input pin. It outputs the results via the Classes, Scores, Regions, Heatmaps, and InferenceImage output pins.

If you use the ROI Creator vTool to specify a region of interest before anomaly detection, the Anomaly Detection vTool accepts the ROI data via its Roi input pin. In that case, anomaly detection is only performed on the region of interest. As a side effect, this results in faster processing.

Anomaly Detection vTool

运作原理#

Assume that you want to check your PCB production line for anomalies.

This is your input image (image source: a) showing a PCB with a bent pin:

Input Image Showing a PCB

You configure the vTool by loading a model that has been trained to detect irregularities in PCBs.

Bundle for Animal Anomaly Detection Selected

Next, you configure the detection settings:

  • Visualize anomalies and output heatmap: The vTool includes a visualization feature, which is enabled by default. This allows you to view the data in the form of a heatmap. You can choose from a variety of color maps for the heatmap. Different color maps highlight anomalies more effectively depending on the data and context.
    If you don't need this visualization and no further analysis of the image data is required, you can deselect the heatmap. This may increase the throughput. If you deselect this option, the Regions, Heatmaps, and InferenceImage output pins are disabled.

    Differentiation of InferenceImage, Heatmaps, and Regions

    The InferenceImage is the input image with the heatmap overlay to visualize anomalies in the pin data view.

    Heatmaps are not intended for visualizing anomalies in the pin data view. They are images of float values representing the anomaly score for each pixel and are used if further analysis in the application is desired.

    Regions are computed by thresholding the heatmap values. Pixels with an anomaly score above a certain value are grouped into a region, thus separating normal from abnormal regions.

  • Anomaly threshold: The anomaly threshold defines the boundary between images classified as normal and those considered abnormal based on the score returned by the model selected. The default threshold value is selected during the model training based on the training data set. However, the optimum threshold depends on the specific requirements of your application and the trade-offs you're willing to accept. You can adjust it as required. A higher threshold means fewer anomalies are detected but the anomalies that are detected come with a higher confidence. A lower threshold means more anomalies are detected but some of them may be false alarms.

    信息

    Pixel intensity variations on non-anomalous inference images indicate slight deviations from the normal image. However, as long as these deviations stay below the predefined anomaly threshold, the image is classified as normal. This observation could be useful for manually adjusting the threshold if needed.

After configuring the vTool, you run the recipe that initiates the inference process. The vTool utilizes the model to analyze the image and classify the objects present. When the analysis has finished, the anomaly detection result is returned. You can then view the result in the pin data view.

Detailed Output with InferenceImage Pin Selected

Anomaly Detection with InferenceImage Pin Selected

InferenceImage: Four areas with anomalies have been detected and are highlighted in the image using pixel intensity heatmaps. However, only one of them exceeds the anomaly threshold, leading to the image being classified as "abnormal".

Scores: The highest anomaly score across the entire image is 52.5901. Even though four anomalous areas have been detected, only one is above the anomaly threshold and gets an anomaly score.

Classes: The image has been classed as abnormal as the score is above the threshold.

Detailed Output with Regions Pin Selected

Anomaly Detection with Regions Pin Selected

Classes: The image has been classed as abnormal as the score is above the threshold.

Scores: The highest anomaly score across the entire image is 52.5901.

Regions: The anomalous region has been highlighted (indicated by the arrow in the screenshot). In contrast to the inference image output, the region output only marks parts of the image where the anomaly score is above a certain value.

配置 vTool#

To configure the Anomaly Detection vTool:

Anomaly Detection vTool Settings

  1. In the Recipe Management pane in the vTool Settings area, click Open Settings or double-click the vTool.
    The Anomaly Detection dialog opens.

  2. In the Server Selection area, select the server you want to connect to.
    The server must be running to be able to connect to it.

    • Option 1: Enter the IP address and port of your Triton Inference Server manually.
    • Option 2: Select the IP address / host name and port of your server in the drop-down list.

  3. Click Connect to establish a connection.
    The vTool then loads the available models.

  4. 在 Model Selection area, select the bundle that contains the desired model in the drop-down list.
    After you have selected a bundle, the information below the drop-down list is populated with the bundle details allowing you to check that you have selected the correct bundle.
    Model Selection Area

  5. If the desired bundle isn't listed, click Deploy New Model in the drop-down list.
    Deploy New Model
    This opens the pylon AI Agent that allows you to deploy more models.
    After you have deployed a new model, you have to click the 刷新 button.

  6. In the Detection Settings area, select the desired color map for your heatmap.

  7. In the Detection Settings area, adjust the anomaly threshold if required.
    The anomaly threshold is part of the model. Changing it may have undesired consequences.

    信息

    The parameters available in this area depend on the model selected. Some parameters are only available after selecting a certain model. Default values are provided by the model.

  8. 单击确定。

You can view the result of the anomaly detection in a pin data view. Here, you can select which outputs to display.

输入#

图像#

直接从 Camera vTool 或从输出图像的 vTool（例如 Image Format Converter vTool）接受图像。

  • 数据类型：图像
  • 图像格式：8 位到 16 位彩色图像

ROI#

Accepts a region of interest from the ROI Creator vTool or any other vTool that outputs rectangles. If multiple rectangles are input, the first one is used as a single region of interest.

  • Data type: RectangleF, RectangleF Array

输出#

类#

Returns the classes (normal or abnormal) for each input image.

  • 数据类型：字符串数组

得分#

Returns the anomaly scores for the class. This indicates the severity of the anomaly detected, i.e., how likely it is that the image is abnormal. Note that this value is not a probability and, depending on the model, it can range from very small to very large numbers. It should be interpreted only in relation to the threshold defined.

  • 数据类型：浮点数组

Heatmaps#

Returns the heatmap. This output can be disabled. Heatmaps are not intended for visualizing anomalies in the pin data view. They are images of float values representing the anomaly score for each pixel and are used if further analysis in the application is desired.

  • Data type: Image Array

区域#

Returns the region for each predicted anomaly. This output can be disabled. Regions are computed by thresholding the heatmap values. Pixels with an anomaly score above a certain value are grouped into a region, thus separating normal from abnormal regions.

  • 数据类型：区域数组

InferenceImage#

Returns the input image with heatmap overlay to visualize anomalies in the pin data view. This output can be disabled.

  • 数据类型：图像

Payload#

Returns additional information regarding the performance of the Inference Server for evaluation. This output pin only becomes available if you enable it in the Features - All pane in the Debug Settings parameter group.

  • 数据类型：字符串

典型前置项#

  1. Visual Anomaly (VisA) was accessed on DATE from https://registry.opendata.aws/visa. Zou, Yang, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. "SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation." arXiv preprint arXiv:2207.14315 (2022).