ROC

ROC is only supported for binary models. The ROC Tuning tab adds a side panel to the standard ROC Test Viewer. The following information is displayed:

• Performance Matrix in the upper right pane, enables you to display these matrices:

  • Overall Accuracy: Cost matrix for the maximum Overall Accuracy point on the ROC chart.

  • Average Accuracy: Cost matrix for the maximum Average Accuracy point.

  • Custom Accuracy: Cost matrix for the custom operating point.

    You must specify a custom operating point for this option to be available.

  • Model Accuracy: The current Performance Matrix (approximately) of the current model.

    You can use the following calculation to derive Model Accuracy from the ROC result provided:

    If there is no embedded cost matrix, then find the 50 percent threshold point or the closest one to it. If there is an embedded cost matrix, then find the lowest cost point. For a model to have an embedded cost matrix, it must have either been tuned or it has a cost matrix or cost benefit defined by the default settings of the Build node.

• The Performance Matrix Grid shows the Performance Matrix for the option selected.

• Click Tune to:

  • Select the current performance option as the one to use to tune the model.

  • Derive a cost matrix from the ROC result at that probability threshold.

  Tune Settings, in the lower part of this panel, is updated to display the new matrix.

• Click Clear to clear any tuning specifications and set tuning to Automatic. In other words, no tuning is performed.