Tuning Classification Models

When you tune a model, you select one of several ways to create a derived cost matrix. The derived cost matrix is used for any subsequent test and apply operations. Each tuning dialog box has a different goal in how the tuning is performed.

If necessary, you can remove tuning and then re-run the node.

To tune a model:

1. Open the Properties pane for the Build node. Right-click the node and select Go to Properties.

2. Go to the Test section. Select Generate Select Test Results for Model Tuning.
   The model test operation then generates the unbiased test results (no cost matrix used) for the corresponding test results. The unbiased test results are determined by the Tune Settings dialog box to initialize tuning options. For example, if only ROC is selected for Test Results, then the test operation generates the regular ROC result and the unbiased ROC result.

3. Run the Build node. You must test the models in the Build node. This is the default behavior of the Classification Build node.

4. In Properties pane for the Build node, go to the Models section. Select the models that you want to tune and click the tune icon in the menu bar.

   Select Tune from the drop-down list.

   
   Description of the illustration tunenb.gif
   
   

5. The Tune Settings dialog box opens with all the available test results.

   You can tune a model using one technique. For example, you can tune using costs or using lift, but not using both costs and lift at the same time.

6. If you are tuning more than one model, then select a model from the Models list in the bottom pane of the dialog box. After you tune the first model, return to this pane and select another model.

7. Click the tab for the test result to tune. The tabs are:

   • Cost

   • Benefit

   • ROC

   • Lift

   • Profit

8. When you finish tuning a model, click Tune in the pane on the right to generate tuning. In the Models list in the bottom pane, the Tune setting changes from Automatic to the new setting.

9. Tune as many models in the node you want. Go to other tuning tabs and perform tuning from there. When you have finished tuning, click OK.

10. All models that have tuning specifications changed during the session have their test results marked as not run. When you run the node again:

    • The new cost matrix is generated and inserted into the model.

    • A new test result is generated showing full test result information for the behavior of the current model.

11. Run the tuned model. After running of the model is complete, the Models section of Properties indicates how each model was tuned. For example, if you tune a model by changing costs, then the Tune entry for that model is Tune - Cost.

12. Right-click the Build node and select View Test Results for the tuned model to see the effects of the tuning.

You may have to repeat the tuning steps several times to get the desired results. If necessary, you can remove tuning for a model.