| Home | People | Research | Publications | Demos |
| News | Jobs |
Prospective Students |
About | Internal |
| Variable Margin Canonical Gradient Boost | ||||||||||||
|
||||||||||||
|
The problem of controlling the margin of a classifier is studied. A detailed analytical
study is presented on how properties of the classification risk, such as
its optimal link and minimum risk functions, are related to the shape of the loss,
and its margin enforcing properties. It is shown that for a class of risks, denoted
canonical risks, asymptotic Bayes consistency is compatible with simple analytical
relationships between these functions. These enable a precise characterization
of the loss for a popular class of link functions. It is shown that, when the risk is
in canonical form and the link is inverse sigmoidal, the margin properties of the
loss are determined by a single parameter. Novel families of Bayes consistent loss
functions, of variable margin, are derived. These families are then used to design
boosting style algorithms with explicit control of the classification margin. The
new algorithms generalize well established approaches, such as LogitBoost. Experimental
results show that the proposed variable margin losses outperform the
fixed margin counterparts used by existing algorithms. Finally, it is shown that
best performance can be achieved by cross-validating the margin parameter.
A number of easily reproducible experiments were conducted to study the effect of variable margin
losses on the accuracy of the resulting classifiers. Canonical logistic and canonical boosting outperform both LogitBoost and
AdaBoost in 7 and 5 of the ten datasets, respectively.
| ||||||||||||
Copyright @ 2007
www.svcl.ucsd.edu
