PhD Seminar Course on

Classifier Ensembles

Cagliari, Sept. 11 -- Sept. 18, 2009


Prof. Ludmila Kuncheva    
Bangor University, Bangor Gwynedd, UK


8 hours, Sept. 11 -- Sept. 18, 2009


Lecture 1 (3 hours):    Friday,             10:00 --- 13:00,     Sept. 11

Lecture 2 (2 hours):    Wednesday,     11:00 --- 13:00,     Sept. 16

Lecture 3 (3 hours):    Friday,             10:00 --- 13:00,     Sept. 18


Mocci Classroom (DIEE, Building A)


Classifier ensembles are a growing area of pattern recognition and machine learning. This course will set up the pattern recognition background and will introduce the basics of classifier ensembles. Ensemble approaches and methods will be presented including the "classics" such as bagging, boosting, random subspace, random forest end ECOC ensembles, as well as more recent additions to the ensemble collection, e.g., rotation forest and random oracle ensembles. We will take a look at the ensemble design strategies and combination rules, and will discuss intuition and theory about why classifier ensembles work. Diversity among the classifiers in the ensemble will be explained, along with its relationship with the ensemble accuracy. Cluster ensembles will be briefly presented, and we will touch upon classifier ensembles for changing environments.
The course (8 lecture hours) will be roughly structured as shown below:   

  1. Pattern Recognition: basic concepts, to make sure that we use the same terminology

  2. Classifier Models: "base" classifiers that will be used to construct ensembles

  3. Classifier Ensembles: general issues; combination rules; voting strategies

  4. "Classic" Classifier Ensemble Methods: error correcting output codes (ECOC); classifier fusion and classifier selection; dynamic classifier selection; Bagging, AdaBoost and Random Forests

  5. Diversity in Classifier Ensembles and Feature Selection: measures of diversity, relationship with accuracy, feature selection for ensembles and from ensembles.

  6. Advanced Ensemble Methods : Rotation Forests, Random Linear Oracle, Random Spherical Oracle.

  7. Cluster ensembles: construction methods, aggregation of the individual decisions, diversity

  8. Classifier Ensembles for Changing Environments : concept drift and types of changes; detecting a change; individual online classifiers; ensemble approaches for concept drift.


Prof. Giorgio Giacinto
Dep. of Electrical and Electronic Engineering
University of Cagliari, Italy