Section 5 : Kernel machines

Commentary

Section Goals

• To introduce support vector machines (SVM), or more generally, kernel machines, a relatively new family of learning methods.

Learning Objectives

Learning Objective 1

• Outline the basic principles of kernel machines or SVMs.
• Illustrate concepts capturing the mechanism of kernel machines, such as the optimal linear separators in a space of sufficiently high dimension.
• Explain the following concepts or terms:
  • Support vector machine
  • Kernel machine
  • Margin
  • Quadratic programming
  • Kernel function
  • Support vector

Objective Readings

Required readings:

Reading topics:

Kernel Machines (see Section 18.9 of AIMA3ed)

Papers:

Sonnenburg, S., Rätsch, G., Schäfer, C., and Schölkopf, B. (2006). Large scale multiple kernel learning.

Special Topic on Machine Learning and Optimization, in Journal of Machine Learning Research, 7(Jul), 1531-1565. Read three other, freely chosen papers about SVMs or kernel machines (Google, Wikipedia, and Google Scholar are recommended for this task).

Objective Questions

• Why are 'support vectors' called by this name?
• What methods can help implement a kernel machine?

Objective Activities

• Explore some open source kernel machine software. Share your findings with your fellow students through the course conference.
• Explore different applications of kernel machines in different fields.