Section 3 : Instance-based learning

Commentary

Section Goals

- To introduce two types of instance-based learning methods, nearest-neighbour methods and kernel methods, which can construct hypotheses directly from the training instances themselves.

Learning Objectives

Learning Objective 1

• Explain the key ideas of the nearest-neighbour models and the k-nearest-neighbour learning algorithm.
• Explain the principle of the kernel model, and ideas associated with the relevant algorithms.
• Explain the following concepts or terms:
  • Parametric learning
  • Nonparametric learning
  • Instance-based learning
  • Nearest-neighbour models
  • Distance metric
  • Kernel model
  • Kernel function
  • Linear classification
  • Logistic regression

Objective Readings

Required readings:

Reading topics:

Instance-Based Learning, Nearest-Neighbour Models, Kernel Models (see Section 18.6 and 18.8 of AIMA3ed)

Supplemental Readings

Jorgensen, Z., Zhou, Y., and Inge, M. (2008). A multiple instance learning strategy for combating good word attacks on spam filters. Journal of Machine Learning Research, 9(Jun), 1115-1146.

Objective Questions

• What is the difference between parametric learning and nonparametric learning?
• How can an algorithm of kernel methods be realized?

Objective Activities

• Explore some open source software capable of k-nearest-neighbour learning algorithms and/or kernel methods (e.g., Weka). Complete a small test of the software with the data provided, and share your findings with your fellow students in the course conference.