Section 4 : Neural networks

Commentary

Section Goals

• To introduce the concepts and principles of neural networks.
• To discuss several learning algorithms associated with neural networks.

Learning Objectives

Learning Objective 1

• Outline the basic concepts and structures of neurons and neural networks.
• Describe the structures, principles, and capabilities of single layer feed-forward neural networks (perceptions) and multilayer feed-forward neural networks.
• Explain the learning algorithms for both kinds of neural network: gradient descent learning and back-propagation.
• Discuss topics such as structural learning of neural networks and recurrent neural networks.
• Explain the following concepts or terms:
  • Neuron
  • Neural networks
  • Connectionism
  • Neural computation
  • Activation function
  • Sigmoid function
  • Feed-forward networks
  • Recurrent network
  • Perceptron
  • Linear separator
  • Weight space
  • Back-propagate

Objective Readings

Required readings:

Reading topics:

Neural Networks (see Section 18.7 of AIMA3ed)

Lee, P. Y., Hui, S. C., Fong, A. C. M. (2002). Neural networks for web content filtering. IEEE Intelligent Systems, 17(5), 48-57. Digital Object Identifier 10.1109/MIS.2002.1039832

Supplemental Readings

Jönsson, H., Söderberg, B. (2002). An information-based neural approach to generic constraint satisfaction. Artificial Intelligence, 142(1), 1-17.

Hansen, J. V., McDonald, J. B., Nelson, R. D. (1999). Time series prediction with genetic-algorithm designed neural networks: An empirical comparison with modern statistical models. Computational Intelligence, 15(3), 171-184.

Objective Questions

• When representing knowledge, what is the main difference between neural networks and other methods, such as logics and probability models?
• What are the differences in expressiveness between perceptron, multi-layer feed-forward neural networks, and recurrent neural networks?

Objective Activities

• Explore freely some open source neural network software. Share your findings with your fellow students through the course conference.
• Explore different applications of neural networks in different fields.
• Explore the following learning algorithms for neural networks related to this section from the textbook's website.
  • Perceptron-Learning
  • Back-Prop-Learning
• Complete Exercise 18.19 and Complete Exercise 18.22 of AIMA3ed.