Artificial Intelligence

Academic year 2017/2018

Instructor: Giorgio Fumera


DIEE Department of Electrical and Electronic Engineering

PRA Lab

Pattern Recognition and Applications Lab

Course description - Syllabus - Class schedule - Teaching material - Grading - Teaching assessment - Useful links - News - How to contact the intructor


Course description

This course introduces the students to some of the main approaches, representations and techniques developed in the field of Artificial Intelligence (AI) for designing systems capable to solve complex information processing problems.

The following three areas are covered:

  • search algorithms
  • knowledge-based systems (knowledge representation using logical languages and inference)
  • machine learning (decision trees and artificial neural networks)
The Lisp programming language is also presented.

Level

This is a graduate course under the M.Sc. program on Electronic Engineering.

Prerequisites

Knowledge of elements of discrete mathematics (combinatorics), computer architecture, and at least one programming language.

Academic period, duration and credits

  • This course is taught during the first semester (October to December)
  • Class time (lectures and exercises): 50 h
  • Credits (ECTS/CFU): 5

Syllabus

Introduction to AI

Historical notes, main AI approaches and subfields, AI achievements and applications, philosophical and etchical issues.

Solving problems by searching

  • Formulating search problems: goal, states, actions, solutions, solution cost, state space, search trees. Search algorithms. Properties of search algorithms: optimality, completeness, computational complexity
  • Uninformed search strategies: breadth-first search, depth-first search, uniform-cost search. Notes on depth-limited search, Iterative deepening depth-first search, bidirectional search
  • Informed (heuristic) search strategies: best-first search, greedy search, A* search. Heuristic functions.

Knowledge representation and inference

  • Introduction: motivating problems, knowledge-based systems
  • Introduction to logic: argumentation (reasoning), correctness of an argument, propositions, proofs, inference algorithms
  • Logical languages, models
  • Propositional logic: syntax, semantics, truth tables of connectives, the model checking inference algorithm. Inference rules and algorithms; the forward and backward chaining inference algorithms
  • Properties of inference algorithms: soundness, completeness, computational complexity
  • Predicate (first-order) logic: syntax and semantics; relationships between quantifiers. Inference rules and algorithms; the forward and backward chaining inference algorithms; notes on the resolution inference rule and algorithm
  • Application: expert systems

Machine learning

  • Hystorical notes and motivating problems
  • Inductive learning (learning from examples)
  • Supervised learning, automatic classification algorithms
  • Decision trees: definition, learning algorithms
  • Artificial neural networks. Hystorical notes. The perceptron and its learning algorithm. Multi-layer feed-forward networks; the backpropagation learning algorithm. Notes on deep neural networks
  • Notes on the statistical formulation of supervised learning problems. Main classifier evaluation methods: hold-out, cross-validation

The Lisp language

  • Introduction: procedural (imperative) and declarative (logical, functional) programming paradigms and languages; hystorical notes on Lisp
  • Main built-in data types: atoms (numbers and symbols), lists
  • Expressions and evaluation rules: self-evaluating atoms (numbers, special symbols), function calls and special expressions (lists)
  • Main built-in functions: mathematics (=, +, -, *, /, sqrt, etc.), list processing (cons, first, rest; list, length, nth, append, null), logic (equal, atom, listp, null)
  • Main special expressions: quote; and, or, not; cond (conditional expression)
  • User-defined functions: the defun special expression
  • Binding values to symbols (variable assignments): the setq and setf special expressions
  • User-defined (structured) data-types: the defstruct special expression

top of the page


Class schedule

  • Monday, 17-19h, room BA
  • Tuesday, 17-18h, room BA
  • Wednesday, 11-13h, room BA

top of the page


Teaching material

Textbook

S.J. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, Pearson, 2010 (3rd ed.).

Several copies of the Italian edition are available at the Faculty Library, and at other libraries of our University and outside (S.J. Russell, P. Norvig, Intelligenza Artificiale: un approccio moderno, UTET, 1998; Pearson Education Italia, 2005).

Useful resources on artificial neural networks

  • A classic textbook (several copies are available at the Faculty Library):
    C.M. Bishop, Neural networks for pattern recognition, Oxford University Press, 1995
    • ch. 1 (par. 1.1 to 1.6): Statistical Pattern Recognition
    • ch. 3: Single-Layer Networks
    • ch. 4: The Multi-layer Perceptron
  • A recent free online book which provides a very nice and gentle introduction to artificial neural networks and deep neural networks:
    M. Nielsen, Neural Networks and Deep Learning

Lecture slides and exercises

An extended version of the slides projected during the lectures is available. Exercises on all the topics on the syllabus will be made available during the lectures.

Lisp resources

  • Common Lisp implementations:
    • Clozure Common Lisp (CCL): open source, with a GUI for Mac OS only
    • Lispworks Personal Edition: a free (limited) version of a commercial product
    • CLisp: open source, command line interpreter (not to be mistaken for the CLIPS expert system tool - see below)
    • Jabberwocky: Java-based IDE (it requires a Lisp interpreter, like CLisp)
  • Common Lisp reference book: Guy L. Steele Jr., Common Lisp the Language, 2nd edition, Digital Press

Expert system tools

CLIPS is a well known tool for building expert systems (not to be mistaken for the CLisp interpreter for the Lisp language - see above).

Chapters 1-6 of the User's guide (cap. 1-6) are enough as a starting point. More details can be found on the Basic Programming Guide.

Machine learning software

A number of implementations of learning algorithms for decision trees and artificial neural networks (beside many other classifiers) are available. Among them:
  • the Python scikit-learn library (you may install before the package manager Miniconda, which also installs Python)
  • Weka: a Java-based software for designing classifiers and carrying out experiments
  • playground: a useful tool for visualizing on a browser the inner working of feed-forward multi-layer artificial neural networks and of the backpropagation learning algorithm, based on the open source TensorFlow library 

top of the page


Grading

The work for this course will consists of:

  • one written examination containing questions and exercises on all the topics on the syllabus
  • one take-home project assignment involving the implementation of one or more methods from one of the topics on the syllabus (search algorithms, expert systems, automatic classifier design) and their application to a given problem; the projects can be done individually or in groups of two

The exam will count for 10/30 of the grade, and the project for 20/30.

top of the page


Teaching assessment

Teaching assessment questionnaires for the previous academic years are available on the  web site of the M.Sc. program on Electronic Engineering.

top of the page


Useful links

top of the page


News

November 19, 2017
The lecture of Wed., Nov. 22, will be held at 14:30 instead of 11:00 (in room BA).

September 25, 2017
Lectures will start on Monday 25, 17h, room BA.

top of the page


How to contact the instructor

Giorgio Fumera
Dept. of Electrical and Electronic Eng. (DIEE), building B, 3rd floor
Phone: 070 675 5754
E-mail: surname AT diee DOT unica DOT it

top of the page


Most recent update: November 19, 2017