Information Theory

Course Information

College	Toyama College	Year	2020
Course Title	Information Theory
Course Code	0261	Course Category	Specialized / Elective
Class Format	Lecture	Credits	School Credit: 1
Department	Department of Electronics and Computer Engineering	Student Grade	5th
Term	First Semester	Classes per Week	2
Textbook and/or Teaching Materials
Instructor	Hayase Yoshikazu

Course Objectives

Aims to learn the basics of information theory and understand the concept and definition of information quantity.
1. Understand a concept and a definition of the information.
2. Understand the model and encoding of information sources.
3. Understand the model and encoding of a communication path.

Rubric

	Ideal Level of Achievement	Standard Level of Achievement	Unacceptable Level of Achievement)
Evaluation 1	Understand a concept and a definition of the information and can calculate.	Understand the concept and a definition of information.	Does not understand the concept and definition of information.
Evaluation 2	Understand and can describe the model and encoding of information sources.	Understand the model and encoding of information sources.	Does not understand the model and encoding of information sources.
Evaluation 3	Understand and can describe the model and encoding of a communication path.	Understand the model and encoding of a communication path.	Does not understand the model and encoding of a communication path.

Assigned Department Objectives

MCCコア科目   （MCCコア科目）

JABEE B3 （情報分野について理論的に説明できる）

ディプロマポリシー 1 （電気電子工学，情報工学，通信ネットワークの知識，技術を総合的に身につけ，それらを応用したものづくりができる。）

Teaching Method

Outline:

Lecture on the concept of information quantity and entropy, probabilistic model of occurrence of information and coding related to it.

Style:

Lecture and exercise

Notice:

Require the score points of 60 or more to reach the evaluation standard of JABEE.
Those whose evaluation is less than 60 points can receive a follow-up program by submitting the approval test application. As a result of the follow-up program, the score will be 60 points if certified. Follow-up program will differ depending on contents that have not been certified, so please check in advance.

Course Plan

			Theme	Goals
1st Semester
	1st Quarter
		1st	Guidance	Understand the subjects, concepts, and necessary basic mathematics of information theory.
		2nd	Representation of information	Understand the basis of information representation in the specific concrete examples (alphabet, encoding, sign etc).
		3rd	Definition of information amount	Understand the amount of information which is the most basic concept of information theory.
		4th	Entropy	Understand the entropy which is the most basic concept of information theory.
		5th	Exercise: Various information amount (1)	Understand the joint entropy.
		6th	Exercise: Various information amount (2)	Understand the conditional entropy and mutual information.
		7th	Information source	Understand the model of information sources, the entropy of information sources.
		8th	Markov source	Understand Markov sources.
	2nd Quarter
		9th	Information redundancy	Understand the redundancy of information sources.
		10th	Noiseless discrete communication path	Understand noiseless discrete communication path.
		11th	Encoding	Understand encoding and elimination of redundancy.
		12th	Noisy discrete communication path	Understand noisy discrete communication path.
		13th	Error correction and correction	Understand error detection and correction.
		14th	Error correction code	Understand Hamming code, linear code, cyclic code.
		15th	Final exam	Confirm the degree of understanding of the information theory basis.
		16th	Evaluation / confirmation	Explanation of answer example and returning the final exam

Evaluation Method and Weight (%)

	Examination	Exercise	Total
Subtotal	80	20	100
Basic Ability	80	20	100