Information Theory

Course Information

College	Anan College	Year	2023
Course Title	Information Theory
Course Code	1555304	Course Category	Specialized / Elective
Class Format	Lecture	Credits	Academic Credit: 2
Department	専門共通科目（本科）	Student Grade	5th
Term	Second Semester	Classes per Week	2
Textbook and/or Teaching Materials	Zukai Jouhou-riron nyumon (Corona Pub., In Japanese)
Instructor	Hirayama Motoi

Course Objectives

1. Understand probability theory and stochastic processes, and calculate the probabilities of various cases.
2. Understand information theory and be able to compare the amount of information using entropy, mutual information, etc.
3. Able to mathematically analyze statistical data and find characteristic figures.

Rubric

	Ideal Level	Standard Level	Minimal Level
Achievement 1	Enable to calculate probabilities of various cases, and apply to problem solving.	Enable to calculate probabilities of various cases.	Enable to perform basic calculations probabilities of various cases.
Achievement 2	Enable to compare the amount of information using entropy, mutual information, etc, and apply to problem solving.	Enable to compare the amount of information using entropy, mutual information, etc.	Enable to calculate the amount of information using entropy, mutual information, etc.
Achievement 3	Enable to model communication channels, to explain coding method, and apply them to problem solving.	Enable to model communication channels, to explain coding method.	Enable to model communication channels.

Teaching Method

Outline:

Based on the basic knowledge of mathematics and physics and knowledge of specialized fields of information learned up to the third grade, students will learn the theory necessary for engineering problem solving in information science through exercises.

Style:

Basically follow the textbook. Deepen your understanding through exercises in self-study.

Notice:

Course Plan

			Theme	Goals
2nd Semester
	3rd Quarter
		1st	Basics of probability theory	Enable to explain definition of probability
		2nd	Basics of probability theory	Enable to calculate basic statistics such as average, variance, etc.
		3rd	Basics of probability theory	Enable to calculate conditional probabilities and to understand Markov processes and descrive them in state diagrams.
		4th	Basics of information theory	Enable to explain amount and source of information.
		5th	Basics of information theory	Enable to calculate limit distribution of information sources.
		6th	Basics of information theory	Enable to calculate entropy of information source.
		7th	Basics of information theory	Enable to calculate mutual information.
		8th	Mid-term examination	Enable to model various communication channels.
	4th Quarter
		9th	Basics of communication channel	Enable to model various communication channels.
		10th	Basics of communication channel	Enable to model various communication channels.
		11th	Basics of communication channel	Enable to explain encoding methods for various communication channels.
		12th	Communication channel encoding	Enable to explain encoding methods for various communication channels.
		13th	Communication channel encoding	Enable to explain encoding methods for various communication channels.
		14th	Communication channel encoding	Enable to explain encoding methods for various communication channels.
		15th	Communication channel encoding	Enable to explain encoding methods for various communication channels.
		16th	Final examination

Evaluation Method and Weight (%)

	Examination	Portfolio	Total
Subtotal	70	30	100
Basic Proficiency	40	20	60
Specialized Proficiency	30	10	40
Cross Area Proficiency	0	0	0