Logistics

Semester Fall 2025
Time & Days TuTh 9:00AM - 10:30AM
Location E2-2 #1501
Instructor Kihyuk Hong (E2-2 #3104)
Office Hours M 3:00 - 4:00 pm (E2-2 #3104)
Email [email protected]
TAs • Mingyu Yang (양민규, [email protected])
• Daho Chun (천다호, [email protected])
Textbook Introduction to Probability Models, 12th Edition, by Sheldon M. Ross.
Lecture Note Lecture Notes (Last updated: December 2, 2025 10:04 PM (GMT+9) )
Website https://www.kihyukhong.com/ie232
Midterm Oct 21 (T) 9:00 - 11:45 am. E2 #1501
Final Dec 16 (T) 9:00 - 11:45 am. E2 #1501

Course Description

This course covers mathematical models and analytical techniques essential for performance evaluation and decision-making in the rational design and operation of engineering systems subject to stochastic variability—such as production and manufacturing systems, computer and communication systems, and service systems. Key topics include the fundamentals of stochastic processes, Poisson processes and arrival process models, Markov chain models, queueing models, reliability models, decision analysis models, Markov decision processes, and stochastic simulation. Students will learn core concepts, modeling methodologies, and analytical tools for these areas. Prerequisite: IE241 or permission of the instructor.

Grading

Components Note %
Participation In-class participation (attendance, polls) 10%
Homework 4 homework sets 20%
Mini Project Modeling/simulation/analysis of a real-world stochastic system 10%
Midterm Exam In-class 30%
Final Exam In-class 30%

Links

Mini-Project

Bounty Program

Course Schedule

No. Date Lecture Notes 📹 Readings Exercise Homework
01 Sep 02 Lecture 01: Introduction. notebook
02 Sep 04 Lecture 02: Probability Theory 1 — Introduction to probability theory Ch 1.2-6, notebook HW1-1
03 Sep 09 Lecture 03: Probability Theory 2 — Discrete random variable Ch 2.1-2,2.4-5
04 Sep 11 Lecture 04: Probability Theory 3 — Poisson distribution, Continuous random variables ▶️ Ch 2.2,2.3 HW1-2
05 Sep 16 Lecture 05: Probability Theory 4 — Exponential distribution Ch 2.3, Ch 3, Ch 5.2
06 Sep 18 Lecture 06: Poisson Process 1 — Poisson process definition and properties ▶️ Ch 5.3, notes HW1-3
07 Sep 23 Lecture 07: Poisson Process 2 — Properties of Poisson processes and their applications ▶️ Ch 5.3, notebook
08 Sep 25 Lecture 08: Poisson Process 3 — Compound Poisson process, nonhomogeneous Poisson process ▶️ Ch 5.4.1, 5.4.2 **HW2-1, P3.ipynb**
Sep 29 HW1 due
09 Sep 30 Lecture 09: Queueing Theory 1 — Little’s Law, M/M/1 Queueing System, M/M/1 Queueing System with finite capacity ▶️ Ch 8.1-3
10 Oct 02 Lecture 10: Queueing Theory 2 — Shoe shine shop, Bulk service, Network of queues ▶️ Ch 8.3-4 HW2-2
Chuseok
11 Oct 14 Lecture 11: Practice problems ▶️ practice problems
12 Oct 16 Lecture 12: Midterm review. practice problems sample solution, sample midterm solution ▶️ sample midterm
Oct 17 HW2 due
Oct 21 Midterm: 9:05 ~ 11:35 am E2 #1501
13 Oct 28 Lecture 13: Markov Chains 1 — Stochastic processes, Markov property ▶️ Ch 4.1, 4.2; Notes 4.1, 4.3, 4.5, 4.7, 4.8, 4.9, 4.10, 4.11
14 Oct 30 Lecture 14: Markov Chains 2 — Classification of states ▶️ Ch 4.3 4.14, 4.16 HW3-1
Nov 3 Proposal due
15 Nov 04 Lecture 15: Markov Chains 3 — Recurrence, Absorption Probabilities, Hitting Times. ▶️ Ch 4.3, Lecture note Lecture note: 2.11, 2.12, 2.13
16 Nov 06 Lecture 16: Markov Chains 4 — Mean Return Time, Limiting/Stationary Distributions ▶️ Ch 4.4, Lecture note Lecture note: 2.14-17 HW3-2
17 Nov 11 Lecture 17: Markov Decision Processes 1 — Definition, Bellman equation ▶️ Ch 4.10, Lecture note
18 Nov 13 Lecture 18: Markov Decision Processes 2 — Policy Evaluation, Policy Iteration ▶️ Lecture note
Nov 14 HW3 due
19 Nov 18 Lecture 19: Markov Decision Processes 3 — Bellman Optimality Equation, Value Iteration, Infinite-horizon setting ▶️
20 Nov 20 Lecture 20: Markov Decision Processes 4 — Demo, Explore then Commit, Epsilon Greedy ▶️ Notebook HW4, HW4.ipynb
21 Nov 25 Lecture 21: Learning from Data 1 — Concentration inequalities and applications ▶️
22 Dec 02 Lecture 22: Learning from Data 2 — Bandit Problem. Explore then Commit ▶️
Dec 03 HW4 due
23 Dec 04 ‣Lecture 23: Learning from Data 3 — Bandit Problem. Upper Confidence Bound ▶️
24 Dec 09 Lecture 24: Practice 1 (shorter lecture: 9:00 am - 9:55 am)
25 Dec 11 Lecture 25: Practice 2
Dec 16 Final - -
Dec 19 Project due

Acknowledgement

The slides have evolved over the years, with contributions from J. R. Morrison, K. Kim, H.-J. Kim, and S. Min.