Model approximation in MDPs and POMDPs
The Centre for Networked Intelligence (CNI) at IISc Bengaluru is pleased to announce the 6th CNI Summer School 2026.
This summer school will focus on model approximation techniques in Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs). The course will introduce fundamental concepts in stochastic systems and optimization, followed by a structured treatment of decision-making under uncertainty using MDPs and POMDPs. It will further explore approximation methods, sub-optimality bounds, and their connections to reinforcement learning. The program is designed for students, researchers, and practitioners interested in stochastic control, optimization, and learning in uncertain environments.
Instructor: Professor Aditya Mahajan, ECE, McGill University
Details
Dates: July 20–24, 2026
Time: 9:00 AM – 12:30 PM
Schedule:
- 09:00 AM – 10:30 AM | Lecture Session 1
- 10:30 AM – 11:00 AM | Break
- 11:00 AM – 12:30 PM | Lecture Session 2
Mode: Offline
Venue: Auditorium 1, MP 20, MP Building, ECE Department, Indian Institute of Science Campus
Location:
📘 Course Outline
Lecture 1
- Examples of stochastic systems
- Modeling of stochastic systems
- Input-output models
- State space models
- Stochastic optimization
- Newsvendor problem
- Selling random wind
Lecture 2
- Markov Decision Problems (MDPs)
- Principle of irrelevant information
- Principle of optimality and dynamic programming decomposition
- Bellman operators and their properties
- Examples:
- Optimal gambling
- Optimal repair
- Inventory management
Lecture 3
- Partially Observable Markov Decision Processes (POMDPs)
- Information state and dynamic programming decomposition
- Belief state and properties of value function
- Examples:
- Sequential hypothesis testing
- Remote estimation
Lecture 4
- Model approximation in MDPs
- Mismatch Bellman operator
- Sample path-based sub-optimality bounds
- Sup-norm and weighted sup-norm sub-optimality bounds
- Integral Probability Metrics (IPMs) and their properties
- IPM-based approximation bounds
Lecture 5
- Model approximation in POMDPs
- Approximate information state
- Generalized certainty equivalence
- From sub-optimality bounds to reinforcement learning
📌 Important Dates
- Application Opens: To be announced
- Application Deadline: To be announced
- Notification of Selection: To be announced
🔗 Registration
(Link to be announced)
📧 Contact
You can email us at outreach.cni@iisc.ac.in with the following subject: “CNI summer school-2025” in case of difficulties in the registration or for any other query.
❓ FAQ
1. Is there a registration fee?
- No, there is no registration fee for the summer school.
2. Will certificates be provided?
- Yes, participation certificates will be provided to those who attend all sessions.
3. Will TA/DA be provided?
- No, TA/DA will not be provided.
4. Where can I have meals?
- There are several cafeterias/restaurants available on the IISc campus.
5. Will accommodation be provided?
- No, participants must arrange their own accommodation.