CASTLE Labs presentations
From time to time, I have the opportunity to give presentations that may be of general interest. Here are the Powerpoint presentations from these talks. Please note that the talks were designed assuming that I am giving the presentation. For this reason, these presentations may be little more than teasers. If a talk looks interesting, email me at email@example.com to see if it would be appropriate for me to give the presentation in person.
Approximate Dynamic Programming: Does Anything Work?, Applied Probability Workshop, Rutgers University, June 6 14, 2014, (powerpoint format)
This talk summarizes a series of efforts at solving a relatively simple energy storage problem. In a nutshell, none of the algorithms were what you could call an unqualified success. Approximate value iteration and approximate policy iteration were disappointments when using any sort of machine learning technique to approximate the value function. Policy search (using a simple error correction) worked surprisingly well, but is limited to very low dimensional parameters (which means we cannot consider time-dependent policies). Lookup table works terribly. Lookup table with structure (convexity or monotonicity) works extremely well! But this is basically fancy lookup table, which means it will not scale.
The (Rocky) Path to 80 Percent Renewables - STEP seminar series, Princeton University, April 14, 2014, (powerpoint format)
This is an evolving talk on the challenge of reaching the Obama administration's goal of 80 percent renewables. The talk highlights some recent studies that argue that we can provide as much as 99 percent of our electricity from wind and solar, and provides a tour of different types of variability and uncertainty. We illustrate two styles of modeling. The first uses a simple spreadsheet to demonstrate the challenge of serving electricity demands with wind and solar. The second a large and very detailed model, SMART-ISO, which models the flow of information and decisions, along with detailed models of generator dynamics, which we are using to do a study of off-shore wind. Preliminary results from this study are presented.
Clearing the Jungle of Stochastic Optimization - APMOD 2014, University of Warwick, April 102014(powerpoint format)
This is my latest version of my talk on computational stochastic optimization, focusing on modeling, defining state variables, and describing the four classes of policies. This was given at the APMOD conference at the University of Warwick in England, April 10, 2014. Two recent tutorial articles for this work are:
Energy and Uncertainty: Models and Algorithms for Complex Energy Systems - to appear in AI Magazine - This is a light version of the TutORials article above.
Energy and Uncertainty: Clearing the Jungle of Stochastic Optimization - CIRRELT, November 25, 2013 (powerpoint format)
This presentation was to have been given at the 3rd Colloquium of the NSERC/Hydro-Quebec Industrial Reearch Chair on Stochastic Optimization of Electricity Generation (but my flight was cancelled!). It contains a brief summary of different types of uncertainty that arise in energy applications, and then describes a modeling and algorithmic framework that encompasses different communities within stochastic optimization. The last half of the talk provides the most recent update of SMART-ISO.
On Languages for Stochastic Optimzation - University of Quebec at Montreal, November 18, 2013 (powerpoint format)
This presentation was given at the University of Quebec at Montreal as part of their commencement exercises, where I was awarded the Docteur honoris causa. Montreal, with its bilingual tradition, has been the place where I seem to keep coming back to the theme of modeling and languages (this is highlighted at the beginning of the talk). The rest of the talk brings out concepts, terminology and notation from the different communities that work in some form of stochastic optimization.
SMART-ISO- A Stochastic, Multiscale Model of the PJM Energy Markts - Fields Institute Workshop on Electricity Markets, August, 2013 (powerpoint format - 20 megs!)
This is the most recent of a series of talks on SMART-ISO, a large-scale and highly detailed simulator of the PJM energy system, includuing generation, distribution and consumption. SMART-ISO carefully models the flow of information and decisions, capturing day-ahead planning, day-of unit commitment and economic dispatch, and real-time economic dispatch. This talk also discusses some general modeling issues for sequential decision problems arising in energy (click here for more information on modeling).
Tutorial: Clearing the Jungle of Stochastic Optimization for Transportation and Logistics - Tristan VIII - June, 2013 (powerpoint format)
This was a tutorial given at Tristan VIII in June 2013. Using the context of transportation and logistics, it describes how to model sequential stochastic optimization problems ("dynamic programs") and then illustrates the four classes of policies. As with other presentations, it was designed to accompany my oral presentation. A version of the talk, designed to be read, is available here; click on lectures, and download at least the first two PowerPoint presentations.
Bridging stochastic programming and dynamic programming - March, 2013 (powerpoint format).
This talk was given at Georgia Tech, with the goal of introducing students to a particular style of modeling stochastic, dynamic optimization problems. It addresses topics such as defining state variables and the five components of a stochastic dynamic problem. We make the transition from finding the best decision in deterministic optimization problems, to finding the best policy for stochastic optimization problems. We identify four fundamental classes of policies, and then describe two of these in more detail: lookahead policies, and policies based on value function approximations. The presentation closes with a step by step translation of "dynamic programming" as it is done by leaders in stochastic programming (in the form of Alex Shapiro) to the classical notation of Markov decision processes.
Approximate dynamic programming for locomotive planning - April 17, 2012 (powerpoint format)
PLASMA uses the modeling and algorithmic framework of approximate dynamic programming to perform strategic, tactical and real-time optimization of locomotives. The system has been developed jointly with Norfolk Southern Railroad (BNSF also made major contributions to the system for several years). PLASMA breaks down the planning problem into a sequence of short-horizon problems that involve the assignment of locomotives to individual trains. Value function approximations capture the value of assignments now on the future, producing solutions that perform well over longer horizons. The system can perform strategic planning over horizons of a month or more, and tactical planning (which uses a live locomotive snapshot) over horizons of a week or more. The architecture is also well-suited to be run as a real-time, interactive assignment system. PLASMA models each locomotive and train at a high level of detail, including operational issues such as consist-breakup, shop routing and the management of foreign power. Arrival and departure times are captured down to the minute. PLASMA can also model randomness in transit times, yard delays, train schedules and equipment failures.
This was a keynote presentation at the IIE Meeting in June, 2010 in Cancun. It provides an introduction to the challenge of how to collect information efficiently. The tutorial provides an overview of the different types of learning problems, along with a brief (and necessarily incomplete) description of the different types of policies that have been used for collecting information. The remainder of the tutorial focuses on the concept of the "knowledge gradient" which is a policy that collects information that provides the greatest value if you are going to do a single measurement.
I have given variations of this talk a number of times, often emphasizing different classes of applications.
SMART: A Stochastic, Multiscale model for the Analysis of energy Resources, Technology and policy (powerpoint format)
SMART uses approximate dynamic programming to model energy flows on an hourly basis over a multidecade horizon (almost 200,000 time periods), under different forms of uncertainty. The model currently captures about 15 types of energy supply (from nuclear, coal to wind, biomass and hydrogen) to satisfy about 10 types of energy demand (from residential and industrial electrical demand to home heating oil and natural gas). Hourly dispatch decisions are linked through the need to decide how much energy to store (right now we only capture a single water reservoir), which means we have to step forward in time to properly manage storage. The model can also handle different types of coarse-grained uncertainty (e.g. will there be a new carbon tax).
Presented (twice) to the future academicians workshop at Informs, this provides a perspective on how to handle work with industry through a university. There are unique opportunities taking on industrial problems, but also unique challenges.
We have worked for several years on a model for managing freight cars for a major railroad. The project started off looking like a textbook stochastic programming problem. By the end, the cars had evolved from a simple car-type attribute to a vector of attributes; the information process had morphed into a series of parallel information processes on customer orders, order attributes, car characteristics and transit times; we had learned about biases in customer orders, and we learned that there are some things that we will simply never learn.
Real Time Optimization for Real-World Problems (powerpoint presentation with sound!)
I have given this talk several times. This is a slightly shorter version that I gave to the Princeton Operations Research Society, where I experimented with recording the presentation. It worked better than expected, but please note that it is a 40 meg file.