Renewable Integration

Stochastic UC · Two-stage SP

Wind & Solar Uncertainty · Reserves · Curtailment

How do you commit thermal generators when tomorrow's wind is 10–20% uncertain and tomorrow's solar depends on cloud cover you can't predict? The classical unit commitment problem assumed a deterministic forecast; a world where 30% of electricity comes from wind and solar demands something new. Renewable integration is the stochastic, robust, or chance-constrained variant of UC that hedges against forecast uncertainty by committing more reserves, pre-positioning flexible capacity, and sometimes curtailing renewable output to maintain operability. The core paper — Morales, Conejo, Madsen, Pinson, Zugno (2014) — defines this as its own sub-field of energy OR.

The problem

Day-ahead scheduling under wind and solar uncertainty

Wind output at a typical onshore site has a 24-hour forecast error around 10% (Mean Absolute Percentage Error, MAPE, state-of-the-art numerical weather prediction). Solar PV under cloud cover has a much larger intra-day error (30%+). These forecast errors have a systematic impact on grid operations: a UC solved using the central forecast may under-commit on days the wind comes in low, requiring expensive emergency thermal response and occasionally load shedding. A UC solved using the worst-case forecast over- commits: thermal units run at low capacity factors, fuel is burned needlessly, and renewable output is curtailed to avoid violating minimum-generation constraints.

The core trade-off is cost vs reliability. Stochastic UC minimizes expected total cost over a scenario tree of possible renewable outputs. Robust UC minimizes the worst-case cost inside a budgeted uncertainty set (Bertsimas & Sim, 2004). Chance-constrained UC enforces that feasibility holds with probability at least $1-\alpha$ for some user-specified risk $\alpha$ (typically 5% or 1%). Each approach yields different schedules and different cost-reliability points on the efficient frontier.

Historical note
The foundational paper is Takriti, Birge & Long (1996) on stochastic UC. Bouffard & Galiana (2008) extended to wind-specific formulations. Papavasiliou & Oren (2013) demonstrated scenario-based stochastic UC at ISO scale (ISO-NE). Bertsimas et al. (2013) introduced adaptive robust UC for SCUC. Morales, Conejo, Madsen, Pinson & Zugno (2014) book Integrating Renewables in Electricity Markets is the textbook consolidation. Zheng, Wang & Liu (2015) is the definitive modern review of stochastic UC. Pinson et al. (2013) pioneered scenario-generation from ensemble NWP forecasts.

Modern deployment is increasingly hybrid: ISO-NE runs stochastic day-ahead UC with 5–10 scenarios; MISO uses a look-ahead dispatch with wind uncertainty; CAISO relies on fast-start units and 5-minute re-dispatch rather than explicit stochastic commitment. Academia is converging on distributionally robust UC (Wasserstein or moment- based ambiguity sets) as the next-generation framework.

Mathematical formulation

Two-stage stochastic UC with scenario tree

Notation

SymbolMeaningUnits
$\mathcal{T}$Hours in horizon—
$\mathcal{G}$Thermal generators—
$\mathcal{S}$Scenario set—
$\pi_s$Probability of scenario $s$pu
$\tilde{W}_{t,s}$Wind+solar output in hour $t$, scenario $s$MW
$u_{g,t}$Commitment (1st stage, scenario-independent){0,1}
$p_{g,t,s}$Dispatch (2nd stage)MW
$r^+_{g,t,s}, r^-_{g,t,s}$Up/down reserve deploymentMW
$w_{t,s}$Wind+solar used (after curtailment)MW
$u_{t,s}^{\mathrm{shed}}$Load shed penaltyMW

Two-stage stochastic UCMorales et al. (2014)

First-stage decisions (commitments) are here-and-now, made before uncertainty resolves. Second-stage decisions (dispatch, reserve deployment, curtailment) are wait-and-see, adapting to each scenario.

$$\min \; \sum_{g,t} SU_g \, v_{g,t} + \sum_{s \in \mathcal{S}} \pi_s \sum_{g,t} c_g(p_{g,t,s}) \cdot u_{g,t} + \rho \sum_{t,s} \pi_s \, u_{t,s}^{\mathrm{shed}} \qquad \text{(1)}$$

where the first two terms are from base UC (1st-stage startup + expected 2nd-stage dispatch) and the third penalizes unserved load at VOLL $\rho$.

Scenario-specific constraints

Non-anticipativity: commitments are identical across scenarios:

$$u_{g,t,s} = u_{g,t} \qquad \forall g, t, s \qquad \text{(2)}$$

Power balance with renewable output and curtailment:

$$\sum_{g \in \mathcal{G}} p_{g,t,s} + w_{t,s} + u_{t,s}^{\mathrm{shed}} = D_t \qquad \forall t, s \qquad \text{(3)}$$

where $w_{t,s} \le \tilde{W}_{t,s}$ allows curtailment (using less than available renewable). Dispatchability and ramps:

$$P_g^{\min} u_{g,t} \le p_{g,t,s} \le P_g^{\max} u_{g,t}, \quad |p_{g,t,s} - p_{g,t-1,s}| \le RU_g \qquad \text{(4)}$$

Minimum up/down times, start-up/shut-down, and reserve adequacy constraints are as in deterministic UC.

Adaptive robust UCBertsimas et al. (2013)

Rather than expected cost over scenarios, minimize the worst-case cost inside an uncertainty set:

$$\min_{u} \left[ \sum_{g,t} SU_g v_{g,t} + \max_{\tilde{W} \in \mathcal{W}} \; \min_{p, r, w} \sum_{g,t} c_g(p_{g,t}) \right] \qquad \text{(5)}$$

with $\mathcal{W}$ a budgeted box: $|\tilde{W}_t - \bar{W}_t| \le \Gamma_t \hat{W}_t$ with $\sum_t \Gamma_t \le \Gamma^{\mathrm{budget}}$. Solved by column-and-constraint generation (Zhao & Zeng, 2012). Less conservative than full worst-case robust because operating decisions adapt to realizations.

Chance-constrained UC

Replace worst-case with probabilistic guarantee:

$$\mathbb{P}\left( \sum_g p_{g,t} + \tilde{W}_t \ge D_t \right) \ge 1 - \alpha \qquad \forall t \qquad \text{(6)}$$

Under Gaussian renewable errors, (6) becomes a second-order cone. Without distributional assumptions, scenario approximation replaces with a sample of constraints.

Scenario generation & reduction

Key practical steps:

  • Generation: sample from ensemble NWP forecasts; use copulas for spatial correlation; trajectory-based (not marginal) scenarios.
  • Reduction: fast-forward algorithm (Gröwe-Kuska et al., 2003) reduces 1000-scenario tree to 20–50 representative scenarios.
  • Quality: validate with out-of-sample evaluation; production systems use 20–100 scenarios.

Real-world data

NREL Wind Integration Toolkit & NSRDB

WIND Toolkit and NSRDB provide high-resolution historical wind and solar production datasets across the continental US. Essential input for scenario generation and backcasting renewable- integration studies.

ENTSO-E Transparency Platform

ENTSO-E publishes hourly actual and forecast renewable generation for every European country. European stochastic UC research heavily relies on these data for realistic forecast- error modeling.

Illustrative stochastic UC (this page)

The interactive solver handles a 24-hour 6-thermal-unit + wind-integrated dispatch with 10 wind scenarios sampled from a user-configurable forecast distribution. Visualizes the uncertainty-driven reserve and the cost curve as a function of renewable penetration.

Interactive solver

10-scenario stochastic UC with wind forecast uncertainty

Wind forecast

400
15%
12%
3000
Adjust parameters and press Solve.

Wind forecast scenarios & dispatch

Wind forecast fan (10 scenarios, gold mean) over 24 hours
Stacked generation under mean scenario: wind (blue), thermal (orange/red), load (white)

Solution interpretation

The forecast fan is the key epistemological object. It shows the full distribution of possible wind outputs the solver must hedge against — not just the mean forecast. A narrow fan (5% std dev) means the operator can commit close to the deterministic-optimal schedule; a wide fan (30% std dev) forces extra thermal commitment as insurance against low-wind scenarios.

Higher reserves come from two sources in modern operations: (1) committing more thermal units at part-load, and (2) holding storage/DR in reserve. Both are visible in the dispatch chart. The unit commitment shifts toward more flexible gas combined-cycle rather than inflexible coal as renewable penetration rises.

Curtailment (renewable output not used) appears during hours of strong wind + low load (mid-day in winter, or overnight in spring). A cost-optimal solution curtails 1–3% of renewable output at 20% penetration, 10–20% at 50%+ penetration, absent large storage or long-distance transmission. This is the economic motivation for storage and transmission expansion.

The expected-cost vs deterministic-cost gap is the Value of Stochastic Solution (VSS) — how much money the operator saves by modeling uncertainty explicitly. Typical VSS is 1–5% of system cost, but can exceed 10% in high-renewables systems. For a continental grid that's billions of dollars.

Extensions & variants

Stochastic UC with reserves

Extends deterministic UC by explicitly co-optimizing energy and reserve across scenarios. Captures reserve scarcity pricing; used by ISO-NE for day-ahead clearing.

Refs: Ela & O'Malley (2012); Papavasiliou, Oren & O'Neill (2011).

Look-ahead dispatch (LMED)

Multi-period real-time re-dispatch with 1-2 hour horizon and wind uncertainty; used in MISO's real-time market. Simpler than full stochastic UC but captures ramp dynamics critical in high-renewables systems.

Refs: Chen et al. (2019); MISO market documentation.

Distributionally robust UC

Optimizes against worst distribution in Wasserstein or moment-based ambiguity set. Bridges stochastic and robust: reduces conservatism of robust while tolerating distribution misspecification. Active research frontier (2018+).

Refs: Zhao & Guan (2018); Duan et al. (2018).

Storage-coupled UC

Co-commits thermal units and storage across scenarios. Storage SOC dynamics provide intertemporal flexibility that substitutes for some reserve. Essential for high-RE systems.

Refs: Pozo et al. (2014); Bhattacharya, Kar & Guan (2019).

Ramp-rate constrained dispatch

High renewable variability imposes steep net-load ramps (CAISO's duck curve). Explicit ramp-rate pricing mechanisms (FERC Order 825, 2016 ramp products) modify ED and UC to co-optimize ramp capability with energy.

Refs: Wang & Hobbs (2016); CAISO flexible ramping product.

Curtailment and priority dispatch

Must-take policies (Europe feed-in tariffs, early US RPS) dispatch renewables ahead of thermal; market design post-Paris tends toward economic curtailment where renewables bid zero and the LMP determines curtailment outcomes.

Refs: Bird et al. (2014); Golden & Paulos (2015).

Key references

[1]
Morales, J. M., Conejo, A. J., Madsen, H., Pinson, P., & Zugno, M. (2014).
Integrating Renewables in Electricity Markets: Operational Problems.
Springer, International Series in Operations Research & Management Science, 205. doi:10.1007/978-1-4614-9411-9
[2]
Zheng, Q. P., Wang, J., & Liu, A. L. (2015).
“Stochastic optimization for unit commitment — a review.”
IEEE Transactions on Power Systems, 30(4), 1913–1924. doi:10.1109/TPWRS.2014.2355204
[3]
Bertsimas, D., Litvinov, E., Sun, X. A., Zhao, J., & Zheng, T. (2013).
“Adaptive robust optimization for the security-constrained unit commitment problem.”
IEEE Transactions on Power Systems, 28(1), 52–63. doi:10.1109/TPWRS.2012.2205021
[4]
Papavasiliou, A., & Oren, S. S. (2013).
“Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network.”
Operations Research, 61(3), 578–592. doi:10.1287/opre.2013.1174
[5]
Bouffard, F., & Galiana, F. D. (2008).
“Stochastic security for operations planning with significant wind power generation.”
IEEE Transactions on Power Systems, 23(2), 306–316. doi:10.1109/TPWRS.2008.919318
[6]
Takriti, S., Birge, J. R., & Long, E. (1996).
“A stochastic model for the unit commitment problem.”
IEEE Transactions on Power Systems, 11(3), 1497–1508. doi:10.1109/59.535691
[7]
Pinson, P., Madsen, H., Nielsen, H. A., Papaefthymiou, G., & Klöckl, B. (2013).
“From probabilistic forecasts to statistical scenarios of short-term wind power production.”
Wind Energy, 12(1), 51–62. doi:10.1002/we.284
[8]
Gröwe-Kuska, N., Heitsch, H., & Römisch, W. (2003).
“Scenario reduction and scenario tree construction for power management problems.”
IEEE Bologna Power Tech Conference Proceedings. doi:10.1109/PTC.2003.1304444
[9]
Zhao, L., & Zeng, B. (2012).
“Robust unit commitment problem with demand response and wind energy.”
2012 IEEE PES General Meeting. doi:10.1109/PESGM.2012.6344860
[10]
Zhao, C., & Guan, Y. (2018).
“Data-driven risk-averse two-stage stochastic program with $\zeta$-structure probability metrics.”
Optimization Online. optimization-online.org
In-browser solver samples 10 wind scenarios and solves a simplified two-stage problem with greedy thermal commitment. Production tools (ISO-NE, Papavasiliou group) handle 50-100 scenarios with Benders decomposition.