NYISO virtual bidding, storage value, and grid security on six months of market data

The question this note answers is whether a simple virtual bid on NYISO's day-ahead versus real-time zone prices was profitable under walk-forward discipline on H1 2026, and what else the same panel implies for storage value and grid security. NYISO is a useful single-ISO case: LMP is published with energy, congestion, and loss components in the same file; history is free and deep; and the load file carries an explicit standard-time / daylight-time column, which matters once a panel has to survive a clock change without mislabeling an hour.

Mean day-ahead and real-time prices by zone over the full six months, highest to lowest:

N.Y.C. is the most expensive zone on average, $15.76/MWh above WEST, with positive mean congestion day-ahead and real-time (+$1.31, +$2.19/MWh). WEST is strongly negative (-$3.34/MWh day-ahead). In the sign convention used here, positive congestion means a binding transmission constraint is raising the nodal price - the signature of an import-constrained sink. That convention was recovered from the raw NYISO file: undoing the published-sign flip disperses the implied system reference price by more than $10 against a 3-cent rounding tolerance. Four of the five most expensive zones sit downstate; the five cheapest sit upstate. Long Island is the exception: geographically downstate, but with negative mean congestion, so its constraints are not explained by distance from Manhattan alone.

Panel: day-ahead and real-time locational marginal prices (energy, congestion, loss) and load for all 11 NYISO internal load zones, 2026-01-01 through 2026-06-30. After joining day-ahead hours to duration-weighted integrated real-time prices¹, the panel has 47,773 zone-hours (65,145 day-ahead hours per zone-set before join filters; zero dropped for missing real-time or load; minimum coverage 1.0 on retained hours). External proxy PTIDs (HQ, NPX, PJM, OH) are excluded.

Weather for the same zones and window: 96,096 rows of ERA5-family reanalysis actuals plus a two-day-ahead archived forecast, both at full coverage. Forecast versus realized temperature differs by a mean 1.64°C (median 1.2°C, max 15.7°C) across the aligned panel - a real forecast, not the outcome relabeled. The two-day lead is the shortest whose issue time can be shown to precede NYISO's day-ahead bid deadline under every model run hour².

All results below are wholesale. No retail tariffs. Transmission topology and generator fleet data are not in the lakehouse (ISOs do not publish line parameters in open form); Section 9 is therefore an illustrative network with a real solver, not a claim about NYISO topology.

The DART spread is DA - RT: the payoff to a virtual supply (INC) that sells day-ahead and buys back real-time. A DEC earns the opposite. No physical delivery. Across the full 47,773 zone-hours the spread averages +$6.29/MWh (median +$2.42, std $70.45; 59.6% of hours DA > RT). The mean is 2.6 times the median - the edge is in a thin right tail, not a typical hour.

That full-sample mean is not uniform across the window. Monthly unconditional INC at a $0.30/MWh fee:

January alone accounts for most of the full-sample edge. On the full panel, always-INC at $0.30/MWh earns +$5.99/MWh. A $0.30 fee against a $6.29 mean is a small haircut, not the reason a trade would fail.

Walk-forward evaluation is the right test for a strategy that would be retrained over time. Expanding-window folds with a minimum train of ~28 zone-days (feature warmup for the 28-day rolling terms) leave 40,381 zone-hours out of sample. On that OOS set the mean DART is +$2.39/MWh (median +$2.22, 59.4% positive). The train-only block that never enters a test fold still carries a much higher mean (+$27.62), because it is almost entirely early January. Strategy P&L below is always measured on the OOS window, never on the full-sample mean.

An earlier revision of this study used min_train = 35% of the panel (~63 calendar days). That permanently excluded all of January from every test fold, left an OOS mean of -$0.87/MWh, and made every rule look broken next to the full-sample +$6.29 figure. That was an evaluation defect, not a market fact: once late January is allowed into OOS, unconditional INC is profitable.

Two further caveats on inference. Zone-hours are not independent: mean pairwise cross-zone DART correlation is about 0.95, because the system energy component is common. Pooled t-statistics over 11 zones overstate independent sample size by roughly a factor of 11; mean P&L per MWh remains a valid per-zone economic figure. And a six-month window with one extreme winter month is a thin basis for any claim about long-run virtual profitability.

Features respect NYISO's bid deadline: a bid for operating day D is due 05:00 Eastern on D-1, before D-1 real-time has finished, so the most recent complete real-time observation is D-2. The primary feature set is small on purpose - 27 columns over roughly 4,300 hours per zone - with lagged price and load terms as zone-hour deviations from a trailing 28-day mean, calendar terms, and nine weather terms: two-day-ahead forecast temperature, wind speed and cloud cover with the temperature degree-hour transforms, plus realized temperature, wind speed, dewpoint, and cloud cover at the two-day lag, each as a deviation from its own trailing 28-day mean. Fuel-mix and Henry Hub features were also tested after detrending; they did not improve OOS economics.

Models: quantile (median) regression, ridge with nested alpha selection, Huber, plus two fit-free baselines (trailing median of DART at the same zone-hour over the prior 28 admissible days, and always-INC). Walk-forward with purged and embargoed folds³, six folds. All models are scored on the same 40,381 OOS hours; a NaN prediction is treated as a flat (no trade), not as a dropped row. Directional accuracy against the majority-class base rate (~59.5%):

None of the fitted models beats always-INC on direction or on dollars. The trailing median has the highest Spearman IC (+0.13) among active rules but still underperforms always-INC on P&L: rank correlation without correct sizing of the large moves is not an edge. The most train/test-displaced feature is a weather degree-hour term in five of six walk-forward folds (standardized displacement 1.36-2.20) and the rolling 28-day DART mean in the earliest, thinnest-training fold (7.25); that is expected when folds span seasons, and is a reason to discount coefficients out of season rather than treat them as stable.

The economically best simple rule on this OOS window is always-INC. Fee sweep from $0.00 to $1.00 per MWh of virtual volume (OOS, always-INC, 40,381 zone-hours):

The $0.30 central assumption is a modeling choice, not a measured tariff: published NYISO rate-schedule components are roughly $0.10-0.20/MWh; the rest is execution cost that is desk-specific and not public. At that fee, OOS always-INC is +$2.09/MWh. Trailing-median timing is +$0.99/MWh - still positive, but worse than always taking the long-run side. Fitted models land around -$0.82 to -$1.06/MWh: they switch sides enough to destroy the unconditional edge without capturing a better one.

Per-zone OOS always-INC at $0.30 is positive everywhere: WEST +$3.73/MWh, LONGIL +$0.85/MWh. The edge is not one lucky zone. Convergence-bidding literature has often found that measured profits shrink to the size of transaction costs once costs are applied carefully (Jha & Wolak 2019 on CAISO; Birge et al. 2020; Hogan 2016)⁴. On this NYISO window and fee assumption, unconditional INC still clears costs; the open question is whether the January-heavy edge persists outside H1 2026, not whether a $0.30 fee alone kills a $6 mean.

Two rolling, point-in-time 95% VaR models - closed-form Gaussian and empirical-quantile historical, each on a trailing 200-observation window using only prior data - backtested on the N.Y.C. zone DART series (4,143 observations):

Both fail the Kupiec proportion-of-failures test at 5%. Gaussian VaR is too conservative (3.9% breaches vs. 5.0% target), which wastes capital. Historical VaR overshoots the other way (6.2%). Both sit about 1.1-1.2 percentage points from target on opposite sides; neither is an acceptable default for this series without adjustment.

Perfect-foresight arbitrage value for a 1 MW battery, re-optimized each operating day with cyclic state of charge and 85% round-trip efficiency, summed over the six-month window. Upper bound on revenue, not a realizable strategy:

Moving from 2h to 8h multiplies value by about 1.6-1.8x in every zone. N.Y.C. at 4h ($41.4k/MW) is only 1.2x WEST at 4h ($34.2k/MW). Duration dominates location under perfect foresight on this window. Literature estimates of forecast-based capture rates (CAISO DMM 2023; Sioshansi et al. 2009) are typically 60-80% of this upper bound; that range is external, not computed here.

A fixed calendar is a useful foil. Charging 10:00-18:00 and discharging the evening peak - a daytime-solar pattern built for markets like California - captures 0.3-10.8% of perfect foresight on N.Y.C. prices. N.Y.C.'s cheapest real hours are overnight (2:00-5:00, about $58-60/MWh), not midday ($67-76/MWh). A calendar fitted to that shape (charge overnight, discharge the 5 p.m. peak) reaches 23.9-34.8% of perfect foresight depending on duration, still well below a price-responsive strategy.

Section 4's result - no fitted linear model beats always-INC on this panel - is not rescued by more architecture. On roughly 4,300 hours per zone, deep sequence models (LSTM, small Transformer, TFT-style attention, NHiTS, PatchTST) and a geographic-proxy graph network were run under the same point-in-time lag rule. Directional accuracies on the N.Y.C. series: LSTM 59.5%, NHiTS 59.6% (majority-class collapse), Transformer 49.5%, TFT-style 51.9%, PatchTST 46.3%, GNN 55.0%, against a 59.6% base rate. None improves on the base rate in a way that would change Section 5. With this sample size that is the expected outcome: capacity is not the binding constraint; information and regime stability are.

Two secondary tools are more useful as diagnostics than as forecasters. A two-state HMM on N.Y.C. DART finds a calm regime (mean +$2.77/MWh, std $11.10, 87% of hours) and a volatile regime (mean +$27.85, std $199, 13% of hours), with the volatile share concentrated in January (31% of January hours) and declining through spring - the same monthly pattern as Section 3. A Student-t copula on cross-zone pairs fits better than a Gaussian at every pair tested, with degrees of freedom pinned at the edge of the search range: tail dependence across zones is extreme.

Locational prices are the dual variables on nodal balance under transmission limits. On an illustrative 3-bus network (cheap $20/MWh generation at the slack bus, $60/MWh local backup, 150 MW load at each of the other two buses) - not real NYISO topology - unconstrained least-cost dispatch violates N-1: losing either of two lines pushes the survivor to 136.4% of emergency rating. Security-constrained dispatch removes the violation. Cost of security as a function of emergency rating:

The premium is exactly $0 at 300 MW because that is the post-contingency flow the unconstrained dispatch already produces on the surviving line; the security constraint stops binding and the secured solution collapses onto the unconstrained one. Locational prices under the secured dispatch: $20 at the slack bus, $60 at both load buses - verified by finite-difference re-solves (raise load 1 MWh, re-solve, read the cost change) and by reduction of SCOPF to plain DC-OPF when the contingency set is empty. Congestion rent is $8,800, checked against its economic definition (load payments minus generator receipts) independently of the module's internal accounting.

Unit commitment on a two-unit, three-period example recovers a known non-monotonicity: a midday demand trough below a baseload unit's minimum load forces a shutdown and a later restart, so raising midday demand from 30 MW to 60 MW can cut total system cost ($10,450 for 330 MWh vs. $7,200 for 360 MWh). Both the MILP and a brute-force enumeration over on/off schedules agree. Continuous relaxation of the same problem understates true cost by 34% and, if rounded, recovers the wrong commitment schedule.

• One ISO, six months. January 2026 dominates the full-sample DART mean. OOS always-INC profitability on this window does not by itself establish a multi-year edge.
• Grid topology and generator fleet are not real. Section 9 uses an illustrative network. The GNN in Section 8 uses geographic distance as a graph proxy, not transmission topology.
• SCOPF is lossless DC without reliability-reserve requirements. Reserves exist in the unit-commitment module but are not wired into the SCOPF run shown here.
• Weather feeds the DART feature set only. It does not yet enter grid, risk, or storage modules.
• Deep sequence and graph models do not beat always-INC on this panel. That is a statement about this sample and feature set, not a general claim about those architectures.
• Weather is ERA5-family reanalysis, not airport METAR observations.
• Wholesale only. No retail pricing.
• Zone-hour pooling overstates independent sample size for t-statistics (cross-zone DART correlation ~0.95).

Independent research by Shubham Gaur. Questions and corrections welcome.