We Tried to Make Fabric Leak Money Across Clouds. It Refused.

Microsoft Fabric · Direct Lake · Cloud Economics

Every architect who has ever drawn a OneLake shortcut from Microsoft Fabric to an Amazon S3 bucket has felt the same quiet anxiety. The arrow on the whiteboard looks innocent. But that arrow crosses a cloud boundary, and cloud boundaries have toll booths. Cross-cloud egress is the line item nobody budgets for and everybody discovers later.

So we stopped speculating and built the thing. A real AWS account. A real Iceberg table sitting in S3. A real OneLake shortcut pointing at it. A real Direct Lake semantic model on top. And — the part that makes this more than a thought experiment — S3 server access logs switched on, so AWS itself would tell us, byte for byte, every time Fabric reached across the ocean to pull data.

The logs don't have opinions. They don't care about marketing claims or architecture diagrams. They just record what left the bucket. That made them the perfect lie detector. What they told us was more interesting — and more nuanced — than we expected.

The setup: a lie detector for egress

Here's the rig. We generated a synthetic sales table with deliberately nasty data — a high-cardinality amount column engineered to defeat dictionary compression, so the Parquet files would be big enough to actually move real bytes. We wrote it to S3 as an Iceberg V2 table, which OneLake virtualizes as Delta. Then we hung a Direct Lake semantic model off a OneLake shortcut and pointed two engines at it: Direct Lake on OneLake (no DirectQuery fallback) and Direct Lake on SQL (fallback allowed).

To know exactly which read belonged to which test, we planted sentinels — tiny marker objects PUT to S3 at the start and end of every single phase. Because those writes show up in the same access log as the data reads, they became precise timestamps we could bracket every measurement with. Cold run, warm run, repeat — each one wrapped in its own START and END beacon.

And we made one methodological decision early that saved us from publishing nonsense: we counted bytes, not requests.

Mistake #1: the "6× re-read" that wasn't

Our first instinct was to count GET requests. Big mistake. The logs showed what looked like Fabric reading the same files six times over — an egress horror story in the making. We almost wrote it up that way.

Then we looked closer. Roughly 77% of those GETs were partial, byte-range reads — Fabric pulling individual column chunks out of a Parquet file, exactly as a columnar engine should. Six "reads" of a file weren't six copies crossing the wire. They were six small slices, together adding up to less than one full file. Count requests and you invent a scandal. Count bytes and the scandal evaporates.

Mistake #2: the button that bricked the model

To test a truly "cold" query we needed to evict Direct Lake's in-memory data. The obvious lever was to clear the model. So we did — and promptly walked into a wall of table not refreshed, fallback disabled errors.

It turns out that on Direct Lake on OneLake, clearing values doesn't just cool the cache — it unprocesses the model. The table goes from warm to non-existent, and with no DirectQuery fallback to lean on, queries simply fail. That's not a cold cache; that's a broken model. We backed out and switched to evicting only the in-memory column segments while keeping the model queryable — the difference between letting the engine forget the data and ripping the table out from under it.

The signal that actually told the truth: temperature

How do you prove a query ran cold? Residency counts — how many column segments are in memory — felt like the obvious tell, but they were noisy. The very act of probing residency pages a segment or two into memory (yes, the observer changes the experiment).

The reliable signal turned out to be segment temperature — Direct Lake's internal measure of how recently and heavily a column has been touched. Watch the temperature sum collapse before a cold run and climb back up afterward, and you know, unambiguously, that the engine genuinely went to storage rather than serving from memory. That became our ground truth for "cold," sitting right alongside the S3 logs for "egress."

The result that matters: warm reads are free

With the methodology finally honest, the numbers fell out cleanly. Here's the money table from the controlled run — egress attributed to each phase, with caching toggled off and on:

Read the warm rows twice, because that's the headline: once Direct Lake has the data resident, repeat queries generated exactly zero cross-cloud egress. Not "a little." Not "negligible." Zero bytes left the bucket. Your dashboard can be refreshed a hundred times by a hundred users and AWS sees nothing.

The cold rows carry the other half of the story. Egress doesn't happen when a user runs a query — it happens at framing, the moment Direct Lake loads columns into memory. In our first run, roughly 90% of all egress for an entire test arrived in a single framing burst. Egress is a loading-time cost, not a query-time cost. That single reframe is your toll booth; everything after it rides free.

The plot twist: we couldn't force a cache miss

Here's where it got genuinely strange. We wanted to capture the miss — the first cold read that fetches from S3 and populates the cache — in isolation. So we did everything you'd think would force one. We turned OneLake shortcut caching off, then on. We cleared the in-memory cache from DAX Studio. We even pressed Fabric's "Reset cache" button, the one that promises to remove all cached shortcut data from the workspace.

Then we pulled an entire day of S3 logs to see the miss land. It never came. Across the whole day, AWS recorded:

• 107,863 HEAD requests — OneLake constantly asking "has this file changed?"
• 21,532 LIST requests — directory scans
• 24 GET requests — every single one of them our own sentinel markers

Zero bytes of actual table data downloaded. Not one.

Sit with that. A massive storm of metadata revalidation — HEAD after HEAD after HEAD — and not a single byte of real data crossing the wire. Even after we forced the engine to reload columns from cold (temperature confirmed it genuinely went cold), the bytes came from OneLake's own copy, not from S3.

Which leads to a conclusion that's either reassuring or frustrating depending on what you're trying to do: you cannot reliably force a cross-cloud miss by evicting caches. The only deterministic way to make OneLake go back to S3 is to change the bytes — write new files to the source. The cache is keyed on what the data is, not on whether you asked it to forget.

Now the cold water: what we did NOT prove

Good engineering writing is honest about its own limits, so here are ours — plainly.

This was Direct Lake, not a pure shortcut-cache test. Direct Lake's in-memory (VertiPaq) layer sits in front of the OneLake shortcut cache. On a warm query, we genuinely cannot tell from the outside whether "zero egress" came from data resident in RAM or from the shortcut file cache — both would produce the same silent S3 log. We proved the combined system leaks nothing on warm reads. We did not cleanly isolate the shortcut cache on its own.

We never captured a clean miss-to-hit transition. Because we couldn't force eviction, every phase we measured was already a cache hit. The cold populate event — the one read that actually pays the toll — happened before our measured windows. We inferred it from the OFF-cold numbers, but we didn't film it happening in a controlled single shot.

It's one table, one shape, one region. A single synthetic table on a single account in a single region is a data point, not a universe. File sizes above 1 GB aren't cached at all. Wider tables, different partitioning, cross-region hops, and concurrency at scale could all move these numbers. Treat our figures as directional, not as a published SLA.

Log delivery lags, and that bites. S3 access logs arrive hours — sometimes a day — after the event, and they're named by delivery time, not event time. More than once we nearly trusted an incomplete picture. Any test like this is only as honest as your verification that the logs for your window have actually landed.

So what should you actually do?

If you're running steady-state dashboards → relax. Repeat reads are free across the cloud boundary. Size your worry around framing events — refreshes, reframes, capacity restarts — not around how many people open the report. Schedule and batch your refreshes; that's where the bytes (and the dollars) actually are.

If your egress bill matters → budget the cold path, not the warm one. Model your cost as: bytes-per-framing × framing-frequency. Keep hot Parquet files under 1 GB so they stay cacheable. Lengthen the shortcut cache retention if your source is slow-changing. And remember the warm path contributes essentially nothing — stop optimizing it.

If you're benchmarking this yourself → steal our methodology, not our numbers. Count bytes, not requests. Use source-side access logs as ground truth, not engine-reported stats. Bracket every phase with sentinels. Trust temperature over residency counts for proving cold. And verify your logs have fully landed before you believe a single figure.

We set out to catch Microsoft Fabric quietly draining an AWS account. Instead we caught it doing something almost boringly sensible: paying the toll once, then refusing to pay it again — so persistently that we couldn't make it pay twice even when we tried. The egress bill we feared mostly never came. The real lesson isn't "shortcuts are free." It's that the cost lives in one specific moment — loading — and once you know that, you can architect around it with your eyes open.

Methodology, raw phase logs, and the S3 access-log parsers behind every number in this article are available on request.