Updated on Mar 31 with Links, more background material and detail.

 

The most talked about improvement in Windows Vista seems to be Aero - the composition based desktop which people either seem to love or hate. This is followed by the UAC which again seems to enjoy a love-hate relationship. However, there is a lot of technology shipping in Windows Vista which anticipates long term changes in storage hardware and helps the operating system be more peformant in light of those changes. This article gives an overview of some of these enhancements; the goal here is not be exhaustive and talk about every improvement, but to provide insight into why these are so important and their long term impact on PC computing.

 

Background

Just to establish the vocabulary, and ensure that there is no ambiguity, I will define some terms here:  

1)      Primary Storage – This is connected to the CPU and is volatile. There are three kinds of primary storage:

a)      CPU Registers: Internal to the CPU – this is the fastest storage

b)      Main Memory: This contains the programs that are currently running and the data the programs are operating on. This is typically implemented as solid state volatile RAM connected to the CPU via a memory bus and a data bus.

c)       Cache Memory: Modern CPUs are too fast for main memory so a cache is often used to prevent the CPU from waiting on main memory. This cache memory is quite small (cache size to RAM size ratio could be anywhere from 1:1 to 1:64 or even more), but is much faster than the main memory. Data is duplicated from Main memory into the Cache. Typically Cache memory is implemented as a hierarchy – primary cache (L1) being closer to the CPU and smaller, faster than the secondary caches (L2, L3)

So you can actually imagine the primary storage to be a hierarchy with the fastest storage being closest to the CPU (in fact there could be hierarchy even within the CPU registers) and storage getting slower and bigger as we move away from the CPU.  

2)      Secondary Storage – This is used for persisting data and is therefore non-volatile. Secondary storage is connected to the CPU using I/O channels. This is mostly implemented as magnetic disks with spinning platters or more recently using solid state drives using non volatile memory such as Flash.

 The key trends in storage hardware are:

1)      Price of flash storage is dropping very fast, falling faster than expected at around 40% / annum. Compare this with an average of 32% for DRAM over the last 30 years or so.

2)      Magnetic disk drives storage is getting bigger and cheaper, but so is flash memory – in fact the trend is almost parallel. There is however more to understanding the size-price trends for both. I’d recommend reading Flash Memory vs. HDD – Who Will Win for more detail. This article is dated, but has a comprehensive discussion.

3)      The disk has been the slowest component in the hardware stack over the past 15 years, and it is unlikely that disk RPM would increase beyond 20,000. So with the introduction of multi-core, the rate of change of CPU power will outpace rate of change of storage speed even more.

What these changes mean is that the slowest component in the system – the disk drive - would get slower and slower with regard to the CPU, so a powerful CPU would need to keep waiting on I/O. This is obviously not acceptable and we have to find a way around this.  The solution is to move data from the slowest storage to the fastest storage in such a way that we can let the CPU work mostly on faster storage, and avoid going to the slower storage as far as possible. This is done at two levels:

1)      At one level we try and move the data from the secondary storage into the primary storage in as optimum a way as possible.

2)      On the other level, we try and move data from the main memory into the CPU caches and the CPU registers.

 

In this article we are going to focus on the optimization mechanisms to point 1), and specifically on moving the data from the disk into the RAM. All of these mechanisms are about buffering against disk I/O and using a faster, cheaper buffer for random I/O.

 

Demand Paging

There has always been this trade-off between RAM and disk – RAM is faster and expensive and volatile, and the disk is slower and cheaper and persistent. Traditionally, operating systems have been able to work out the trade off by a scheme called demand paging. The idea of demand paging comes with virtual memory, and is simple to understand – there is limited main memory, so the disk is used as an extended store, and the OS abstracts away the memory by giving each process an address space. Based on usage of the address space, the OS allocates memory from the combined disk + main memory store to the process. A page is a unit of allocation, and depending on when that part of memory was last used, the page could be in main memory or on the disk. So when an application needs to make use of data which is held in a page, if that page is not already in physical memory, it would be read off from the disk and loaded into main memory. This is called demand paging since the pages are loaded into the main memory on demand.

 

Pre-Fetch

The trouble with demand paging is that disk IO is terribly slow, so when the page is read from the disk instead of RAM, there is a huge performance hit. If this I/O cost could be reduced, performance would improve significantly. An interesting observation which can help here is that random IO is slower than sequential IO. Tests reveal that depending on disk and workload, sequential IO can be up to 30-100 times faster than random IO. So if the OS was to do more sequential IO than random IO for demand paging, performance would improve in a big way.  With this goal, starting from Windows XP, the OS started identifying patterns in application usage and would try and group together the files on the disk which would be used frequently.  This would convert a lot of random IO into sequential IO and make frequently used files and applications load faster. This feature in Windows XP was called Pre-Fetch.

 

Super-Fetch

While Pre-Fetch improves things by converting random I/O into sequential I/O, this can get even better if we were not to do any I/O at all. If the OS could somehow predict which pages would need to be read from the disk ahead of time and pre-load them into main memory, when the time comes to use them, there would be zero disk I/O cost.  This feature in Windows Vista is called SuperFetch – Vista speculatively loads code and data likely to be used by a user ahead of time from the disk into memory so that an application gets launched (or data is read by the application) “warm out of memory” instead of “cold out of disk”.

A simple example of this is the scenario where you have a couple of applications running (say Outlook and IE), and you go for lunch. While you are away, a background program (search indexer, defragmenter, anti-virus, etc.) runs, consumes a lot of memory and releases it. To get this program to run, the Virtual Memory Manager in Windows would have to evict the pages of your foreground apps to disk. So when you come back and start reusing your apps, you would find them sluggish. This is where SuperFetch comes in – once physical memory becomes available, it would ask the memory manager to reload the memory with the data that was recently paged to the disk. So when you start using the system with SuperFetch running, you would not have to wait for I/O – SuperFetch would have loaded the pages ahead of time.

Now given the fact that the size of disk drives is about 20 - 100 times the size of memory (I am writing this article on a 2 GB RAM / 60 GB HDD machine), SuperFetch needs to prioritize what data to pre-load, and what to leave behind on the disk, and also which activities are more important than others.  For this purpose, Vista ships with a set of heuristics that provide the prioritization, and once it starts getting used, Vista does a lot of data gathering and analysis so that it is able to recognize the user’s usage patterns and anticipate when you would need what data.  As the user keeps using her system, the data Vista gathers becomes more and more tuned to the usage and system performance improves. While the performance would improve for any scenario, some scenarios are going to benefit more than others.  The biggest impact of SuperFetch is seen today in application launching and resuming from hibernation / suspension.

So to summarize, SuperFetching is different from demand paging in the sense that the OS is not waiting for the demand to occur – it has been recognizing patterns in demand over a period of time, and now pages based on anticipated need rather than demand.  As the gap between disk IO and processing power increases even more, making the data available in memory ahead of time would become even more important going forward.

 

ReadyBoost

ReadyBoost tries to solve a simple problem: RAM is great for disk caching, but adding more memory to a system is both difficult and expensive.  So what ReadyBoost allows you to do is to use a fast flash-based device (USB Stick, SD cards, Compact Flash, etc.) as an extension of physical memory which can be used for disk caching.  This works because flash is faster than disk for random I/O (and most I/O is random), and Vista performs a check on a USB disk on insertion, and if it meets the perf criteria (2.5 MB/s for random 4K reads and 1.75 MB/s for random 512K/s writes) and the minimum space requirements (230 MB free space), will let a user use it as an extension of physical memory for disk caching.  ReadyBoost is therefore ideal as a supplementary cache for SuperFetch, and in fact was developed by the same team.

There are a number of optimizations used:

1)      Readyboost recognizes large sequential reads and lets it be serviced by the hard disk instead of the flash since hard disks are faster at sequential I/O

2)      The data is compressed by a factor of ½ before storing it to derive the maximum out of the storage

3)      The data is encrypted with a randomly per boot generated key using AES 128

From a bigger picture perspective, as the prices of flash continue to fall, we can expect that within 2007, PCs would start shipping with in-built fast-flash storages. (Intel recently announced a solid state drive product line aimed at embedded market and at motherboards in servers, PCs and notebooks) These storages would actually be much bigger and faster than the typical USB drive that you would plug into Vista today. The OS would recognize this as non-removable, non-volatile fast flash storage and would automatically use it for disk caching.  Since this storage would be non-removable, the encryption that happens today would not be required and would make the system even faster.

 

ReadyDrive

While ReadyBoost looks at the idea of Flash augmenting RAM in terms of increasing its size so that it can do disk caching, ReadyDrive is about Flash augmenting the magnetic disk by providing a solid state buffer.  It is a technology that ships with Vista that allows the OS to use new types of disks in the market which are called Hybrid Disks. These disks have an integrated non-volatile solid state flash memory (NVRAM) in addition to the standard rotating media. One could think of this as being a cache on the disk itself.  This cache buffers disk I/O and allows the disk to stay spun down for a longer time, thus improving perf as well as battery life.  The key areas which benefit from ReadyDrive are disk contention scenarios, booting, and resumption from hibernation.

Ready-Drive controls the contents of the NV Cache and divides it into multiple parts. These are:

1)      Write Cache (32 MB): Buffers write requests to the disk, allowing the disk to stay spun down longer

2)      OEM-pinned data (1 MB+): Used by OEMs to pin key data for quickly launching preferred user experiences or Windows HotStart experiences

3)      SuperFetch Pinned Data (remainder): Used by SuperFetch

These drives are not far off - in fact, Samsung has recently released its Hybrid Drive to computer makers. With the falling prices of flash, it is obvious that over a period of time all disks will ship with a NVRAM cache and that the ratio of the cache to disk size would increase in the favor of the flash based cache.  ReadyDrive will therefore become more and more important going forward. In fact in near future, one can expect solid state disks to become the norm as compared to spinning disks – this trend has been there for a pretty long time, and is now coming of age.

 

I/O Prioritization

One can buffer against disk access using all the mechanisms discussed above, but at the end of the day, disk I/O still needs to happen.  From an overall system perspective, there is therefore a big need to be optimum about the I/O itself.  The I/O Prioritization changes in Windows Vista seek to do exactly this.

Today the priority of an IO request from a background process is treated at par with the one from an application which may be in the foreground. If we consider applications like disk defragmenters and anti-malware agents, which do a lot of I/O requests as compared to a typical foreground application like Microsoft Word, this treatment at par can slow down the foreground applications considerably. So in Windows Vista, an application which would mostly run in the background can choose to specify that its I/O request should be given a low priority. This is done by calling SetPriorityClass and SetThreadPriority and specifying background mode.

What this does is to embed the low priority setting in the flags field of each I/O request packet (IRP) originating from that process, and this can be read by a driver by calling IoGetPriorityHint. This ensures that I/O – a costly operation – gets prioritized correctly across the system. I/O intensive background tasks in Vista – Windows Defender, Indexer, and Defragmenter – are already using this feature, and more ISVs who develop such applications are expected to start using I/O prioritization.

 

Better Together

It is important to note that Super Fetch, ReadyBoost and ReadyDrive compliment each other.  ReadyBoost and ReadyDrive provide with faster caches.  This provides SuperFetch with a bigger canvas to work with – now it can put some pages in RAM, some other in the ReadyBoost storage and some other in ReadyDrive. The fact that SuperFetch does a great job in pre-loading the main memory with code and data from disk gives an opportunity for the disk to stay spun down for a longer time which alongwith ReadyDrive improves battery life and disk ruggedness.

In terms of the entire I/O stack, SuperFetch is towards the top of the I/O stack – when your code tries to access a particular piece of data in the memory, the goal SuperFetch has is to anticipate that data and the page for that data ahead of time.  For this, it will try and pull that page out of the disk, and put it in ReadyDrive or ReadyBoost storage if they are available, or the RAM.  ReadyDrive and ReadyBoost on the other hand, are below the filesystem in the I/O stack – both of these mechanisms think in terms of disk sectors and do not think about data.

Finally, we need to think about the impact of these optimizations – SuperFetch itself cause extra I/O and extra CPU usage, so how does one do the trade-off? The solution lies with prioritization – SuperFetch uses low priority I/O and low priority threads, so the CPU and Disk I/O overhead is minimal and does not interfere with the actual jobs being performed. In fact, SuperFetch reads only a few pages per second at Very Low Priority.  There is certainly a memory overhead which is caused by having to maintain information about what all is happening in the system, but that overhead is minimal and more than makes up with all the advantages provided.