Originally Posted by gregzoll
What they are doing with the hard drive space is really nothing more than what has been used for Linux, just that MS is trying to make people think that they have reinvented the wheel, along with the fact, that they are dumbing down the interface to partition the hard drive, but in a virtual way, which is already done with Virtual machines. What I am talking about is the offline storage, which is and will always be Skydrive. 8 Takes advantage of Skydrive, by loading files up in the cloud, so that if you switch devices, or computers, the files will always be available.
Here, to make things simple for you, Microsoft so called "Storage Pooling" is the Samba SMB 2.2 protocol. So again, Microsoft did not invent anything new, they just decided to start implementing Open Source into Windows 8, which has been done since Windows Vista & 7.
You need to read up on storage space before you start discussing it. Yes-it takes advantage of the SMB 2.2 protocol but that's not what it is either. It will allow you to pool any type of drive attached to the machine. SATA, USB, Firwire, SAS. etc. Use of virtual disks which behave like physical disks for all purposes with powerful, new capabilities associated with them such as thin provisioning as well as resiliency to failures of the underlying physical media. You also have the capability of striping, parity, 2 way and 3 way mirrors.
The magic that allows us to create a 10TB mirrored space on 4TB of total raw capacity is called thin provisioning. Thin provisioning ensures that actual capacity is reserved for the space only when you decide to use it, for example, when you copy some files to the volume on the space. Previously allocated physical capacity can be reclaimed safely whenever files are deleted, or whenever an application decides that such capacity is no longer needed. This reclaimed capacity is subsequently available for usage by either the same space, or by some other space that is carved out from the same pool. We achieve all of this through architected cooperation between the underlying file-system (NTFS) and Storage Spaces.
Storage Spaces Features
With thin provisioning, you can augment physical capacity within the pool on an as-needed basis. As you copy more files and approach the limit of available physical capacity within the pool, Storage Spaces will pop up a notification telling you that you need to add more capacity. You can do so very simply by purchasing additional disks and adding them to your existing pool.
2 way & 3 way Mirroring
Another core (also optional) capability associated with a space is resiliency to failure of the physical disks comprising the storage pool. For example, the space we’ve illustrated above is a mirrored space (in other words, it has the mirrored resiliency attribute associated with it). This mirrored setting ensures that we always store at least two (and optionally three) complete copies of data on different physical disks within the pool. This way, despite partial or complete disk failure, you’ll never need to worry about loss of data. As a matter of fact, the physical disks comprising the pool are typically not even visible to other components within Windows or to applications running on your PC. By extension, the fact that some physical disks within the pool have failed, is completely shielded from other Windows components or applications. They continue to operate on the space, completely oblivious to the fact that Storage Spaces is working quietly in the background to maintain data availability. Additionally, upon disk failure, Storage Spaces automatically regenerates data copies for all affected spaces as long as sufficient alternate physical disks are available within the pool.
There’s another resiliency attribute, called parity, which directs Storage Spaces to store some redundancy information alongside user data contained within the space, thereby enabling automatic data reconstruction in the event of physical disk failure. While conceptually similar to mirroring, parity-based resiliency utilizes capacity more efficiently than mirrored spaces do, but with higher random I/O overhead. Parity spaces are well suited for storing data such as large home videos, which have large capacity requirements, large sequential (predominantly append) write requests, and an infrequent-to-minimal need to update existing content.
SMB 2.2 in Windows 8 server-Continous Availability
This brings even more than Linux or any other OS.
In W2008 R2 a failover is not transparent. There is brief downtime to take down, move over, bring up the clustered service or role. 99% uptime at best
Failover in W8 is transparent to the server application. Supported planned and unplanned failovers, e.g. maintenance, failures, and load balancing. Requires Windows Failover Cluste, and both server and client must be running Windows Server 8. All operations, not just IO, must be continuous and transparent – transparent for file and directory operations.
This means we can have an application cluster that places data on a back end file server cluster. Both can scale independently.
Changes to Windows Server 8 to make transparent failover possible:
- New protocol: SMB 2.2
- SMB 2.2 Client (redirector): client operation replay, end-to-end for replay of idempotent and non-idempotent operations
- SMB 2.2 Server: support for network stte persistence, singles share spans multiple nodes (active/active shares – wonder if this is made possible by CSV?), files are always opened write-through.
- Resume Key – used to failover to: resume handle state after planned or unplanned failover, fence handle state information, mask some NTFS issues. This fences file locks.
- Witness protocol: enables faster unplanned failover because clients do not wait for timeouts, enables dynamic reallocation of load (nice!). Witness tells the client that a node is offline and tells it to redirect.
SMB2 Transparent Failover Semantics:
Server side: state persistence until the client reconnects. Example: delete a file. The file is opened, a flag is set to delete on close, and you close the file -> it’s deleted. Now you try to delete the file on a clustered file share. A planned failover happens. The node closes the file and it deletes. But after reconnect the client tries to close the file to delete it but its gone. This sort of circumstance is handled.
In Hyper-V world, we have “surprise failover” where a faulty VM can be failed over. The files are locked on file share by original node with the fence. A new API takes care of this.
SMB2 Scale Out
In W2008 R2 we have active-pasive clustered file shares. That means a share is only ever active on 1 node, so its not scalable. Windows Server 8 has scale out via active-active shares. The share can be active on all nodes. Targeted for server/server applications like SQL Server and Hyper-V. Not aimed at client/server applications like Office. We also get fewer IP addresses and DNS names. We only need one logical file server with a single file system namespace (no drive letter limitations), and no cluster disk resources to manage.
We now have a new file server type called File Server For Scale-Out Application Data. That’s the active/active type. Does not support NFS and certain role sevices such as FSRM or DFS Replication. The File Server for General Use is the active/passive one for client/server, but it also supports transparent failover.
VSS for WIndows Server 8 File Shares
Application consistent shadow copyof server application data that is stored on Windows Server 8 file shares. Bckup agent on the application server triggers backup. VSS on app server acts with File Share Shaow Copy Provider. It hits the File Share Shadow Copy Agent on the file server via RPC, and that then triggers the VSS on the file server to create the shadow copy. The backup server can read the snapshot directly from the file server, saving on needless data transfer.
Performance for Server Applications
SMB2.2 makes big changes. Gone from 25% to 97% of DAS performance. MSFT used same DAS storage in local and file share storage with SQL Server to get these numbers. NIC teaming, TCP offloads and RDMA improved performance.
Perfmon counters are added to help admins troubleshoot and tune. IO size, IO latency, IO queue length, etc. Can seperately tune SQL data file or log file.
Some of these may be Windows 8 server only features but most are available in Windows 8
Skydrive is the least of Windows 8 new features.