One of the most critical, and challenging, architecture decisions an IT professional has to make is choosing a virtual server storage setup. Even with such a wide variety of choices, there isn't a singular solution that is the absolute best for virtual servers. And because every work environment varies greatly, a data storage system that complements one solution may not work well with a different system.
Although Fibre Channel (FC) has been the primary choice for several years,network-attached storage (NAS) and iSCSI have become increasingly popular alternatives for virtualization because they provide solid performance for companies working with smaller budgets.
Fibre Channel storage
While FC storage has strong performance and reliability, those factors add up to both high cost and complications in the data center. FC is the most popular choice for larger virtual environments specifically because of its speeds -- 8 Gbps is common now, and 16 Gbps is finding more widespread availability. In addition, FC is a much more reliable technology, making it a more prominent choice for companies. Storage networks built on FC are more isolated, therefore more secure than storage devices that are Ethernet-based. In comparison, however, the requirement of host bus adapters (HBAs) and switches make FC more costly. In addition, FC environments are much more complex to manage and implement because their initial configurations differ greatly from a traditional network infrastructure. Not all companies have staff with FC storage-area network (SAN) administration experience, so building and overseeing a SAN often translates to hiring a specialized IT admin or sending a current IT admin into training. This additional training or hiring translates to an added implementation cost that companies might not initially anticipate. Other advantages of FC storage include the ability to boot from SAN so finding a local host storage isn't necessary. VMware vSphere VMFS volumes can be used with block-level storage, meaning integration is less of a hassle for VMware customers.
For customers with many high disk input/output (I/O) virtual machines (VMs) running on their hosts, FC storage can translate well as a maximum performance option. If you're interested in running FC storage over traditional Ethernet components, Fiber Channel over Ethernet (FCoE) can be another option, however because it requires 10 Gbps Ethernet networking, it can be a pricey option. FC storage also means IT administrators must acquire specialized (and often expensive) components, including switches, cables and HBAs. Another thing to keep in mind is that within FC storage, authentication and encryption are more difficult to implement, as the storage solution has fewer security controls.
Sidebar: I/O virtualization
I/O virtualization is increasingly being used with server virtualization. It enables a single physical I/O adapter to appear as multiple virtual network adapters (NICs) or host bus adapters (HBAs). One of the challenges with server virtualization is that hosts require a large number of I/O adapters to connect to both data and storage networks. A typical host may have six to eight NICs used for general network connectivity, and at least two NICs or HBAs to connect to storage networks. I/O virtualization lets you consolidate many I/O adapters on a host into a single adapter or two that can handle all I/O requirements. I/O virtualization is implemented in several different ways. Companies like Xsigo Systems Inc. emulate HBAs and NICs over either standard Ethernet or InfiniBand fabrics. Virtensys takes a different approach and uses a PCIe extension adapter to connect to an appliance that contains shared I/O adapters. Both approaches can greatly simplify host I/O connectivity, and reduce hardware costs and host power consumption.
Though it may seem obvious, if you already have an FC SAN implemented in your environment, using it with virtualization is the logical choice, as expanding an existing SAN is cheaper and simpler than a brand new implementation.
iSCSI storage
iSCSI storage is a popular and solid alternative to Fibre Channel. iSCSI is ablock-based storage like FC but it uses traditional Ethernet network components for connectivity between hosts and storage devices. Because you can use existing Ethernet components, iSCSI is often much cheaper to implement. iSCSI works by using a client called an initiator to send SCSI commands over a local-area network (LAN) to SCSI devices (called targets) located on a storage device.
iSCSI initiators can be software or hardware based. Software initiators use device drivers that are built into the hypervisor to use Ethernet network adapters and protocols to write to a remote iSCSI target. Hardware initiators use a dedicated iSCSI HBA that combines a network adapter, a TCP/IP offload engine (TOE) and a SCSI adapter into one device to help improve the performance of the host server. While software initiators work just fine in most cases, hardware initiators offer slightly better I/O performance and use less host resources. You can also boot from hardware initiators; in addition, a new technology called iSCSI Boot Firmware Table (iBFT) will let you boot using a software initiator if the installed network interface card (NIC) and hypervisor support it.
iSCSI performs very well on 1 Gbps Ethernet networks, but switching to 10 Gbps can give it a huge boost and put it on par with (or better than) FC. Most hypervisors support 10 Gbps iSCSI, but the cost may be so high that it will be just as expensive as FC to implement. The biggest risks to using iSCSI are the additional CPU overhead when using software initiators (which can be mitigated with hardware initiators), and the more fragile and volatile network infrastructure it relies on. The latter issue can be addressed by completely isolating iSCSI traffic from other network traffic.
Advantages of iSCSI storage
- Lower cost alternative to FC storage that uses standard Ethernet components; iSCSI storage arrays also tend to cost less than FC arrays
- Software initiators can be used for ease of use and lower cost; hardware initiators offer maximum performance
- Block-level storage (like FC) that can be used with vSphere VMFS volumes
- Speed and performance is greatly increased with 10 Gbps Ethernet
- No special training/skills needed to implement and manage the technology
- Supports authentication (CHAP) and encryption for security, as well as multipathing for increased throughput and reliability
- Can be deployed more quickly than FC
Disadvantages of iSCSI storage
- Because iSCSI is most commonly deployed as a software protocol, it adds to CPU overhead vs. using hardware-based initiators
- Performance is typically less than that of FC SANs
- Typically doesn’t scale as high as FC storage systems
- Network latency and non-iSCSI network traffic can diminish performance
iSCSI also offers more variety and greater flexibility when it comes to choosing data storage devices. You can purchase a range of iSCSI storage products, from small dedicated iSCSI storage devices for less than $2,000 to large enterprise-class devices. Keep in mind that when it comes to performance you generally get what you pay for. If you have a large number of VMs and heavy workloads, you need to spend more for a storage system. iSCSI is a great choice for many companies that want affordability and simplicity. While iSCSI is often criticized for its performance, a dedicated, properly configured iSCSI system can perform nearly as well as a Fibre Channel setup and will be adequate for many environments.
NAS
The big difference between iSCSI and NAS (specifically, NFS) is the type of protocol used to write data to the storage device. iSCSI uses a block protocol and data is written in blocks by the virtual host to the storage device. The host server is responsible for maintaining the disk file system on the storage device just as it would with local disk. NAS, on the other hand, uses a file-sharing protocol and the host server simply communicates with the storage device that maintains the disk file system.
With NAS, you’re essentially offloading the storage device functions responsible for writing data to the drives from the host server to the storage device. NAS uses a special software NFS client built into the hypervisor that uses a network adapter in the host to communicate with the NFS server.
All the major virtualization platforms support using NAS devices for theirvirtual machines. Because NFS is a widely supported protocol, there are many different options for using NAS storage with your virtual hosts. These can range from taking a standard physical server and converting it into an NAS server, using virtual NAS software or using a dedicated storage appliance. The costs and performance characteristics with each option can vary greatly; dedicated appliances offer the best performance at the greatest cost.
Almost every storage vendor offers a NAS storage device that supports NFS. With block storage devices, allocating storage will consume the full space right away, but with NAS, capacity grows as data is written to it. Regardless of your budget, you can easily find a good NAS device that will meet your requirements.
In most cases, NAS won’t equal the performance of a Fibre Channel SAN, but a properly architected NAS system can meet the performance needs of most workloads. Similar to iSCSI, NAS uses NICs to communicate with storage devices, which may mean a 1 Gbps speed limit, but newer 10 Gbps NICs offer a huge speed increase if you can bear the cost. The performance of NAS is nearly the same as iSCSI. As long as the CPU doesn’t become a bottleneck, the maximum throughput of both iSCSI and NAS is limited by the available network bandwidth.
Advantages of NAS
- Many NAS storage devices use thin provisioning by default, which can help conserve disk space
- File locking and queuing are handled by the NAS device, which can result in better performance vs. iSCSI/FC where locking and queuing are handled by the host server
- NAS doesn’t have a single disk I/O queue like block storage devices, which can result in greater performance; NAS performance is based on the bandwidth of the network connection and the capabilities of the disk array
- Can be less costly to implement than FC storage as it uses standard Ethernet components; NAS arrays tend to cost less than FC arrays
- No special training/skills are needed to implement and manage the technology
- Expanding virtual datastores is done easily by increasing the disk on the NFS server; datastores will automatically increase accordingly
- Snapshots, cloning and so on are done at the file system level instead of the LUN level, which can offer greater flexibility and more granular support
Disadvantages of NAS
- Booting directly from a shared storage device isn’t supported with NAS devices
- There is CPU overhead as the hypervisor must use a software client to communicate with the NAS server
- Some vendors don’t recommend NAS storage for certain sensitive transactional applications due to latency that can occur
- Support for new virtualization features sometimes lags vs. block storage devices
- NAS doesn’t support multipathing from a host to the NAS server; only a single TCP session will be opened to a NAS datastore, which may limit performance
You shouldn’t be discouraged by some of the disadvantages of using NAS, as they may only apply to specific circumstances or result from poorly architected NAS solutions. With a properly sized solution that can handle the VM workloads on your hosts, NAS is usually as good a choice as a block-storage device. In the past, NAS had limited support from virtualization, but it’s now fully supported.
And the winner is . . .
There are a lot of factors to consider when choosing a storage device for your virtual environment, but decisions ultimately come down to simple factors such as budget, performance and capacity. Many storage devices now come with direct integration support for virtualization so this can also be a big factor. VMware vStorage APIs offer many benefits that allow for tighter integration between the storage device and the hypervisor, as well as offload many storage-related tasks from the hypervisor to the storage array.
Another area of concern is support. While Microsoft Hyper-V has pretty broad support for just about any storage array supported by Windows, VMware has a strict hardware compatibility guide that lists all supported storage devices. One reason for this is that VMware has very deep API integration and the guide ensures that storage devices have been tested with vSphere. It also lists the various integration features supported for each array.
While Fibre Channel is a well-established storage platform, don’t be afraid to try iSCSI or NAS devices as more affordable alternatives. With a wide variety of iSCSI and NAS products to choose from, you’ll have to research their capabilities and scalability to ensure that they’ll meet your requirements. Storage is the most critical design decision you’ll make for your virtual environment, so spend the time researching the alternatives to understand the different technologies and features that are available.
Eric Siebert is an IT industry veteran with more than 25 years of experience who now focuses on server administration and virtualization. He’s the author of VMware VI3 Implementation and Administration (Prentice Hall, 2009) andMaximum vSphere (Prentice Hall, 2010).
This story was previously published in Storage magazine
No comments:
Post a Comment