Tag: SAN

Vendor Lock-In or One Stop Shop

I was recently discussing load balancers with someone. I said I was much happier with F5 than I was with Cisco and he countered that although he preferred F5 head to head, going with Cisco for all the network was better for them in the long run.

The situation with storage is similar. EMC makes a great SAN but a pretty bad NAS. Is it worth getting EMC”s NAS for the One Stop Shop factor?

Since Oracle’s acquisition of Sun, I’ve been looking forward to the success of their “One Stop Shop” philosophy. Successfully bringing all their offerings under one roof promises better and faster support all around.

Unfortunately, it has been almost a year and Oracle is still not sure how they are to unify the customer support systems. New support contracts don’t work in either system.  To make things a little less clear, Oracle recently announced that everything will be migrated to “My Oracle Support” but they don’t know when- very reassuring.

A simple pattern emerges. One Stop Shop is a dream for IT people. Support is hard enough to get when you’ve isolated a problem to a specific vendor. It is even harder when your problems are between two vendors and each points the finger at the other.

When does the One Stop Shop strategy become a rationalization for Vendor Lock-In? It is a delicate balance around how much better your IT could be with Best of Breed vs. how much worse they will be integrating all the different pieces of the puzzle.

Regarding Cisco vs. F5, I’m also pretty happy letting Cisco handle everything Layer 3 and under and I don’t worry too much about the integration. I’m also optimistic regarding Sun and Oracle. I think they’ll have the wrinkles ironed out by the second half of 2011. If they don’t, it will be a serious let down.

No ZFS Support for EMC Replication Manager

As I originally blogged, I was hoping to use EMC snapshots to perform server-less/network-less backups. EMC provides two main tools for managing snapshots in this type of situation:

  • EMC Replication Manager
  • EMC PowerSnap Networker Module

The PowerSnap Module supposedly automates taking snapshots for the purpose of backups, while Replication Manager supposedly provides a much more robust package.

With Replication Manager you might create a policy to take a snapshot every five minutes, keep the last 10, and use those for backups whenever necessary.

To make a long story short, Replication Manager is useless for LUNs with ZFS. According to EMC, this won’t change in the near future. PowerSnap also has no support for taking snapshots of LUNs with ZFS on them so basically EMC has no server-less backup offerings for Solaris with ZFS.

As an IT guy in general, ZFS is the best thing that has happened to file systems in the last 10 years and it is only getting better. ZFS is already standard in FreeBSD and NetBSD. Linux supports ZFS over FUSE due to license issues but I’m confident those will be solved. The file system is platform independent, meaning you can move the data transparently between Intel and Sparc architectures. Deduplication has just been added to the feature set and disk encryption is on it’s way.

As a Solaris admin, I really can’t figure out why EMC would decide to cut off their own foot like this. It is clear that UFS will remain for legacy and backwards compatibility but ZFS is the future. Not planning to support ZFS is like not planning to support Solaris.

The only possibility that I can see is that EMC sees Sun, Solaris, and ZFS as enough of a threat, that they are strategically trying to limit options? For operations local to a server, ZFS has largely replaced the need for heavy hardware like EMC on the SAN. Some would argue that ZFS RAID + JBOD is better than ZFS + RAID on EMC. You can do the snapshots without the EMC. On a simple level, you can send snapshots asynchronously to another system, similar to MirrorView, without the EMC. You can do deduplication without the EMC. Now with Sun’s Flash Cache technology which integrates with ZFS, you can get the performance without the EMC. Along the same lines, you see Sun changing the rules of the storage/database game with solutions like Exadata V2. The integration of Zones with ZFS may be challenging Vmware on the virtualization front, especially with the serious advantage Sun’s Coolthreads servers have in terms of consolidation.

That said, I still prefer to offload this work to dedicated storage hardware for the time being and probably in the future. If EMC chooses not to support ZFS, they will only force us not to buy EMC arrays. We will stop buying disks, stop buying tools, etc.

Instead, they should be providing better support for ZFS, integrating with ZFS to get better performance, providing tools which make EMC the preferred disk array behind a ZFS filesystem.

RAID 10 vs RAID 5: Performance, Cost, Space, and HA

DISCLAIMER: I am not a SAN storage expert but I have spent a lot of time looking into SAN storage systems from the business side and I thought I’d share some of my conclusions.

It seems that the proverbial question is how to balance the performance, cost, usable space, and availability of a storage solution. Any DBA will ask you to give him RAID 10 on small fast disks. Anyone paying the bills will ask “Why can’t I use half the disks I bought?”

I took a couple hours with your friendly neighborhood spreadsheet and did the math. I base my calculations on EMC Clariion storage and tried to follow the EMC best practices guide as much as possible.

According to the best practices, I started my calculations based on a necessary performance level consisting of total IOPS, read percentage, and write percentage.
Then, using the following formulas, I calculate the actual disk IOPS required to provide the requested performance:

  • RAID 5 (4+1 Groups)
    Disk IOPS = (Read % * Required IOPS) +
                    (Write % * RAID5 write penalty * Required IOPS)
  • RAID 10
    Disk IOPS = (Read % * Required IOPS) +
                    (Write % * RAID10 write penalty * Required IOPS)

The RAID 5 write penalty in a 4+1 RAID group is 4 while the RAID 10 write penalty is 2.
Before you even put this in a spreadsheet you know what it will tell you-

  • In a 100% Read Only environment RAID 5 and RAID 10 will give the same performance. RAID 5 may use less disks to do it but not necessarily.
  • In a 100% Write Only environment, RAID 5 will require twice as many disk IOPS and almost twice the number of disks.
  • Anywhere in between those two extremes, the more writes required, the less number of RAID 10 disks you will need to achieve the performance.

If we stop there, it doesn’t seem like there is any point in using RAID 5 since even in the best case scenario, there is only a partial chance that we will use less disks. That is where the cost and space effectiveness issues come in.

  • Space Effective Storage Allocation

If I want 2000 IOPS, 100% Read Only, I can do that using 15 x 146GB 15k RPM disks in RAID 5 or in RAID 10. In RAID 5 I will get ~1.5TB net space while in RAID 10 I will get ~1TB.

  • Cost Effective Storage Allocation

So far, we have compared different RAID types using the same size and speed disks and we saw that theoretically we can use less disks to reach the same performance but at the expense of usable disk space.

If we use bigger disks for the RAID 10, does it make up for the lost space? What effect does using RAID 10 with fewer large disks as opposed to RAID 5 with lots of smaller disks have on the cost of my solution?

That brings us back to the spreadsheet. Using the required disk IOPS we can figure out the required number of physical disks of each type. For the sake of comparison I use the following information which I found on the Internet (your mileage may vary):

  • 146GB 4GbFC 15k RPM, 140 IOPS, $1256
  • 300GB 4GbFC 10k RPM, 120 IOPS, $1348
  • 1TB 4Gb SATA II 7.2k RPM, 80 IOPS, $2088

For each of these I calculate the minimum number of physical disks required for to reach the required IOPS with the required read/write profile for both RAID 10 and RAID 5. Then I figure in the RAID group sizes and calculated the usable disk space.

Using the prices above, I calculate the price per TB of disk space in each RAID configuration and find:

  • 146GB, RAID 5 (4+1): $11.91K/TB
  • 300GB, RAID 5 (4+1): $6.35K/TB
  • 1TB, RAID 5 (4+1): $2.87K/TB
  • 146GB, RAID 10 (4+1): $19.01K/TB
  • 300GB, RAID 10: $10.15K/TB
  • 1TB, RAID 10: $4.59K/TB

What is really interesting here is how close the 300GB RAID 10 is to the 146GB RAID 5! Is this a coincidence?

Looking at the IOPS/TB relationship and $K/IOPS, we find that the ratios are dependant on the read/write profile of the required IOPS. Given the similar Price/TB of 300GB RAID 10 and 146GB RAID 5, I look there for a price/performance/disk space sweet spot.

The following table shows the difference between 146GB RAID 5 IOPS/TB and 300GB RAID 10 IOPS/TB.
Each column represents a different Read percentage (the Write percentage is the inverse).
Negative numbers mean that for this Read percentage and IOPS requirement, RAID 10 gives more IOPS/TB of disk. Positive numbers mean that RAID 5 gives better IOPS/TB.

What you see from this is that for any read workload under 70%, you will get more IOPS/TB from 300GB 10k RPM disks using RAID 10 than you will with RAID 5 on 146GB 15k RPM disks.
Even if you hit 80%, RAID 5 will gain less than 100 IOPS over the RAID 10 configuration and you are still better off paying less for your disks- let the cache do it’s job. Combine all this with our previous conclusion – that the 300GB RAID 10 configuration is ~$1.75K less expensive per TB and I say you have a winner.