Tag Archive for emc

Vendor Lock-In or One Stop Shop

I was recently discussing load balancers with someone. I said I was much happier with F5 than I was with Cisco and he countered that although he preferred F5 head to head, going with Cisco for all the network was better for them in the long run.

The situation with storage is similar. EMC makes a great SAN but a pretty bad NAS. Is it worth getting EMC”s NAS for the One Stop Shop factor?

Since Oracle’s acquisition of Sun, I’ve been looking forward to the success of their “One Stop Shop” philosophy. Successfully bringing all their offerings under one roof promises better and faster support all around.

Unfortunately, it has been almost a year and Oracle is still not sure how they are to unify the customer support systems. New support contracts don’t work in either system.  To make things a little less clear, Oracle recently announced that everything will be migrated to “My Oracle Support” but they don’t know when- very reassuring.

A simple pattern emerges. One Stop Shop is a dream for IT people. Support is hard enough to get when you’ve isolated a problem to a specific vendor. It is even harder when your problems are between two vendors and each points the finger at the other.

When does the One Stop Shop strategy become a rationalization for Vendor Lock-In? It is a delicate balance around how much better your IT could be with Best of Breed vs. how much worse they will be integrating all the different pieces of the puzzle.

Regarding Cisco vs. F5, I’m also pretty happy letting Cisco handle everything Layer 3 and under and I don’t worry too much about the integration. I’m also optimistic regarding Sun and Oracle. I think they’ll have the wrinkles ironed out by the second half of 2011. If they don’t, it will be a serious let down.

EMC Fully Automated Storage Tiering

Storage Tiering is nothing new. We use fast 15K RPM disks for high performance applications, slower 10K RPM disks for less demanding applications, and 7.2K RPM SATA disks for archive storage. Recently, solid state disks (SSDs) have also become more common for really high performance needs. The trick is managing it all.

Two or three years ago, if you wanted to implement automatic storage tiering, I would have pointed you in the direction of Sun’s Storage and Archive Manager- SAM and QFS, Sun’s tightly integrated shared file system. SAM-QFS automatically moves files from one storage tier to another based on the SAM policy and transparently retrieves the files when requested. With tape still the least expensive storage available, this is still a great solution for archiving petabytes of documents/files.

Unfortunately, SAM works at the file level so it will not help our databases run faster. What will help us is ZFS. ZFS is still making some fairly big waves in the storage community with it’s Hybrid Storage Pool feature. In a standard configuration, ZFS uses RAM for a Layer 1 read cache (ARC).  In advanced configurations, the zpool can be configured to use a Layer 2 cache (L2ARC) on faster disks ie. SSDs compared to SAS compared to SATA , etc. The zpool can also be configured to use separate, possibly faster disks for the ZFS Intent Log (ZIL) which is basically a write cache (without getting into why it is more than a write cache). Even without faster disks, the ability to store the read/write cache on a separate device can increase performance just by dedicating more IOPS to the cause.

Oracle/Sun’s 7000 series storage builds on the success of the ZFS Hybrid Storage Pool, using Logzilla devices for the ZIL and Readzilla devices for the L2ARC. With the powerful flash acceleration in the storage pool, even 7.2K RPM disks can give performance equal to that of higher speed 15K RPM disks.

Although ZFS does great things for performance by utilizing multiple tiers of storage devices, all the data is still physically stored on the same tier of storage in addition to having the hot data stored again in the caches. This is arguably a waste of capacity but can also lead to performance issues in some cases. For example, a cold L2ARC cache after reboot could give slower performance until fully warmed up. Oracle will probably fix this at some point by allowing the L2ARC to persist if stored on a non-volatile device (bug_id=6662467).

In the meantime, EMC recently announced an interesting new feature called FAST, short for Fully Automated Storage Tiering. FAST is available from FLARE version 04.30.000.5.004. FAST allows you to define a pool in the array composed of multiple RAID Groups, and then define a LUN on the pool as opposed to defining a LUN on the RAID Groups themselves. Once the LUN begins filling with data, the EMC will transparently begin transparently migrating data between the tiers of the pool in 1GB chunks, storing hot data on the fastest tiers and coldest data on the slowest tier.

FAST sounds like a dream come true. No more complicated storage configurations for the database. No more packages and processes to move historical data to slower disk groups. On the other hand, I am skeptical as to whether or not this technology is really mature. Do all EMC products treat the FAST LUNS the same as traditional LUNS (SnapView, Replication Manager, etc.) Also, are the ramifications of disk failures for a FAST LUN the same or does failure of a Tier 1 disk in a FAST pool mean alot more high performance eggs in one basket? Time will tell.

Sun Oracle Webcast Wrap Up

Last night I watched almost the entire 5 hour live webcast announcing Oracle’s strategies regarding the Sun Microsystems acquisition. As a near-evangelist for Sun and Solaris, I’m very happy with the deal finally going through and even happier that most of what Oracle said makes sense to me as a customer.

What I liked:

  • The clear commitment to the SPARC roadmap especially the T series. I honestly don’t know what I would have done if the T series servers disappeared. I’m very happy that they put raising the clock speed into the roadmap because some applications just can’t be deployed on these servers.
  • The clear commitment to making waves in Enterprise Storage. NetApp was specifically mentioned and obviously the 7000 series arrays are best suited to compete with the NetApp arrays but I hope they will draw some EMC blood as well. I like the plans for integrating backup capabilities.
  • The plans to integrate really great Solaris tech into Oracle applications like DTrace, and RBAC
  • The plans to offer direct support. Honestly this was one of the most annoying parts of working with Sun was having to work with different support providers in every location.
  • The plans to change the supply chain and ship direct- no more out of stock excuses.
  • The plans to integrate Ops Center with Oracle Enterprise Manager.
  • Larry Ellison’s stand up comedy
  • And completely unrelated- the flashing disk lights on the Exadata V2 🙂

I didn’t like:

  • The obvious cut planned for the x64 line of hardware. While they are keeping x64 where convenient (storage appliances, database machines, various other “clusters”) it looks like Oracle has no plans for dealing in x64 server business as a server business. I’m not a big user of the x64 stuff for servers but Sun doesn’t really offer anything reasonable for entry level anymore except the x64 line. This brings me to my next point-
  • The SPARC roadmap is slightly sucky as in how much processing power do you really want inside a single box.  According to the roadmap, their next plan is to double the amount of cores in a T3 processor so you’ll have one cpu with 16 cores and 128 threads. Their going to put two in a machine? four?  Here is how I see the servers they have today:
    • T1000- useless poorly designed server
    • T2000- ok server but a waste of rack space at 2RU
    • T5120- ok server but a waste of rack space considering I could put a T5140 in the same space
    • T5220- worse than the T5120 at 2RU
    • T5140- The best server ever built with exactly the right amount of everything
    • T5240- 2RU again???
    • T5440- I could serve ~8.64 billion web requests per day from one of these but I’d need a 1.6Gbit uplink and two servers for redundancy = 8RU, or else use 4 T5140 machines, deliver the same performance, and use 4RU?- maybe 5RU including n+1 redundancy.
    • NONEXISTANT – little SPARC machine for backup/monitoring/insert your SPARC only app that doesn’t deserve a minimum of 32 threads and 2RU  here.

    At some point, you just want more smaller machines for less points of failure. I really have uses for low end SPARC machines and they don’t make them any more.

  • I don’t really like the “server phone home” idea.
  • No mention of OpenSolaris- I’m not really a user but I didn’t like that it wasn’t mentioned- What does that mean??
  • No mention of Webstack. I really like Sun Webstack as an idea. I’m not sure what is happening to it now?
  • No mention of how Oracle will be combining the knowledge bases? Sunsolve? Bigadmin? docs.sun.com? forums.sun.com (looks like this already had an Oracle makeover :?)

One thing I’m not sure about is the integration of Sun virtualization technologies into Oracle VM. On one hand it sounds good, on the other hand, I think this was the only part of the presentation where I noticed there were no due dates. Virtualization is super important to me so I really want to know where things stand.

Obviously, it is easy to  get up and say everything will integrate but doing it is much harder. Just getting past the internal politics of this will be a major issue. Now we can only wait and see if Oracle can pull it off.

I used to get upset with “Oracle people” for always thinking that Oracle was the solution to every problem. If they pull off this acquisition, I much just become an “Oracle person” myself.

No ZFS Support for EMC Replication Manager

As I originally blogged, I was hoping to use EMC snapshots to perform server-less/network-less backups. EMC provides two main tools for managing snapshots in this type of situation:

  • EMC Replication Manager
  • EMC PowerSnap Networker Module

The PowerSnap Module supposedly automates taking snapshots for the purpose of backups, while Replication Manager supposedly provides a much more robust package.

With Replication Manager you might create a policy to take a snapshot every five minutes, keep the last 10, and use those for backups whenever necessary.

To make a long story short, Replication Manager is useless for LUNs with ZFS. According to EMC, this won’t change in the near future. PowerSnap also has no support for taking snapshots of LUNs with ZFS on them so basically EMC has no server-less backup offerings for Solaris with ZFS.

As an IT guy in general, ZFS is the best thing that has happened to file systems in the last 10 years and it is only getting better. ZFS is already standard in FreeBSD and NetBSD. Linux supports ZFS over FUSE due to license issues but I’m confident those will be solved. The file system is platform independent, meaning you can move the data transparently between Intel and Sparc architectures. Deduplication has just been added to the feature set and disk encryption is on it’s way.

As a Solaris admin, I really can’t figure out why EMC would decide to cut off their own foot like this. It is clear that UFS will remain for legacy and backwards compatibility but ZFS is the future. Not planning to support ZFS is like not planning to support Solaris.

The only possibility that I can see is that EMC sees Sun, Solaris, and ZFS as enough of a threat, that they are strategically trying to limit options? For operations local to a server, ZFS has largely replaced the need for heavy hardware like EMC on the SAN. Some would argue that ZFS RAID + JBOD is better than ZFS + RAID on EMC. You can do the snapshots without the EMC. On a simple level, you can send snapshots asynchronously to another system, similar to MirrorView, without the EMC. You can do deduplication without the EMC. Now with Sun’s Flash Cache technology which integrates with ZFS, you can get the performance without the EMC. Along the same lines, you see Sun changing the rules of the storage/database game with solutions like Exadata V2. The integration of Zones with ZFS may be challenging Vmware on the virtualization front, especially with the serious advantage Sun’s Coolthreads servers have in terms of consolidation.

That said, I still prefer to offload this work to dedicated storage hardware for the time being and probably in the future. If EMC chooses not to support ZFS, they will only force us not to buy EMC arrays. We will stop buying disks, stop buying tools, etc.

Instead, they should be providing better support for ZFS, integrating with ZFS to get better performance, providing tools which make EMC the preferred disk array behind a ZFS filesystem.

EMC Replication Manager in Solaris

UPDATE: No ZFS Support for Replication Manager in the near future

Using storage level snapshots can be used to run backups without directly requiring resources from the original host.

EMC Replication Manager coordinates the creation of application consistent snapshots across all the hosts in your network. It handles scheduling creation/expiration of snapshots,  mounting and unmounting from backup servers, etc. from a single console.

Although it is not tightly integrated into EMC Networker like the similar Networker PowerSnap module, it can be used to start a backup process after taking a new snapshot and it has the capability to manage snapshots unrelated to backups from a GUI.

While the data sheet claims support for Solaris, there are several caveats which I have run into.

  1. There is no mention of ZFS support in the data sheet and apparently, there is no support in the software either. One would expect this to be a non-question since ZFS has been part of Solaris since 2006.
  2. The data sheet is missing the word “SPARC” next to the word Solaris. There is no support for x86.

Honestly, this has put a dent in my plans since my backup server is an x86 box. I’m hoping the lack of ZFS support will work out as long as we can script any FS specific magic we need. I don’t have an option of running something like Linux on it (just to get the software working) because I won’t be able to even mount the ZFS filesystems- let alone back them up.

In the meantime, I’ll have to move my backups to a SPARC server and considering the lack of low end SPARC machines, I’ll have to allocate something way too expensive to be a backup server.