Tag: Flash

EMC Fully Automated Storage Tiering

Storage Tiering is nothing new. We use fast 15K RPM disks for high performance applications, slower 10K RPM disks for less demanding applications, and 7.2K RPM SATA disks for archive storage. Recently, solid state disks (SSDs) have also become more common for really high performance needs. The trick is managing it all.

Two or three years ago, if you wanted to implement automatic storage tiering, I would have pointed you in the direction of Sun’s Storage and Archive Manager- SAM and QFS, Sun’s tightly integrated shared file system. SAM-QFS automatically moves files from one storage tier to another based on the SAM policy and transparently retrieves the files when requested. With tape still the least expensive storage available, this is still a great solution for archiving petabytes of documents/files.

Unfortunately, SAM works at the file level so it will not help our databases run faster. What will help us is ZFS. ZFS is still making some fairly big waves in the storage community with it’s Hybrid Storage Pool feature. In a standard configuration, ZFS uses RAM for a Layer 1 read cache (ARC).  In advanced configurations, the zpool can be configured to use a Layer 2 cache (L2ARC) on faster disks ie. SSDs compared to SAS compared to SATA , etc. The zpool can also be configured to use separate, possibly faster disks for the ZFS Intent Log (ZIL) which is basically a write cache (without getting into why it is more than a write cache). Even without faster disks, the ability to store the read/write cache on a separate device can increase performance just by dedicating more IOPS to the cause.

Oracle/Sun’s 7000 series storage builds on the success of the ZFS Hybrid Storage Pool, using Logzilla devices for the ZIL and Readzilla devices for the L2ARC. With the powerful flash acceleration in the storage pool, even 7.2K RPM disks can give performance equal to that of higher speed 15K RPM disks.

Although ZFS does great things for performance by utilizing multiple tiers of storage devices, all the data is still physically stored on the same tier of storage in addition to having the hot data stored again in the caches. This is arguably a waste of capacity but can also lead to performance issues in some cases. For example, a cold L2ARC cache after reboot could give slower performance until fully warmed up. Oracle will probably fix this at some point by allowing the L2ARC to persist if stored on a non-volatile device (bug_id=6662467).

In the meantime, EMC recently announced an interesting new feature called FAST, short for Fully Automated Storage Tiering. FAST is available from FLARE version 04.30.000.5.004. FAST allows you to define a pool in the array composed of multiple RAID Groups, and then define a LUN on the pool as opposed to defining a LUN on the RAID Groups themselves. Once the LUN begins filling with data, the EMC will transparently begin transparently migrating data between the tiers of the pool in 1GB chunks, storing hot data on the fastest tiers and coldest data on the slowest tier.

FAST sounds like a dream come true. No more complicated storage configurations for the database. No more packages and processes to move historical data to slower disk groups. On the other hand, I am skeptical as to whether or not this technology is really mature. Do all EMC products treat the FAST LUNS the same as traditional LUNS (SnapView, Replication Manager, etc.) Also, are the ramifications of disk failures for a FAST LUN the same or does failure of a Tier 1 disk in a FAST pool mean alot more high performance eggs in one basket? Time will tell.

The Systems Architect

What is a Systems Architect?

Systems Architects have years of experience in the various parts of the systems they work with. Most probably, they specialize in a specific area area of expertise where they began their careers, but have since expanded their knowledge by learning from their colleagues and from life’s lessons. To get to their position, they have proven their ability to analyze and understand the needs and constraints of the business they work in. They are responsible for deciding what technologies will provide the best solutions for a business, how to integrate them with existing systems, and how to retire them when they are obsolete.

There are several flavors of Systems Architects.

  1. The Bang-For-Your-Buck Architect– They can cut your OP-EX by 90% and consolidate all your servers on to two desktops running VMWare if you give them enough time and rewrite your application in assembler. They live and get fired by the Project Management Triangle, always aiming for High Quality, Cheap, but Slower than molasses.
  2. The Unbreakable Architect– Their systems do not know the meaning of the word Downtime but come at the price of top of the line co-location, ISP level network equipment, 2n+1 hardware, months of code freezes, release testing cycles, staging and performance test runs, security scans and rejects.
  3. The Performance Architect– Optimization is the name of the game. Your site will be in the customer’s browser before they can blink (remember no flash, no heavy images, no javascript and no calls to google analytics). Your database will not have time to read from the disk before it answers your queries (you didn’t really invest in multiple Oracle RAC nodes to access the same data at the same time? everyone knows that just slows the database down!)
  4. The All-Of-The-Above Architect– Everything is important in the right balance. Consolidation and cutting costs is a factor but so is ease of implementation. The system should be flexible enough to handle new requirements quickly but getting something done quickly is not an excuse for poor implementation. The system must be robust but not every risk is worth mitigating. Cost/Performance is quantitative. It should be measured and improved if necessary in the most cost effective manner. The work of finding balance in a system never ends but the most important thing is to always move towards the end goal.

If you know a Systems Architect like those described above, please have pity on him. He is really trying to do what he thinks is best and he really does (probably) have a lot of experience.

Next time- “The Software Architect”

Sparc Solaris 10 Jumpstart Flar DVD – Part 1

The Solaris Flash installation feature enables you to use a single reference installation of the Solaris OS on a system, which is called the master system. Then, you can replicate that installation on a number of systems, which are called clone systems. You can replicate clone systems with a Solaris Flash initial installation that overwrites all files on the system or with a Solaris Flash update that only includes the differences between two system images. A differential update changes only the files that are specified and is restricted to systems that contain software consistent with the old master image.

By combining Flash installation with Custom Jumpstart, and packaging all that on a re-mastered Solaris installation DVD, you can create very fast and efficient, standalone, and automated installation media.

I ran into several issues trying to create such a DVD when following the standard Google results so I thought I’d summarize my experiences. This is a work in progress- I might hit a brick wall at some point, but I hope not.

First, I built the prototype system. I’m running Solaris 10 11/06 with one non-global zone based entirely on a ZFS file system. This will make things challenging since Solaris Flash Archives are not completely compatible (or even supported) for these kinds of configurations and Jumpstart is not ZFS aware.

Creating the Flash Archive

  1. Make sure you have the right packages installed (SUNWinst, SUNWadmc, SUNWadmfw, SUNWbtool) Theoretically, you should install platform support for all possible hardware- I forget the name of the cluster- but if you will only be installing on the same hardware, this isn’t necessary. NOTE- If you try to install packages from inside single user mode with non-global zones it will give you issues.
  2. Put the prototype system into single user mode
  3. Create a text file, called for example ‘exclude’, with the directories not to include in the flash archive (man flarcreate)
  4. flarcreate -n system -X exclude -c system.flar
    Full Flash
    Checking integrity...
    Integrity OK.
    Running precreation scripts...
    Precreation scripts done.
    Determining the size of the archive...
    cpio: File size of "etc/mnttab" has decreased by 136
    2259925 blocks
    1 error(s)
    The archive will be approximately 764.41MB.
    Creating the archive...
    2259925 blocks
    Archive creation complete.
    Running postcreation scripts...
    Postcreation scripts done.

    Running pre-exit scripts...
    Pre-exit scripts done.
  5. Verify your archive: flar info -l system.flar

More to come…