NetApp Initiator Group Best Practices for VMFS LUNs

I’m often asked by my clients the best way to configure NetApp igroups when connecting to VMware VMFS LUNs, especially after I deploy a new system for them and I’m training them on their use.  I appreciate the question because it means someone’s actually thinking through why something is configured the way it is rather than just throwing something together.

The Problem

So this is what I see a lot of out in the field.  Single igroups are created with multiple initiators from multiple hosts.  This can be a problem, though, as I’ll show you.  Functionally, this configuration will work – each host will be able to see each LUN, all things being equal.  The problem arises when you want to either 1. remove a host from the igroup or 2. stop presenting a LUN to only a subset of hosts.

With this configuration, all hosts can access all LUNs, as shown below. Host 1 can talk to LUN 1 and LUN 2 and Host 2 can talk to LUN 1 and LUN 2.  This is represented by the green arrows.  Now, what if you wanted to stop Host 1 from accessing LUN 1?

You could try two things. First, you could try removing the masking of LUN 1 from the igroup.  That would leave you with something like the picture on the left.  Host 1 can no longer access LUN 1, but it can still access LUN 2.  Success!  But wait a minute…there’s more.  You get an angry call from Bobby Joe the SysAdmin saying that Host 2 can’t access LUN 1 any more.  It’s been ripped out from underneath Host 2.  Ouch.  That’s not good.  So while you succeeded in your goal, you also failed…miserably  So we see why this won’t work.

Option 2 to stop presenting LUN 1 to Host 1 includes removing Host 1’s initiator from the igroup.  That seems like a fine idea!  Aaand boom goes the dynamite.  Here’s what we’re left with when you do that.  Host 1 certainly can’t access LUN 1 any more, buuut it also can’t access LUN 2.  This is clearly not what we wanted. 

While no problems are seen initially when using a single initiator group – after all, all the hosts can see all the LUNs they’re supposed to see – the real problem arises down the road when you need to maintain the environment.  If you can afford downtime of your hosts, you can work around this problem.  But if you can’t, you’re in trouble.

So we see why grouping all hosts into a single igroup is a bad idea.  It provides little flexibility in removing LUN maskings from a subset of hosts.

The Fix

The proper way to create igroups in most environments is to create one for each host and place all of that host’s initiators in it.  I say, “most environments” because there are corner cases – for instance if you have dedicated HBAs for separate environments, Production and Test, for instance, you may want to create igroups for each of those environments per host.  You want something like this on the left.  Each host has a dedicated igroup and each LUN is mapped to each igroup.  This provides the most flexibility when removing LUNs from subsets of hosts.  There are more igroups to manage, sure, but the benefits of the added flexibility are much greater.

So now if you want to remove LUN 1 from Host 1, you simply unmask it from igroup 1 as shown here.  Host 1 can still access LUN 2 and no other hosts are affected.

An example with only two hosts and two LUNs may look trivial, but imagine vSphere clusters where you’re adding and removing hosts, maybe during a migration or hardware refresh and you’re trying to avoid downtime as much as possible.  Perhaps there’s new storage or things are just moving around, in general, this does, indeed, provide the flexibility you need.