What’s the (SnapMirror) syntax, Kenneth?

I had the opportunity to configure SnapMirror for a client today and it gave me a bit of a headache.  I did what I thought was my due diligence: reading the relevant vendor documentation for SnapMirorr for each version of Data ONTAP, 7.3.2 and 8.0.3P3.  What I failed to do was read a few lines further than I actually did – I missed a simple piece of syntax that turned a 30 minute WebEx into a 2 hour ordeal. I learned a good lesson about SnapMirror during this engagement, though, and I’d like to share it.

The SnapMirror of these three volumes were actually for a data migration because the source filer is being decommissioned.  The general steps required in the engagement today were as follows:

<> Run a cable between what will be the dedicated replication links on each filer

<> Configure each interface with IP settings

<> Ensure SnapMirror is licensed and enabled on each filer

<> Configure /etc/hosts and /etc/snapmirror.allow files on source filer

<> Configure /etc/hosts and /etc/snapmirror.conf files on destination filer

<> Initialize the baseline replicaction

The destination volumes had already been created, though I needed to restrict them as well as grow one by a couple hundred GB when the initialization failed.

I created a text file with these steps enumerated with an understanding I would fill in the missing specifics once I had access to the client systems.  Here’s what I had:

#==============================
#Check network connectivity first using the dedicated
#replication link
#==============================
netapp2-e4a - 192.168.10.100
netapp5-e0M - 192.168.10.102

After the interface configuration,

I was told there were, indeed, open interfaces on each filer for a dedicated replication link.  When I found out what had been cabled, I learned that the “open” interface on one filer was the management interface, e0M.  Well, I figured this would work (remember it’s temporary) but warned that e0M is limited to 100Mbps and essentially multiplies our calculated transfer times by 10. We have a baseline of 2TB with an unknown daily change rate, though it’s likely in the single digit GB.  So now we’re looking at over 40 hours to baseline the SnapMirror instead of 5 hours over a 1Gbps link.  This is not so bad because 1) it’s over a dedicated replication link, 2) both filers are in the same data center and 3) we have about a month before we’re scheduled to cutover the SnapMirrored volumes. This leaves time for the baseline and even replicating the daily change just before we cut over.

 #==============================
#To be configured on BOTH source and destination
#filers, netapp2 and netapp5
#==============================
#Add the names and IP addresses of each filer's replication link
#to their respective /etc/hosts files
rdfile /etc/hosts
...
...
#copy the current /etc/hosts and append the replication link info
wrfile /etc/hosts
...
...
<netapp2_replication_ip_address> <netapp2-replication-name>
<netapp5_replication_ip_address> <netapp5-replication-name>

#Is SnapMirror licensed?
license
#If not, then
#license add XXXXXXX

#Is SnapMirror enabled?
options snapmirror #looking for snapmirror.enable on
#If not, then
#options snapmirror.enable on

So far so good. No problems seen.

#==============================
#To be applied on source system, netapp2
#==============================
options snapmirror.access legacy
#this option allows literal IP addresses to be used in snapmirror.allow file
options snapmirror.checkip.enable on
rdfile /etc/snapmirror.allow

#Be sure to make a copy of the current snapmirror.allow file
wrfile /etc/snapmirror.allow
<netapp5_replication_ip_address>

Awesome. Still doing good.

 
#==============================
#To be applied on the destination system, netapp5
#==============================
#Ensure the same vol types are created on the destination
#system, i.e. FlexVols or TradVols
#The volumes must be as large or larger than the source vols
#Restrict the volumes
vol restrict volume2
vol restrict volume3
vol restrict volume4

No problems so far.

 
rdfile /etc/snapmirror.conf
#Be sure to make a copy of the current snapmirror.conf file
wrfile /etc/snapmirror.conf
netapp2-e4a:/vol/volume2 netapp5-e0M:/vol/volume2 - 0 23 * * 
netapp2-e4a:/vol/volume3 netapp5-e0M:/vol/volume3 - 0 23 * * 
netapp2-e4a:/vol/volume4 netapp5-e0M:/vol/volume4 - 0 23 * * 

Ah, yes. There’s a problem. I received this error a few seconds after writing to the snapmirror.conf file

netapp5*> /etc/snapmirror.conf: line 1 Invalid destination: /vol/volume2: The argument must be a volume name or a qtree path. Please specify either "volname" or "/vol/volname/qtreename"., ignoring line
/etc/snapmirror.conf: line 2 Invalid destination: /vol/volume3 : The argument must be a volume name or a qtree path. Please specify either "volname" or "/vol/volname/qtreename"., ignoring line
/etc/snapmirror.conf: line 3 Invalid destination: /vol/volume4 : The argument must be a volume name or a qtree path. Please specify either "volname" or "/vol/volname/qtreename"., ignoring line

So the short of it is that when using VSM, you simply use the volume names, not the path to the volumes.  Use pathnames when using QSM.  Innocent enough, right? So the following is correct.

wrfile /etc/snapmirror.conf
netapp2-e4a:volume2 netapp5-e0M:volume2 - 0 23 * * 
netapp2-e4a:volume3 netapp5-e0M:volume3 - 0 23 * * 
netapp2-e4a:volume4 netapp5-e0M:volume4 - 0 23 * *
</blockquote> 

<p>But then I get this error:</p> 


netapp5*> /etc/snapmirror.conf: line 1 destination filer "netapp5-e0M" does not match hostname "netapp5", ignoring line
/etc/snapmirror.conf: line 2 destination filer "netapp5-e0M" does not match hostname "netapp5", ignoring line
/etc/snapmirror.conf: line 3 destination filer "netapp5-e0M" does not match hostname "netapp5", ignoring line

Turns out this is the bit where I didn’t quite read far enough.  There’s a difference in the format of the source and destination systems in the snapmirror.conf file.  From the Data ONTAP 7.3 Data Protection Online Backup and Recovery Guide (also the same in newer versions of ONTAP!)

Did you catch that?  Yeah, if you’re using specific replication links, you only specify the source replication link name and the destination hostname. I had specified netapp2-e4a as well as netapp5-e0M.  This is not correct.  The correct format of the snapmirror.conf file is

wrfile /etc/snapmirror.conf
netapp2-e4a:volume2 netapp5:volume2 - 0 23 * * 
netapp2-e4a:volume3 netapp5:volume3 - 0 23 * * 
netapp2-e4a:volume4 netapp5:volume4 - 0 23 * * 

Finally, once these things were squared away, I received this gem when I tried to initialize one particularly large volume.

<p>netapp5*> snapmirror initialize -S netapp2-e4a:volume3 netapp5:volume3 Tue Feb 19 16:10:27 CST [netapp5: replication.dst.err:error]: SnapMirror: destination transfer from netapp2-e4a:volume3 to volume3: destination volume too small; it must be equal to or larger than the source volume.
Transfer aborted: destination volume too small; it must be equal to or larger than the source volume.

Too easy.  The fella who created the destination volumes mentioned he tried to size them based on how much data was actually used but he made them a bit too small.  I resized the volume a bit and I was golden.

So this wasn’t too bad, but I did have to take time to re-read the vendor docs during the setup  and while the client was waiting.  We were recording the WebEx session, too, so it ended up being close to two hours long.  I know I wouldn’t want to sit there and watch two hours worth of configs.  I guess I’ll know better for next time.

Cheers,

Mike

Google