NFS performance issue

NFS ops increases and high performance in Filer.

Step 1 :

See below output. NFS OPS ~90000 (column 2 )

Begin: Tue Sep 09 17:52:30 GMT 2008 

 CPU   NFS  CIFS  HTTP   Total    Net kB/s   Disk kB/s     Tape kB/s Cache Cache  CP   CP Disk    FCP iSCSI   FCP  kB/s

                                  in   out   read  write  read write   age   hit time  ty util                 in   out

 61% 93233     0     0   93233 16978 17120      0      0     0     0   >60  100%   0%  -    0%      0     0     0     0

 61% 94057     0     0   94057 17171 17170     24     24     0     0   >60  100%   0%  -    3%      0     0     0     0

 61% 93384     0     0   93384 16998 16997      0      0     0     0   >60  100%   0%  -    0%      0     0     0     0

 60% 93020     0     0   93020 16932 16930      0      0     0     0   >60  100%   0%  -    0%      0     0     0     0

 59% 87212     0     0   87212 24549 16171     32     32     0     0   >60  100%   0%  -    3%      0     0     0     0

 62% 89947     0     0   89947 23317 16618      0      0     0     0   >60  100%   0%  -    0%      0     0     0     0

 61% 90205     1     0   90206 19157 16517      4      0     0     0   >60   99%   0%  -    1%      0     0     0     0

 59% 87641     0     0   87641 18877 16021   3728  22480     0     0   >60  100%  93%  Hf  33%      0     0     0     0

 61% 91401     0     0   91401 19355 16697    428   1012     0     0   >60  100%  13%  :    8%      0     0     0     0

 62% 92058     3     0   92061 19027 16943    392      8     0     0   >60  100%   0%  -

Step : 2

TIME: Tue Sep 09 18:00:02 GMT 2008TIME_DELTA: 07:33 (453s)nfsv3:nfs:nfsv3_read_latency:1426.57usnfsv3:nfs:nfsv3_read_ops:16/snfsv3:nfs:nfsv3_write_latency:267.78usnfsv3:nfs:nfsv3_write_ops:74/s

Step : 3

We can see 100 % of GETATTR request from the clients. SO most of the 90000 NFS OPS are for GETATTR . This can be due to your application behaviour orsome of your client is mis behaving in this environment.   
Server nfs V3: (43308280 calls)
null       getattr    setattr    lookup     access     readlink   read    
1 0%       43181988 100% 5302 0%    49543 0%   10529 0%   6 0%       3673 0% 
write      create     mkdir      symlink    mknod      remove     rmdir   
47953 0%   11 0%      1 0%       6 0%       0 0%       15 0%      1 0%    
rename     link       readdir    readdir+   fsstat     fsinfo     pathconf
1 0%       0 0%       5068 0%    3945 0%    129 0%     0 0%       108 0%  
commit     0 0%  

Step :4

192.168.4.101                 NFSOPS =  413629159 ( 1%)
192.168.4.95                  NFSOPS =  402901550 ( 1%)
192.168.4.78                  NFSOPS =  402401562 ( 1%)
192.168.4.79                  NFSOPS =  271381208 ( 1%)
192.168.4.83                  NFSOPS =  267008241 ( 1%)
192.168.4.102                 NFSOPS =  256459663 ( 1%)
192.168.4.66                  NFSOPS =  247692988 ( 1%)
192.168.4.84                  NFSOPS =  207956370 ( 0%)
192.168.4.100                 NFSOPS =  205229829 ( 0%)
192.168.4.80                  NFSOPS =  200581076 ( 0%)
192.168.4.97                  NFSOPS =  196879823 ( 0%)

Summary : Based on the perfstat we do not see any NFS latency on the filer end. CPU/DISK is normal on the filer but we do see high NFS OPS which looks

abnormal. Most of the NFS OPS are GETATTR calls and this can be either your application is malfunctioning or some of your client is malfunctioning. So checking 

your application or your clients is recomended. Also please check the client mount options and make sure you are not using "noac". Option "noac" will not 

cache the attribute information on the client end

Please also refer:

https://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb36582

SnapManager for VI 1.0 (Virtual Infrastructure)

In Vmware environments :

1. snapshots may not be granular enough.
2. vmware is best in NFS than ISCSI.
3. Traditional backups disadvantages.. (i) dont match your VI (ii) No cpu power left on the server.
4. Recommended.. only 4 snapshot copies a day.
   
5. how SMVI works.. (i) Backup initiated. (ii) Vm snapshot copy created. (iii) Snapmanager for VI triggers instant NetApp snapshot. (iv) vm snapshot removed.
6. only VMFS/NFS support. No RDM support.
   
7. Protocols - ISCSI, FCP, NFS.
8. snapmirror is integrated.
9. Backup scheduling and retention policies
10. SMVI only support netapp storage.
11. Efficiently meet the backup and DR needs of your ESX environment. (i) Leverages netapp snapshot tech (ii) provides a way to restore VM and data stores

Snap Manager

SnapManager provides rapid online backup and near instantaneous restoration of
Exchange databases by using online Snapshot™ technology that is part of the
Data ONTAP® software and integrating the Microsoft Exchange backup and
restore APIs and VSS. SnapManager can also leverage the SnapMirror
capabilities of storage systems to provide on-site or off-site SnapManager backup
set mirroring for disaster recovery.

Data management:

SnapManager supports the following data management
capabilities:
◆ Migrating Exchange databases and transaction logs to LUNs on storage
systems
◆ Backing up Exchange databases and transaction logs from LUNs on storage
systems
◆ Verifying the backed-up Exchange databases and transaction logs
◆ Managing the SnapManager backup sets
◆ Restoring Exchange databases and transaction logs from previously created
SnapManager backup sets

Data archival for long-term or remote storage of backups: You can
use SnapManager to create offline archives of Snapshot copies containing
Exchange backup sets for long term or remote backup storage. Three different
archive methods are supported:
◆ Manually initiated archival using NDMP (Network Data Management
Protocol) or the storage system’s dump command
◆ Manually initiated archival using a Windows backup utility
◆ Automatic archival using the Run Command After Operation feature with
your backup operation
The SnapVault software can be used to archive backup sets.

What Snap Manager does NOT do :


SnapManager 4.0 for Microsoft Exchange does not support the following uses:
◆ SnapManager does not support the capability to restore individual mailboxes
or public folders.
◆ SnapManager does not support SnapDrive versions prior to 4.2.1.
◆ SnapManager does not create or restore backups of Microsoft Exchange
databases that are stored on storage devices that are provided by companies
other than NetApp.
◆ SnapManager does not support Microsoft Windows 2000.
◆ SnapManager does not restore Microsoft Exchange Server 2003 and
Microsoft Exchange Server 2007 databases to an alternate location.
◆ SnapManager does not back up and restore Microsoft Exchange 5.5 and
Exchange 2000 databases.
◆ SnapManager is not supported for use on Windows Server 2003 IA-64
edition.

Ontap Upgrade 7.2.3 to 7.2.4 (Non disruptive Upgrade)

NetApp 3020 upgrade of Data OnTap 7.2.3 to Data OnTap 7.2.4

Performing a non-disruptive upgrade of our Network Appliance FAS 3020 (clustered filer configuration)

One of the benefits of having the clustered filers (FAS3020c) is that I can, in most cases, perform a system upgrade without having to disrupt services running on either system. The process is a little complex but well worth the payoff as in our environment I literally have thousands of students connecting to the storage at a time. Below is a slightly modified version of my notes from the upgrade (use at your own risk). I followed the directions from NetApp's upgrade guide. Although, I will note that their directions were not exact. I had differing outputs from commands at times which made me a little nervous. All in all the upgrade went pretty smooth and the systems have been running solid since.

Download from now.netapp.com under Download Software – Data OnTap – FAS 3020c

• new Shelf Firmware from now.netapp.com (all shelf firmware updates)
• new Disk Firmware from now.netapp.com (all disk firmware updates)
• newest release of Filer Firmware CFE 3.1
• newest GA Release of Data OnTap 7.2.3
• docs for Data OnTap 7.2.3

Copied and Made backups of files

• Mounted \\filerA\c$
• Mounted \\filerB\c$
• Made of backup of c$\etc\ folder on both systems (minus log files)
  - Copy to c$\backup\etc_8-24-2007
• From shelf zip file to the etc\shelf_fw on the both filerA and filerB
• From shelf zip file to the etc\shelf_fw on the both filerA and filerA
• From disk zip file to the etc\disk_fw on the both filerA and filerB
• From disk zip file to the etc\disk_fw on the both filerA and filerA

Shelf Firmware

• Login to the appliance console.
• Check current shelf firmware version ( > sysconfig -v )
• Enter Advanced privileges ( > priv set advanced )
• Start the update (> storage download shelf)
  - This will upgrade the shelf firmware on all the disk shelves in the system. (If you wish to only update the disk shelves attached to a specific adapter, enter storage download shelf adapter_number instead).
• Accept the update, Press y for yes and hit enter.
• To verify the new shelf firmware, ( > sysconfig -v )
• Exit Advanced privileges ( > priv set admin )

Disk Firmware

Disk firmware is automatically updated on reboot if there are updated files in the disk_fw folder. To keep the system from updating too many disks at once set or verify the following option.

• ( > options raid.background_disk_fw_update.enable)
  - if it is set to off, I recommend you change it to on

DataOnTap Update

1. Downloaded the newest General Deployment Release, in this case it was Data ONTAP 7.2.4.
2. Verified our system met all requirements for running the downloaded release, updates were required for Disk firmware and shelf firmware (which was done above)
3. Checked known problems and limitations of the new release to see if any would affect our environment.
4. Compared bug fixes from current version of OnTap 7.2.3 to new version of 7.2.4.
5. Downloaded newest documentation for 7.2.4

Update Procedure

With C$ mapped on both filers I ran the downloaded OS install (self extracting zip files) to the respective \etc directories. This is the first step and copies all the needed files over to the filers. Once completed, we perform the procedure below from the NOW upgrade guide for Windows Clients.

1. start the install on both systems ( > download )
2. Checked the cluster status ( > cf status ) to make sure cluster failover was enabled
3. Had filerB takeover services for filerA ( > cf takeover )
   - This causes filerA to reboot
4. During reboot of filerA hit ( ctrl-c ) to enter into maintenance mode
5. From maintenance mode type ( > halt ) to do a full reboot
6. Hit ( del ) during memory test to get to the CFE prompt
7. start the firmware update of the filer from the CFE> prompt using ( CFE> update_flash )
8. Now reboot, type ( bye ) at console after update was finished to reboot filerA
9. filerA is now in a …waiting for giveback state
10. Now to give services back to filerA we have to force it using ( > cf giveback –f ) from filerB
    - This is required since we are now on different version of DataOnTap between systems in the cluster.
11. Giveback successful, checked firmware and OS version on filerA using ( > sysconfig –v )
12. After checking services on both systems it's time to upgrade filerB
13. Have filerA take over the services of filerB ( > cf takeover –n )
14. Type ( > halt ) from filerB to reboot it
15. During reboot of filerB hit ( ctrl-c ) to enter into maintenance mode
16. From maintenance mode type ( > halt ) to do a full reboot
17. Hit ( del ) during memory test to get to the CFE prompt
18. start the firmware update of the filer from the CFE> prompt using ( CFE> update_flash )
19. Typed ( bye ) at console after update was finished to reboot filerB
20. filerB is now in a …waiting for giveback state
21. Now to give services back to filerB we have to force it using (> cf giveback –f ) from filerA
    - This is required since we are now on different version of DataOnTap between systems in the cluster.
22. Giveback successful, checked firmeware and os version on filerB using ( > sysconfig –v )
1. Both systems should now show the updated firmware and OnTap version 7.2.4
23. You should also notice that any out of date disk firmware is automatically updated.

My final steps were to test system connections

1. We use the following NetApp services: CIFS, FTP, HTTP, FCP via VMWARE. All worked fine. I Also checked our student websites and our web based FTP software that connects to the filer.
2. Checked Domain connection using cifs testdc ( filerA> cifs testdc )
   - appeared fine

Addt. Note: If you have MCCS cluster. Please make sure it is properly brought down before the reboot. Sometimes your luns doesn't come up.

Upgrading Shelf Firmware

To update the firmware without rebooting, during a maintenance window, complete the following:

1. Login to the storage system's console.

2. Type priv set advanced.

3. Enter storage download shelf. This will upgrade the shelf firmware on all the disk shelves in the system. (If you wish to only update the disk shelves attached to a specific adapter, enter storage download shelf adapter_number instead).

4. Press y for yes and hit enter.

5. To verify the new shelf firmware, run sysconfig -v.

VIFS on NetApp

Vifs (Virtual interfaces) are used for multi-homing and load balancing

A Single mode vif is for failovers only

Multi-mode vif is for load balancing

You can use a combination of multi-mode and single mode to be fully redundant and load balanced.

For more information on VIFS please check the NetApp now site.

Note: as of OnTap 7.2.3. there is a bug in Filerview with vifs and interfaces DO NOT modify interfaces or vifs using Filerview use command line and edit RC and hosts files to preserve settings.

To create a single mode vif

vif create single sample-vif0 e0a e0c

To configure an interface to use a vif

ifconfig sample-vif0 `hostname`-pub mediatype auto netmask 255.255.255.0 wins partner cluster-vif0

Make sure you reference the vif name not the interface name

To view all interfaces including vifs

Ifconfig –a

To view a single interface or vif (note you must view the vif in order to see ip address)

Ifconfig sample e0a

Ifconfig sample-vif0

To view all vifs status

vif status

To view single vif

vif status sample-vif0

To view statistics

vif stat sample-vif0

If you need to remove an interface from a vif

vif delete sample-vif0 e0c (This will remove e0c from sample-vif0)

To favor one interface in a VIF (this always be the one that’s used if up, if it ever goes down and then comes back up it will fail back to it)

vif favor e0a

To see more commands type the following at a command prompt.

Vif ?

A-SIS DeDuplication - Nut Shell

Supported volume size

3020 - 1 TB
3040 - 3 TB
3050 - 2 TB

1. Volume can never have crossed the maximum supported volume size to have the A-SIS deployed. Else A-SIS will fail. (Check the spec of Max volume size for rest of the filers)
2. Active/Active configuration fail over is not supported.

• Enable license & start A-SIS on a flexible volume.
  

license add a-sis

sis on volname

• Have it scheduled.

sis config -s schedule path

• Have it run on a existing volume.

sis start -s path

• A-SIS dedup needs to be re-enabled on copied, restored, cloned & renamed volumes.

sis off path
sis on path

• check volume dedup status.

sis status -l path

• Space saved by dedup status.

df -s volname

• Stop current A-SIS during an operation.

sis stop path

• Disable A-SIS

sis stop path
sis off path

RAID 4 vs. RAID DP (Video)

Comparison between RAID 4 vs. RAID DP

VMware VDI on NetApp (Video)

Vmware Virtual Desktop Infrastructure (VDI)

WAFL Check

Wafl check is a Data ONTAP utility to repair filer volumes which repair the inconsistency in the volumes.

Wafl check steps :

Data ONTAP (test.com)

login: root
Password:


Filer1> reboot

Total number of connected SCSI clients: 1 Number of r/w, online, mapped LUNs: 4

Warning: Rebooting with clustering disabled will terminate SCSITarget services and might cause data loss and application visible errors, or other OS failures on storage clients!!

CIFS local server on vfiler vfiler11 is shutting down...
CIFS local server on vfiler Vfiler12 is shutting down...
CIFS local server on vfiler vfiler11 has shut down...
CIFS local server on vfiler vfiler12 has shut down...
CIFS local server on vfiler vfiler0 is shutting down...
CIFS local server on vfiler vfiler0 has shut down...

Enter the number of minutes to wait before disconnecting [5]:
11 minute left until termination (^C to abort)...[LCD:info] REBOOTING

*** Warm reboot...

Clearing memory
Probing devices
Finding image...
Loading /pc-card:1,\x86\kernel\primary.krn

Starting Press CTRL-C for special boot menu
.................................................................................................................................................................................................................
Special boot options menu will be available.
Mon Feb 26 21:43:52 GMT [cf.ic.linkEstablished:info]: The Cluster Interconnect link has been established.
Mon Feb 26 21:43:53 GMT [cf.nm.nicTransitionUp:info]: Interconnect link 0 is UP

NetApp Release 7.0.5: Wed Aug 9 00:27:38 PDT 2006
Copyright (c) 1992-2006 Network Appliance, Inc.
Starting boot on Mon Feb 26 21:43:47 GMT 2007
Mon Feb 26 21:44:02 GMT [diskown.isEnabled:info]: software ownership has been enabled for this system
LUN Ownership using low range


Please choose one of the following:


(1) Normal boot.
(2) Boot without /etc/rc.
(3) Change password.
(4) Initialize owned disks (65 disks are owned by this filer).
(4a) Same as option 4, but create a flexible root volume.
(5) Maintenance mode boot.

Selection (1-5)? WAFL_check


In a cluster, you MUST ensure that the partner is (and remains) down,
or that takeover is manually disabled on the partner node,
because clustering software is not started or fully enabled
in WAFL_check mode.

FAILURE TO DO SO CAN RESULT IN YOUR FILESYSTEMS BEING DESTROYED
Continue with boot? y

add net 127.0.0.Mon Feb 26 21:48:01 GMT [cf.noDiskownShelfCount:info]: Disk shelf count functionality is not supported on software based disk ownership configurations.
0: gateway 127.0.0.1Mon Feb 26 21:48:04 GMT [fmmbx_instanceWorke:info]: Disk disk1:0-4.125L1 is a primary mailbox disk
Mon Feb 26 21:48:04 GMT [fmmbx_instanceWorke:info]: normal mailbox instance on primary side
Mon Feb 26 21:48:10 GMT [fmmbx_instanceWorke:info]: Disk disk1:0-4.125L0 is a backup mailbox disk
Mon Feb 26 21:48:10 GMT [fmmbx_instanceWorke:info]: normal mailbox instance on backup side
Mon Feb 26 21:48:10 GMT [cf.fm.partner:info]: Cluster monitor: partner 'filer2'
Mon Feb 26 21:48:10 GMT [cf.fm.timeMasterStatus:info]: Acting as cluster time slave
Mon Feb 26 21:48:14 GMT [localhost: cf.fm.launch:info]: Launching cluster monitor
Mon Feb 26 21:48:14 GMT [localhost: cf.fm.partner:info]: Cluster monitor: partner 'filer2'
Mon Feb 26 21:48:14 GMT [localhost: cf.fm.notkoverClusterDisable:warning]: Cluster monitor: cluster takeover disabled (restart)
Mon Feb 26 21:48:15 GMT [localhost: cf.fsm.takeoverOfPartnerDisabled:notice]: Cluster monitor: takeover of Filer2 disabled (cluster takeover disabled)
Mon Feb 26 21:48:15 GMT [localhost: raid.cksum.replay.summary:info]: Replayed 0 checksum blocks.
Mon Feb 26 21:48:15 GMT [localhost: raid.stripe.replay.summary:info]: Replayed 0 stripes.
Check vol01? y
Check vol02? n
Check vol03? y
Check vol04? y
Check vol0? n
Check vol05? y

Checking vol01...

WAFL_check NetApp Release 7.0.5

Starting at Mon Feb 26 21:50:32 GMT 2007

Phase 1: Verify fsinfo blocks.
Phase 2: Verify metadata indirect blocks.
Phase 3: Scan inode file.
Phase 3a: Scan inode file special files.
Phase 3a time in seconds: 6
Phase 3b: Scan inode file normal files.
(inodes 5%)
(inodes 10%)
(inodes 15%)
(inodes 20%)
(inodes 25%)
(inodes 30%)
(inodes 35%)
(inodes 41%)
(inodes 46%)
(inodes 51%)
(inodes 56%)
(inodes 61%)
(inodes 66%)
(inodes 71%)
(inodes 76%)
(inodes 82%)
(inodes 87%)
(inodes 92%)
(inodes 97%)
(inodes 99%)
(inodes 99%)
Phase 3b time in seconds: 2989
Phase 3 time in seconds: 2995
Phase 4: Scan directories.
Phase 4 time in seconds: 0
Phase 5: Check volumes.
Phase 5a: Check volume inodes
Phase 5a time in seconds: 0
Phase 5b: Check volume contents

Checking volume vol03...
Phase [5.1]: Verify fsinfo blocks.
Phase [5.2]: Verify metadata indirect blocks.
Phase [5.3]: Scan inode file.
Phase [5.3a]: Scan inode file special files.
Phase [5.3a] time in seconds: 20
Phase [5.3b]: Scan inode file normal files.
(inodes 5%)
(inodes 10%)
(inodes 15%)
(inodes 20%)
(inodes 25%)
(inodes 30%)
(inodes 35%)
(inodes 40%)
(inodes 45%)
(inodes 50%)
(inodes 55%)
(inodes 60%)
(inodes 65%)
(inodes 70%)
(inodes 75%)
(inodes 80%)
(inodes 85%)
(inodes 90%)
(inodes 95%)
Phase [5.3b] time in seconds: 5
Phase [5.3] time in seconds: 26
Phase [5.4]: Scan directories.
Phase [5.4] time in seconds: 6
Phase [5.6]: Clean up.
Phase [5.6a]: Find lost nt streams.
Phase [5.6a] time in seconds: 5
Phase [5.6b]: Find lost files.
Phase [5.6b] time in seconds: 16
Phase [5.6c]: Find lost blocks.
Phase [5.6c] time in seconds: 0
Phase [5.6d]: Check blocks used.
Phase [5.6d] time in seconds: 722
Phase [5.6] time in seconds: 744

Clearing inconsistency flag on volume vol03.

Volume vol03 WAFL_check time in seconds: 776
Inconsistent vol vol03 marked clean.
WAFL_check output will be saved to file /vol/vol03/etc/crash/WAFL_check

Checking volume vol04...
Phase [5.1]: Verify fsinfo blocks.
Phase [5.2]: Verify metadata indirect blocks.
Phase [5.3]: Scan inode file.
Phase [5.3a]: Scan inode file special files.
Phase [5.3a] time in seconds: 55
Phase [5.3b]: Scan inode file normal files.
(inodes 5%)
(inodes 10%)
(inodes 15%)
(inodes 20%)
(inodes 25%)
(inodes 30%)
(inodes 35%)
(inodes 40%)
(inodes 45%)
(inodes 50%)
(inodes 55%)
(inodes 60%)
(inodes 65%)
(inodes 70%)
(inodes 75%)
(inodes 80%)
(inodes 85%)
(inodes 90%)
(inodes 95%)
Phase [5.3b] time in seconds: 1419
Phase [5.3] time in seconds: 1474
Phase [5.4]: Scan directories.
Phase [5.4] time in seconds: 13
Phase [5.6]: Clean up.
Phase [5.6a]: Find lost nt streams.
Phase [5.6a] time in seconds: 10
Phase [5.6b]: Find lost files.
Phase [5.6b] time in seconds: 42
Phase [5.6c]: Find lost blocks.
Phase [5.6c] time in seconds: 4
Phase [5.6d]: Check blocks used.
Phase [5.6d] time in seconds: 1166

FS info: setting total inodes used to be 1692 (was 1693).
Phase [5.6] time in seconds: 1222
Clearing inconsistency flag on volume vol04.
Volume vol04 WAFL_check time in seconds: 2710
1530 error messages discarded
Indirect blocks cleared: 3
Block counts corrected: 1
Total inodes corrected.
1530 lost blocks collected into 1530 files in lost+found.
Volume vol04 Inconsistent vol vol04 marked clean.
WAFL_check output will be saved to file /vol/vol04/etc/crash/WAFL_check

Checking volume vol05...
Phase [5.1]: Verify fsinfo blocks.
contains snapmirrored qtrees. Committing these changes willbreak the synchronization between these qtrees andPhase [5.2]: Verify metadata indirect blocks.
their sources.Use snapmirror resync or initiPhase [5.3]: Scan inode file.
Phase [5.3a]: Scan inode file special files.alize to re-establish the mirror(s).
Phase [5.3a] time in seconds: 798
Phase [5.3b]: Scan inode file normal files.
(inodes 5%)
(inodes 10%)
(inodes 15%)
(inodes 20%)
(inodes 25%)
(inodes 30%)
(inodes 35%)
(inodes 40%)
(inodes 45%)
(inodes 50%)
(inodes 55%)
(inodes 60%)
(inodes 65%)
(inodes 70%)
(inodes 75%)
(inodes 80%)
(inodes 85%)
(inodes 90%)
(inodes 95%)
(inodes 100%)
Phase [5.3b] time in seconds: 3512
Phase [5.3] time in seconds: 4310
Phase [5.4]: Scan directories.
Phase [5.4] time in seconds: 15
Phase [5.6]: Clean up.
Phase [5.6a]: Find lost nt streams.
Phase [5.6a] time in seconds: 12
Phase [5.6b]: Find lost files.
Phase [5.6b] time in seconds: 49
Phase [5.6c]: Find lost blocks.
Phase [5.6c] time in seconds: 0
Phase [5.6d]: Check blocks used.
Phase [5.6d] time in seconds: 707
Phase [5.6] time in seconds: 768
Clearing inconsistency flag on volume vol05.
Volume vol05 WAFL_check time in seconds: 5094
Inconsistent vol vol05 marked clean.
Volume vol05 is a snapmirrored volume. Committing these changes will break thesynchronization between this
WAFL_check output will be saved to file /vol/vol05/etc/crash/WAFL_check
Phase 5b time in seconds: 9697
Phase 6: Clean up.
Phase 6a: Find lost nt streams.
Phase 6a time in seconds: 0
Phase 6b: Find lost files.
volume and its source, forcing you to restartthe snapmirror process. A better alternative may be toWAFL_check the source volume and reinitialize snapmirror by issuing a"snapmirror initialize" command on this volume.Phase 6b time in seconds: 4
Phase 6c: Find lost blocks.
phase 6c time in seconds:17
Phase 6d: check blocks used
phase 6d time in seconds: 51
phase 6 time in seconds: 72
Clearing inconsistency flag on aggregate vol05.
WAFL_check total time in seconds : 12764
Commit changes for aggregate aggr1 to disk? ? y

___________________________________________________________________________

Vfiler Limits

• 3 for filers with less than 1 GB of memory
• 5 for filers with between 1 GB and 3 GB of memory
• 11 for filers with 3 GB of memory or more

vfiler limit command to increase the limit to a maximum of up to 33 :

• 11 for filers with less than 1 GB of memory
• 26 for filers with between 1 GB and 2GB of memory
• 33 for filers with over 2 GB of memory

Disk Firmware upgrade

Upgrade steps:

• Download the disk firmware file X276_S10K7288F10.NA07.LOD from now.netapp.com
• Mount the root file system and place this file into /etc/disk_fw directory.
• Run disk_fw_update or set the flag for background upgradation as options raid.background_disk_fw_update.enable on
• Confirm the status for the upgrade by executing syconfig -v.

Why the upgrade is done...

Refer : http://netapp-solutions.blogspot.com/

Backup DFM configuration

Backup DFM configuration

dfm service stop

dfm service start sql

dfm backup create backup_file_name

Add Host filers to DFM

If your management station is on the same subnet as the appliances you want to manage, DataFabric Manager will probably discover them right away. But if your management station is on a different subnet, you must give DataFabric Manager the list of networks on which to look for those appliances.

dfm option set discoverHosts=on

dfm host add name-or-IP-address

Vmware - SRM (Site recovery Manager 1.0)

Netapp & Vmware - Disaster Recovery & Backup

• Vmware has introduced a beta version of SRM (Site Recovery Manager 1.0)
• Key feature include virtualization for Disaster recovery.
• Have reduced hardware requirement in production as well as at DR site.
• Data replicated to DR site through Netapp Snapmirror. (Through port 10566, fyi).
  
• Vmware SRM 1.0 supports ISCSI & FCP only. Support for other protocols in later versions.
• Netapp also have plans to introduce SnapManager VI. (specially for virtualization)

Also Refer : http://blogs.netapp.com/storage_nuts_n_bolts/2008/03/sneak-preview-1.html

Volume Autosize

Volume Autosize

vol autosize volname -m 600g -i 20g on

@ By default autosize on a volume is disabled. The "on" subcommand is used to enable the auto size. (Remember 600gb included the snapshots area as well). Plan the max spacing accordingly.

==================================================================
Tips:

The point at which the volume_grow process kicks off is defined by the option wafl_reclaim_threshold which in Data ONTAP versions prior to 7.2.4 was set at 98%

Depending on the size of the volume this may not give the volume enough time to grow before running out of space. With Data ONTAP 7.2.4 this flag was revised to have 5 levels:

wafl_reclaim_threshold_t: tiny vols < threshold =" 85%">

wafl_reclaim_threshold_s: small vols from 20gb to < threshold =" 90%">

wafl_reclaim_threshold_m: medium vols from 100gb to < threshold =" 92%">

wafl_reclaim_threshold_l: large vols from 500gb to < threshold =" 95%">

wafl_reclaim_threshold_ xl: xlarge vols from 1tb up threshold = 98%

With Data ONTAP versions prior to 7.2.4 you can modify this flag to a value lower than 98% (90% works for most block environments) using the following commands:

To view current value: priv set diag; printflag wafl_reclaim_threshold

To set current value: priv set diag; setflag wafl_reclaim_threshold

If you choose to modify the value you will need to add an entry to /etc/rc on each controller head in a cluster to preserve it over a reboot:

priv set –q diag; setflag wafl_reclaim_threshold 90; priv set;

It is NOT considered a best practice to modify the setting of wafl_reclaim_threshold in early versions of OnTap, the best option is to upgrade to 7.2.4

DFM requirements

DFM 3.2 requirements

Windows 2000 server
Number of Devices monitored - 101 - 250

Hardware Requirements

PC based on Intel with Dual 2-GHZ CPU (Xeon or Pentium 4)
8GB of free disk space, 40 GB recommended
ISCSI, FC LUN, or internal RAID disk
2GB of memory minimum

ISCSI fundamentals

ISCSI

iSCSI is a protocol defined by the Internet Engineering Task Force (IETF) which enables SCSI commands to be encapsulated in TCP/IP traffic, thus allowing access to remote storage over low cost IP networks.

What advantages would using an iSCSI Storage Area Network (SAN) give to your organization over using Direct Attached Storage (DAS) or a Fibre Channel SAN?

• iSCSI is cost effective, allowing use of low cost Ethernet rather than expensive Fibre architecture.

• Traditionally expensive SCSI controllers and SCSI disks no longer need to be used in each server, reducing overall cost.

• Many iSCSI arrays enable the use of cheaper SATA disks without losing hardware RAID functionality.

• The iSCSI storage protocol is endorsed by Microsoft, IBM and Cisco, therefore it is an industry standard.

• Administrative/Maintenance costs are reduced.

• Increased utilisation of storage resources.

• Expansion of storage space without downtime.

• Easy server upgrades without the need for data migration.

• Improved data backup/redundancy.