VNX/Celerra Multi path file system (mpfs) allocation.

Procedure might varies (VNX/Celerra)

If we are moving data from NetApp filer to VNX which is dedicated to NFS and CIFS and some of it is accessed by both cifs and nfs users. we may opt to choose mpfs setup on the clients and on VNX. create mpfs volumes share it using cifs and nfs and see the mpfs stats if the data is moving through mpfs. For that to be successful we need to install mpfs client software and need to have the usermappers setup on VNX and start testing from both the clients. Later on we can use emcopy or robocopy to copying data from NetApp to VNX

Step:1 Start MPFS service

server_setup server_2 -P mpfs -option start=32

Step:2 Create a new volume with a 256K stripe per EMC's recommendations

nas_volume -name mpfsvol -create -Stripe 262144 d54,d55,d56,d57,d58,d59,d60,d61

Step:3 Create a new metalvolume for the new volume:

nas_volume -name mpfsmtvol -create -Meta mpfsvol

Step:4 Create a new file system on this metavolume

nas_fs -name mpfs -create mpfsmtvol

Step:5 Create a new mountpoint

server_mountpoint server_2 -create /mpfs

Step:6 Mount file system

server_mount server_2 -option nolock,accesspolicy=NATIVE mpfs /mpfs

Step:7 Create new mountpoint for testing

server_mountpoint server_2 -create /mpfs/test

Step:8 Create CIFS share for this work

server_export server_2 -P cifs -n test$ -o netbios=server1 -comment 'MPFS CIFS Share for test' /mpfs/test

Step:9 Validate

server_mpfs server_2 -mountstatus | grep -i mpfs

server_df server_2 /mpfs

server_mpfs server_2

nas_volume -info mpfsmtvol

Step:10 Make sure system administrators are having MPFS client software installed!

Step:11  Verify stats if data is going through fiber channel interface or via CIFS

mpfsctl stats

DFM threshold values set from DFM server

These are some commands on how to set DFM alerts

From DFM server:

C:\Windows>dfm eventtype modify -v Critical aggregate-full
Event "aggregate-full" was left unchanged.
C:\Windows>dfm eventtype modify -v Error aggregate-almost-full
Modified event "aggregate-almost-full".

C:\Windows>dfm option set aggrFullThreshold=95
Changed aggregate full threshold (%) to 95.

C:\Windows>dfm option set aggrNearlyFullThreshold=90Changed aggregate nearly full threshold (%) to 90.

Moving Vfiler from one filer to another

Pre-requisite:

                 Required checklist info to implement changes

                                     To move vfiler  

·         Filer name

·         Aggregate name

·         Make sure to setup DR replications after moving vfiler to new filer?

 Start replicating volumes:

 (Example: we are moving labvfilerfrom lafiler1 to labfiler3. Here labfiler1 DR is labdrfiler1 and labfiler3 DR is labdrfiler3)

1.       ssh to labfiler1 to get volume info for labvfiler

labfiler1>  vfiler status -r labvfiler

labvfiler                       running

   ipspace: vfilers1

   IP address: 10.10.1.2 [vif1-110]

   Path: /vol/labvfiler [/etc]

   Path: /vol/labvfiler_vol1

   Path: /vol/labvfiler_vol2

   Path: /vol/labvfiler_vol3

   Path: /vol/labvfiler_vol4

   UUID: 2a0000d6-4594-1ew1-9e2f-007891w2fc3

labfiler1> vol size labvfiler

Warning: Volume 'labvfiler' has fs_size_fixed option set.  The file system

size may differ from the volume size.

See 'vol status -b' for more detail.

vol size: Flexible volume 'labvfiler' has size 5g.

labfiler1> vol size labvfiler_vol1

vol size: Flexible volume 'labvfiler_vol1' has size 2202g.

labfiler1> vol size labvfiler_vol2

vol size: Flexible volume 'labvfiler_vol2' has size 1975g.

labfiler1> vol size labvfiler_vol3

vol size: Flexible volume 'labvfiler_vol3' has size 1714g.

labfiler1> vol size labvfiler_vol4

vol size: Flexible volume 'labvfiler_vol4' has size 3319g. 

2.       Once you collect volume info, ssh to labfiler3 and check on which aggregate has enough space

3.       Create NEW volumes on labfiler3 and start replicating

Ø  vol create volumename –s none aggrname XXXg

Ø  vol restrict volumename

Ø  snapmirror initialize –S source:volumename destination:volumename

labfiler3> vol create labvfiler -s none aggr0 5g

Creation of volume 'labvfiler' with size 5g on containing aggregate

'aggr0' has completed.

labfiler3> vol create labvfiler_vol1 -s none aggr0 2202g

Creation of volume 'labvfiler_vol1' with size 2202g on containing aggregate

'aggr0' has completed.

labfiler3> vol create labvfiler_vol2 -s none aggr0 1975g

Creation of volume 'labvfiler_vol2' with size 1975g on containing aggregate

'aggr0' has completed.

labfiler3> vol create labvfiler_vol3 -s none aggr0 1714g

Creation of volume 'labvfiler_vol3' with size 1714g on containing aggregate

'aggr0' has completed.

labfiler3> vol create labvfiler_vol4 -s none aggr0 3319g

Creation of volume 'labvfiler_vol4' with size 3319g on containing aggregate

labfiler3> vol restrict labvfiler

Volume 'labvfiler' is now restricted.

labfiler3> vol restrict labvfiler_vol1

Volume 'labvfiler_vol1' is now restricted.

labfiler3> vol restrict labvfiler_vol2

Volume 'labvfiler_vol2' is now restricted.

labfiler3> vol restrict labvfiler_vol3

Volume 'labvfiler_vol3' is now restricted.

labfiler3> vol restrict labvfiler_vol4

Volume 'labvfiler_vol4' is now restricted.

labfiler3> snapmirror initialize -S labfiler1.rr.snd.com:labvfiler labfiler3:labvfiler

Transfer started.

Monitor progress with 'snapmirror status' or the snapmirror log.

labfiler3> snapmirror initialize -S labfiler1.rr.snd.com:labvfiler_vol1 labfiler3:labvfiler_vol1

Transfer started.

Monitor progress with 'snapmirror status' or the snapmirror log.

labfiler3> snapmirror initialize -S labfiler1.rr.snd.com:labvfiler_vol2 labfiler3:labvfiler_vol2

Transfer started.

Monitor progress with 'snapmirror status' or the snapmirror log.

labfiler3> snapmirror initialize -S labfiler1.rr.snd.com:labvfiler_vol3 labfiler3:labvfiler_vol3

Transfer started.

Monitor progress with 'snapmirror status' or the snapmirror log.

labfiler3> snapmirror initialize -S labfiler1.rr.snd.com:labvfiler_vol4 labfiler3:labvfiler_vol4

Transfer started.

Monitor progress with 'snapmirror status' or the snapmirror log. 

4.       Update snapmirror.conf with scheduling depends on cutover/move date

5.       As labfiler3 d2d is labdrfiler3, we have to replicate all volume from labfiler3 to labdrfiler3.

Create same volumes on labdrfiler3 and start snapmirror initialize once labfiler1 to labfiler3 base transfer completes. Here we can also start copying from labdrfiler1 to labdrfiler3 

ON DAY OF CUTOVER

********************************

** ON labfiler1 ***************

*********************************

vfiler stop labvfiler

_______________________________

********************************

** ON labfiler3 ***************

*********************************

snapmirror update -S labfiler1.rr.snd.com:labvfiler        labfiler3:labvfiler 

snapmirror update -S labfiler1.rr.snd.com:labvfiler_vol1   labfiler3:labvfiler_vol1

snapmirror update -S labfiler1.rr.snd.com:labvfiler_vol2     labfiler3:labvfiler_vol2

snapmirror update -S labfiler1.rr.snd.com:labvfiler_vol3     labfiler3:labvfiler_vol3 

snapmirror update -S labfiler1.rr.snd.com:labvfiler_vol4     labfiler3:labvfiler_vol4

snapmirror status labfiler3:labvfiler 

snapmirror status labfiler3:labvfiler_vol1

snapmirror status labfiler3:labvfiler_vol2

snapmirror status labfiler3:labvfiler_vol3 

snapmirror status labfiler3:labvfiler_vol4

snapmirror quiesce labfiler3:labvfiler 

snapmirror quiesce labfiler3:labvfiler_vol1

snapmirror quiesce labfiler3:labvfiler_vol2

snapmirror quiesce labfiler3:labvfiler_vol3 

snapmirror quiesce labfiler3:labvfiler_vol4

snapmirror break labfiler3:labvfiler 

snapmirror break labfiler3:labvfiler_vol1

snapmirror break labfiler3:labvfiler_vol2

snapmirror break labfiler3:labvfiler_vol3 

snapmirror break labfiler3:labvfiler_vol4

vol options labvfiler_vol1 fs_size_fixed off

vol options labvfiler_vol2 fs_size_fixed off

vol options labvfiler_vol3 fs_size_fixed off

vol option labvfiler_vol4 fs_size_fixed off

vfiler create labvfiler -r /vol/labvfiler

ifconfig vif1-110 alias 10.10.1.2 netmask 255.255.255.0

Updated below entry in \etc\rc 

ifconfig vif1-110 alias 10.10.1.2 netmask 255.255.255.0

vfiler run labvfiler cifs shares

vfiler run labvfiler exportfs

_______________________________

********************************

** ON labfiler1 ***************

********************************

vol offline labvfiler 

vol offline labvfiler_vol1

vol offline labvfiler_vol2

vol offline labvfiler_vol3 

vol offline labvfiler_vol4

vol status labvfiler 

vol status labvfiler_vol1

vol status labvfiler_vol2

vol status labvfiler_vol3 

vol status labvfiler_vol4

********************************

** ON labdrfiler1 ***************

********************************

vol offline labvfiler 

vol offline labvfiler_vol1

vol offline labvfiler_vol2

vol offline labvfiler_vol3 

vol offline labvfiler_vol4

After 1 days you can destroy volumes on OLD filer once you have no issues with vfiler 

********************************

** ON labfiler1 & labdrfiler1 **

********************************

After confirming with NAS engineering team and L3's, proceed with vol destroy

vol destroy labvfiler

y

vol destroy labvfiler_vol1

y

vol destroy labvfiler_vol11

y

vol destroy labvfiler_vol2

y

vol destroy labvfiler_vol3 

y

vol destroy labvfiler_vol4

y

Please leave your comments if you have any questions.

Hope this procedure helps many.

Thanks

SANNASDAS

NetApp - Cluster takeover and giveback procedure.

1.  To take over partner node, we have to make sure there will be 1-2 minutes connectivity lost (CIFS/NFS)
2. 2.      Release ASUP before giveback options autosupport.doit per-takeovernode1
   3.       node2> cf takeover –f (node1 is takeover by node2)
   4.       node2> cf: takeover initiated by operator
   5.       node2 (takeover)> – Which means takeover completed.
   6.       node2 (takeover)> partner  (To see partner status )
   7.       node2 /node1>
   8.       To do giveback follow from step 9
   9.       When the system isn’t in partner context  and if prompt shows as “ node1/node2>”, follow from step 4, if prompt shows as “node2(takeover)>” follow from step 5
   10.   node1/node2>  partner
   11.   node2(takeover)> cf status
   node2 has taken over node1.
   node1 is ready for giveback.
   12.   Release ASUP before giveback options autosupport.doit per-giveback1node
   13.   node2(takeover)> cf giveback
   14.   If cf giveback is unsuccessful, try forcegiveback..
   15.   node2(takeover)>  cf giveback –f
   16.   once the giveback completes, login to node1 and do health check
   17.   node1> sysconfig –r
   18.   node1>  sysconfig –v
   19.   node1>  storage show disk –p
   20.   node1>  vfiler status –r
   21.   node1>  vfiler run vfilername exportfs
   22.   node1>  vfiler run vfilername cifs shares
   23.   node1>  vol status –s
   24.   node1>  vif status
   25.   node1>  ifconfig –a
   26.   node1> options autosupport.doit post givenback1node

NetApp - Disk Sanitization

To Sanitize all disks except vol0 disks (root volume)

Sanitization is enabled by default for all DOT 8.2+ code, if not enabled, run below command to enable.

options licensed_feature.disk_sanitization.enable on 

If DOT 7.xx to 8.1

·            license add XXXXXX  (You can get license from NetApp)

§  Remove controllers from DFM server to avoid any critical/error/warning alerts from DFM.

§  Offline/destroy all volumes and aggregates leaving vol0 and aggr0.

·                        vol offline volumename

·                        vol destroy volumename

·                        aggr offline aggregatename

·                        aggr destroy aggregatename

·                        CF disable (Disable partner node)

·                        Reboot the filer (Obtain approval prior to rebooting the filer)

·                        Send Final ASUP is issued -- options autosupport.doit decommissioning”

§  Once ASUP is Complete, turn off alerts. 

§  Determine all spare drives using the following command(s):

·         vol status -r

·         vol status -s

§  Begin the sanitization process using the following command(s):

·         disk sanitize start  <all numbered drives>  or disk sanitize start -c 7 <all numbered drives>

·         Monitor status using the following command(s):

§  disk sanitize status

§  vol status –m

§  Unsanitized disks will go thru a SECOND PASS process, rerun the “disk sanitize start” command against drives that fail the sanitization

§  Save sanitization logs

§  rdfile /etc/log/sanitized_disks or rdfile /etc/sanitized_disks and save logs 

To Sanitize vol0 disks (root volume)

Ø  Release 3 sanitize disks (disk sanitize release <disk_list>

Ø  Once you release the sanitized disk, it will be in unowned state (disk show –n)

Ø  Assign the disks back to same node (disk assign <disk_list>

Ø  Once you assign disk, it will be in broken disks (vol status –f)

Ø  Unfail the broken/bad disk (disk unfail –s <disk>

Ø  Now you will see all disks under spare (vl status –s)

Ø  create new aggregate with released disks (aggr create newaggr0 –d <disk_list>), it will take time to create aggregate as disks has to be zeroed, check status with (aggr status new aggr0 –v)

Ø  check current vol0 size (vol size vol0)

Ø  create new volume in newly created aggregate (vol create newvol0 newaggr0 <size>)

Ø  restrict newvol0 before starting copy (vol restrict newvol0)

Ø  Copy vol0 to newvol0 (vol copy start –s vol0 newvol0)

Ø  Once copy is 100%, make newvol0 online (vol online newvol0)

Ø  Make newvol0 as root (vol options newvol0 root)

Ø  Rename vol0/newvol0 & aggr0/newaggr0

Ø  To rename – vol rename vol0 oldvol0, vol rename newvol0 vol0, aggr rename aggr0 oldaggr0, aggr rename newaggr0 aggr0

Ø  Reboot node (reboot)

Ø  Once reboot completes, it will take newly created volume (vol0) as root volume

Ø  Check the status (vol status, aggr status)

Ø  Delete oldvol0 and oldaggr0 (vol offline oldvol0, vol destroy oldvol0, aggr offline oldaggr0, aggr destroy oldaggr0)

Ø  Once we delete old volume/aggregate it will release all disks which are in oldaggr0

Ø  Check the spares (vol status –s)

Ø  Start sanitize (disk sanitize start  <disk_list>)

Ø  Check status (disk sanitize status)

Ø  Once sanitization is 100%, collect logs.

Changing root volume and aggregates - NetApp

Here is the procedure for changing root volume (VOL0) and aggr0 for NetApp Filers

- To do this we need 3 spared disks (Note all disk ID's)
- Priv set advanced
- aggr create newaggr0 -d x.x.x  x.x.x  x.x.x  (Here x.x.x means disk ID's)
- vol create newvol0 newaggr0 337g
- vol restrict newvol0
- vol copy start -S vol0 newvol0 (It will take probably 5 mins to copy)
- vol online newvol0 (Only after copy completes)
- vol options newvol0 root
- reboot (root volume will change from vol0 to newvol0 on reboot)

We will follow this procedure to scrub disks which are owned by vol0.

TCP/IP & Subnet