Migration to AIX PCM combined with OS update using Alternate Disk Copy

Source:

root@pdw02vir:/#lsdev -l hdisk0 -F uniquetype

disk/fcp/2145

root@pdw02vir:/#lsattr -El hdisk0 -a PCM -F value

PCM/friend/sddpcm

root@pdw02vir:/#lsattr -El hdisk0 -a queue_depth -F value

20

root@pdw02vir:/# manage_disk_drivers -d IBMSVC -o AIX_AAPCM

********************** ATTENTION *************************

For the change to take effect the system must be rebooted

root@pdw02vir:/#alt_disk_copy -d hdisk0 -B

Calling mkszfile to create new /image.data file.

Checking disk sizes.

Creating cloned rootvg volume group and associated logical volumes.

Creating logical volume alt_hd5.

Creating logical volume alt_hd6.

Creating logical volume alt_hd8.

Creating logical volume alt_hd4.

Creating logical volume alt_hd2.

Creating logical volume alt_hd9var.

Creating logical volume alt_hd3.

Creating logical volume alt_hd1.

Creating logical volume alt_hd10opt.

Creating logical volume alt_hd11admin.

Creating logical volume alt_loglv.

Creating logical volume alt_livedump.

Creating logical volume alt_sysdumplv2.

Creating logical volume alt_ocsinv.

Creating logical volume alt_nagios.

Creating logical volume alt_netbackup.

Creating logical volume alt_paging01.

Creating /alt_inst/ file system.

Creating /alt_inst/admin file system.

Creating /alt_inst/home file system.

Creating /alt_inst/logs file system.

Creating /alt_inst/opt file system.

Creating /alt_inst/opt/nagios file system.

Creating /alt_inst/opt/netbackup file system.

Creating /alt_inst/opt/ocsinventory file system.

Creating /alt_inst/tmp file system.

Creating /alt_inst/usr file system.

Creating /alt_inst/var file system.

Creating /alt_inst/var/adm/ras/livedump file system.

Generating a list of files

for backup and restore into the alternate file system…

Backing-up the rootvg files and restoring them to the alternate file system…

Modifying ODM on cloned disk.

Building boot image on cloned disk.

forced unmount of /alt_inst/var/adm/ras/livedump

forced unmount of /alt_inst/var/adm/ras/livedump

forced unmount of /alt_inst/var

forced unmount of /alt_inst/var

forced unmount of /alt_inst/usr

forced unmount of /alt_inst/usr

forced unmount of /alt_inst/tmp

forced unmount of /alt_inst/tmp

forced unmount of /alt_inst/opt/ocsinventory

forced unmount of /alt_inst/opt/ocsinventory

forced unmount of /alt_inst/opt/netbackup

forced unmount of /alt_inst/opt/netbackup

forced unmount of /alt_inst/opt/nagios

forced unmount of /alt_inst/opt/nagios

forced unmount of /alt_inst/opt

forced unmount of /alt_inst/opt

forced unmount of /alt_inst/logs

forced unmount of /alt_inst/logs

forced unmount of /alt_inst/home

forced unmount of /alt_inst/home

forced unmount of /alt_inst/admin

forced unmount of /alt_inst/admin

forced unmount of /alt_inst

forced unmount of /alt_inst

Changing logical volume names in volume group descriptor area.

Fixing LV control blocks…

Fixing file system superblocks…

root@pdw02vir:/#alt_rootvg_op -W -d hdisk0

Waking up altinst_rootvg volume group …

root@pdw02vir:/#mdownload

root@pdw02vir:/#alt_rootvg_op -C -b update_all -l /download/AIX/PTF/Aix-71-05-03/


installp: * * * A T T E N T I O N ! ! !
Software changes processed during this session require
any diskless/dataless clients to which this SPOT is
currently allocated to be rebooted.
install_all_updates: The following filesets have been selected as updates
to currently installed software:

bos.cluster.solid 7.1.1.15

<< End of Fileset List >>

install_all_updates: Performing installp update.
+—————————————————————————–+
Pre-installation Verification…
+—————————————————————————–+
Verifying selections…done
Verifying requisites…done
Results…

WARNINGS
——–
Problems described in this section are not likely to be the source of any
immediate or serious failures, but further actions may be necessary or
desired.

Already Installed
—————–
The number of selected filesets that are either already installed
or effectively installed through superseding filesets is 1. See
the summaries at the end of this installation for details.

NOTE: Base level filesets may be reinstalled using the “Force”
option (-F flag), or they may be removed, using the deinstall or
“Remove Software Products” facility (-u flag), and then reinstalled.

<< End of Warning Section >>

+—————————————————————————–+
BUILDDATE Verification …
+—————————————————————————–+
Verifying build dates…done
FILESET STATISTICS
——————
1 Selected to be installed, of which:
1 Already installed (directly or via superseding filesets)
—-
0 Total to be installed


Pre-installation Failure/Warning Summary
—————————————-
Name Level Pre-installation Failure/Warning
——————————————————————————-
bos.cluster.solid 7.1.1.15 Already superseded by bos.cluster.rte

install_all_updates: Log file is /var/adm/ras/install_all_updates.log
install_all_updates: Result = SUCCESS
root@pdw02vir:/#

root@pdw02vir:/#vi /alt_inst/etc/firstboot

#! /bin/ksh
print “INFO: adjust hdisk attributes”

lsdev -c adapter | grep fcs | awk ‘{print $1}’ | xargs -I [] chdev -l [] -a num_cmd_elems=2048 -a max_xfer_size=0x200000
lsmpio -q | grep 2145| awk ‘{print $1}’ | xargs -I [] chdev -l [] -a queue_depth=128 -a max_transfer=0x100000 -a algorithm=shortest_queue -a reserve_policy=no_reserve
print “INFO: uninstalling SDDPCM”
installp -u devices.sddpcm.$(uname -v)$(uname -r).rte devices.fcp.disk.ibm.mpio.rte
print “INFO: perform reboot”
reboot

root@pdw02vir:/#chmod a+x /alt_inst/etc/firstboot

root@pdw02vir:/#alt_rootvg_op -S -t

Putting volume group altinst_rootvg to sleep …

Building boot image on cloned disk.

forced unmount of /alt_inst/var/adm/ras/livedump

forced unmount of /alt_inst/var/adm/ras/livedump

forced unmount of /alt_inst/var

forced unmount of /alt_inst/var

forced unmount of /alt_inst/usr

forced unmount of /alt_inst/usr

forced unmount of /alt_inst/tmp

forced unmount of /alt_inst/tmp

forced unmount of /alt_inst/opt/ocsinventory

forced unmount of /alt_inst/opt/ocsinventory

forced unmount of /alt_inst/opt/netbackup

forced unmount of /alt_inst/opt/netbackup

forced unmount of /alt_inst/opt/nagios

forced unmount of /alt_inst/opt/nagios

forced unmount of /alt_inst/opt

forced unmount of /alt_inst/opt

forced unmount of /alt_inst/logs

forced unmount of /alt_inst/logs

forced unmount of /alt_inst/home

forced unmount of /alt_inst/home

forced unmount of /alt_inst/admin

forced unmount of /alt_inst/admin

forced unmount of /alt_inst

forced unmount of /alt_inst

Fixing LV control blocks…

Fixing file system superblocks…

root@pdw02vir:/#shutdown -Fr

Fonte: http://powercampus.de/en/migration-to-aix-pcm-combined-with-os-update-using-alternate-disk-copy/

On most AIX systems the SP or TL level is updated at regular intervals. It makes sense to perform the migration from SDDPCM to AIX PCM together with such an update. This saves time and some reboots, which otherwise have to be done because of the multipathing migration.

In our blog post “Migration from SDDPCM to AIX-PCM” we had already shown the migration for standalone systems.

Here, the migration from SDDPCM to AIX-PCM will be shown as part of an OS update, using the Alternate Disk Copy method. The procedure is roughly the following:

  1. Unmirroring the rootvg to get a free disk for Alternate Disk Copy.
  2. Change the Path Control Module (PCM) to AIX PCM.
  3. Creating the altinst_rootvg.
  4. Removal of fixes in the altinst_rootvg.
  5. Performing the OS update on the altinst_rootvg.
  6. Installing fixes in the altinst_rootvg.
  7. Adding a firstboot script to set disk attributes.
  8. Change the Path Control Module (PCM) back to SDDPCM.
  9. Booting from the altinst_rootvg.

On our example system AIX 7.1 TL5 SP2 is installed, the disks are SVC disks connected via virtual FC adapters. SDDPCM is the currently active multipathing driver:

# oslevel -s
7100-05-02-1810
# lsdev -l hdisk0 -F uniquetype
disk/fcp/2145
aix01:/root> lsattr -El hdisk0 -a PCM -F value
PCM/friend/sddpcm
#

As stated in the blog post above, some disk attributes change when migrating to AIX PCM. Therefore, you should take a close look at the current attributes in order to take them over later (at least partially). By way of example, we only look at the attribute queue_depth, which currently has the value 120:

# lsattr -El hdisk0 -a queue_depth -F value
120
#

Our system has a mirrored rootvg:

# lsvg -p rootvg
rootvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk3            active            399         232         00..01..71..80..80
hdisk0            active            399         240         00..01..79..80..80
#

The system was booted from the hdisk0:

# bootinfo -b
hdisk0
#

So we leave hdisk0 in the rootvg and remove  hdisk3 from the rootvg to get a free disk for Alternate Disk Copy.

# unmirrorvg rootvg hdisk3
0516-1246 rmlvcopy: If hd5 is the boot logical volume, please run 'chpv -c <diskname>'
        as root user to clear the boot record and avoid a potential boot
        off an old boot image that may reside on the disk from which this
        logical volume is moved/removed.
0516-1804 chvg: The quorum change takes effect immediately.
0516-1144 unmirrorvg: rootvg successfully unmirrored, user should perform
        bosboot of system to reinitialize boot records.  Then, user must modify
        bootlist to just include:  hdisk0.
# reducevg rootvg hdisk3
# chpv -c hdisk3
# bootlist -m normal hdisk0
#

Before we create a copy of the rootvg using Alternate Disk Copy, we temporarily change the system to AIX PCM without, however, rebooting. If then the altinst_rootvg is generated, the conversion to AIX PCM is already done in altinst_rootvg!

# manage_disk_drivers -d IBMSVC -o AIX_AAPCM
********************** ATTENTION *************************
  For the change to take effect the system must be rebooted
#

At the end of the OS update, we then undo this change on the rootvg to have the original state with SDDPCM.

After these preparations we start now the alt_disk_copy command:

# alt_disk_copy -d hdisk3 -B
Calling mkszfile to create new /image.data file.
Checking disk sizes.
Creating cloned rootvg volume group and associated logical volumes.
Creating logical volume alt_hd5.
Creating logical volume alt_hd6.
Creating logical volume alt_hd8.
…
#

Some fixes are installed on the system, which we remove from the altinst_rootvg before the update:

# emgr -l
ID  STATE LABEL      INSTALL TIME      UPDATED BY ABSTRACT
=== ===== ========== ================= ========== ======================================
1    S    102m_ifix  10/14/18 10:48:18            IFIX for Openssl CVE on 1.0.2m       
2    S    IJ03121s0a 10/14/18 10:49:04            IJ03121 for AIX 7.1 TL5 SP00         
3    S    IJ05822s2a 10/14/18 10:49:18            a potential security issue exists    
…
#

Activation of the altinst_rootvg:

# alt_rootvg_op -W -d hdisk3
Waking up altinst_rootvg volume group ...
#

And removal of the fixes:

# INUCLIENTS=1 /usr/sbin/chroot /alt_inst /usr/sbin/emgr –r -n 3
+-----------------------------------------------------------------------------+
Efix Manager Initialization
+-----------------------------------------------------------------------------+
Initializing log /var/adm/ras/emgr.log ...
Accessing efix metadata ...
Processing efix label "IJ05822s2a" ...
…
Operation Summary
+-----------------------------------------------------------------------------+
Log file is /var/adm/ras/emgr.log

EFIX NUMBER       LABEL               OPERATION              RESULT           
===========       ==============      =================      ==============   
1                 IJ05822s2a          REMOVE                 SUCCESS          

Return Status = SUCCESS
# INUCLIENTS=1 chroot /alt_inst /usr/sbin/emgr -r -n 2
…
# INUCLIENTS=1 chroot /alt_inst /usr/sbin/emgr -r -n 1
…

(Note: please do not forget the variable INUCLIENTS, this signals that the operation is taking place in an alternate boot environment!)

Now we mount the LPP source for the OS update via NFS from our NIM server:

# mount aixnim:/export/nim/lpps/aix710503lpp /mnt
#

The OS update can now be done in the altinst_rootvg:

# alt_rootvg_op -C -b update_all -l /mnt
Installing optional filesets or updates into altinst_rootvg...
install_all_updates: Initializing system parameters.
install_all_updates: Log file is /var/adm/ras/install_all_updates.log
install_all_updates: Checking for updated install utilities on media.
…
installp:  * * * A T T E N T I O N ! ! !
        Software changes processed during this session require
        any diskless/dataless clients to which this SPOT is
        currently allocated to be rebooted.
install_all_updates: Log file is /var/adm/ras/install_all_updates.log
install_all_updates: Result = SUCCESS
#

Finally, we install some fixes. We first mount the directory /mnt with the fixes in the altinst_rootvg:

# mount -v namefs /mnt /alt_inst/mnt
# INUCLIENTS=1 chroot /alt_inst /usr/sbin/emgr -e /mnt/emgr/ppc/102p_fix.181127.epkg.Z
+-----------------------------------------------------------------------------+
Efix Manager Initialization
+-----------------------------------------------------------------------------+
Initializing log /var/adm/ras/emgr.log ...
Efix package file is: /mnt/emgr/ppc/102p_fix.181127.epkg.Z
…
EPKG NUMBER       LABEL               OPERATION              RESULT           
===========       ==============      =================      ==============   
1                 102p_fix            INSTALL                SUCCESS          

Return Status = SUCCESS
# INUCLIENTS=1 chroot /alt_inst /usr/sbin/emgr -e /mnt/emgr/ppc/IJ09621s3a.181001.epkg.Z
…
# INUCLIENTS=1 chroot /alt_inst /usr/sbin/emgr -e /mnt/emgr/ppc/IJ11545s0a.181127.epkg.Z
…
# umount /alt_inst/mnt
#

To set the desired disk attributes and uninstall SDDPCM we use a firstboot script:

# cat /alt_inst/etc/firstboot

#! /bin/ksh

print “INFO: adjust hdisk attributes”

chdev -Pl hdisk0 -a queue_depth=128 -a max_transfer=0x100000 -a algorithm=shortest_queue -a reserve_policy=no_reserve

chdev -Pl hdisk1 -a queue_depth=128 -a max_transfer=0x100000 -a algorithm=shortest_queue -a reserve_policy=no_reserve

chdev -Pl hdisk2 -a queue_depth=128 -a max_transfer=0x100000 -a algorithm=shortest_queue -a reserve_policy=no_reserve

chdev -Pl fcs0 -a num_cmd_elems=2048 -a max_xfer_size=0x200000

chdev -Pl fcs1 -a num_cmd_elems=2048 -a max_xfer_size=0x200000

print “INFO: uninstalling SDDPCM”

installp -u devices.sddpcm.$(uname -v)$(uname -r).rte devices.fcp.disk.ibm.mpio.rte

print “INFO: perform reboot”

reboot

# chmod a+x /alt_inst/etc/firstboot
#

The script should, if used, be adapted to your own needs. There, you should customize all the desired disk attributes (queue_depth, reserve_policy, …). The sample script here is just to indicate what you could do!

The altinst_rootvg is now updated and converted to AIX PCM. We disable the altinst_rootvg so that it can be booted.

# alt_rootvg_op –S -t
Putting volume group altinst_rootvg to sleep ...
forced unmount of /alt_inst/var/adm/ras/livedump
…
forced unmount of /alt_inst
Fixing LV control blocks...
Fixing file system superblocks...
#

(Note: please do not forget the option “-t“, this creates a new boot image!)

But before we boot from the altinst_rootvg, we change the multipathing driver back to SDDPCM on the rootvg!

# manage_disk_drivers -d IBMSVC -o NO_OVERRIDE
********************** ATTENTION *************************
  For the change to take effect the system must be rebooted
#

Finally we change the bootlist to altinst_rootvg (hdisk3):

# bootlist -m normal hdisk3
#

And last but not least we reboot:

# shutdown –r now

SHUTDOWN PROGRAM
Tue Apr 16 19:49:08 CEST 2019

Broadcast message from root@aix01 (tty) at 19:49:08 ...

PLEASE LOG OFF NOW ! ! !
System maintenance in progress.
All processes will be killed now.
…

-------------------------------------------------------------------------------
                                Welcome to AIX.
                   boot image timestamp: 19:45:08 04/16/2019
                 The current time and date: 19:51:11 04/16/2019
        processor count: 2;  memory size: 4096MB;  kernel size: 36847630
       boot device: /vdevice/vfc-client@3000000a/disk@5005076XXXXXXXXX:2
-------------------------------------------------------------------------------
…
Multi-user initialization completed
INFO: adjust hdisk attributes
hdisk0 changed
INFO: uninstalling SDDPCM
…
Installation Summary
--------------------
Name                        Level           Part        Event       Result
-------------------------------------------------------------------------------
devices.sddpcm.71.rte       2.7.1.1         ROOT        DEINSTALL   SUCCESS   
devices.sddpcm.71.rte       2.7.1.1         USR         DEINSTALL   SUCCESS   
devices.fcp.disk.ibm.mpio.r 1.0.0.25        USR         DEINSTALL   SUCCESS   
INFO: perform reboot
Rebooting . . .
…

AIX Version 7
Copyright IBM Corporation, 1982, 2018.
Console login:

(In the output you can see the actions of the firstboot script: changing disk attributes, uninstalling SDDPCM and rebooting.)

After logging in we check the OS version, the used multipathing driver and some disk attributes:

# oslevel -s
7100-05-03-1846
# lsdev -l hdisk0 -F uniquetype
disk/fcp/mpioosdisk
# lsattr -El hdisk0 -a PCM -F value
PCM/friend/fcpother
# lsattr -El hdisk0 -a queue_depth -F value
120
# genkex|grep pcm
         5ae0000    60000 /usr/lib/drivers/aixdiskpcmke
# lslpp -l|grep sddpcm
#

We have successfully completed the migration from SDDPCM to AIX PCM together with an OS update. Using scripts this can be automated further.

We have tested this procedure for AIX 7.1 and AIX 7.2. So far, we have not been able to carry out a test for PowerHA for reasons of time.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.