xaminmo: (Josh 2004 Happy)
[personal profile] xaminmo
I ran into an issue that might be procedural, but I though you guys might want to know anyway.

We are pursuing with IBM HW support as of 2013-07-18.
I am going to test further in my lab. I suspect this may be related to
bkprofata and rstprofdata copying over some internal seed for MAC addresses

I plan to try this in my lab on p5 rackmount servers via an HMC.
If I can reproduce it there, then I expect support's response to be "don't do that".
As such, I also will try a factory reset to see if that will clear the condition.

If I cannot reproduce it there, then it's either SDMC/FSM related (which is going away),
or it's blade/Flex Node related (No other test resources, but maybe L3 can help).

If L3 decides that rstprofdata cannot be used on a different system,
then I would want them to A) Limit the command to that functionality,
and B) Update documentation for both commands to reflect this.

bkprofdata & rstprofdata were used to clone the LPAR layout from one blade to another.
To reset the WWNs, I was able to delete and re-add the virtual fibre adapters.
New LPARs and new virtual fibre adapters automatically get WWNs with the blade/node number as part of the WWN.
This part works as I would expect.

To reset the MAC addresses, this did not work.
Delete and re-add virtual ethernet adapters does not change the MAC addresses.
Adding a new adapter that did not exist before to the same slot number,
on the same LPAR ID, on two different Flex nodes, and both get the same MAC accress.

Current resolution is to override the MAC address with a user specified value in the LPAR profile.
This can be done from Profile -> Virtual -> Ethernet -> Advanced -> checkbox

Change from commandline:
chsyscfg -m Server-7895-23X-SN1012345 -r prof -i \

To remove and Readd:
chsyscfg -m Server-7895-23X-SN1012345 -r prof -i \
chsyscfg -m Server-7895-23X-SN1012345 -r prof -i \

I've never seen this happen on any other POWER series servers, and I've built a lot of p7 systems, ranging from p710 to p780, including matching LPARs between CECs. This is on top of the whole slew of LPARable systems I've built and/or supported.

I looked into the profile data backup files themselves, and there is no mention of system serial, system name, WWN prefix, or MAC prefix.

I restored mode 3 of the profile data backups prior to any config work, and when adding new virtual NICs to LPARs, the MAC addresses still mirror eachother.

I plan to test this with two p505 systems on an HMC to see if similar issues occur.

I don't have the resources to test this on blades, or on another SDMC.

We are pursuing with IBM HW support as of 2013-07-18

### END NOTICE ###

After a week, still no no response from support,
but I think I found out why this was a problem.

On physical hardware, "lssyscfg -r lpar" will show virtual_eth_mac_base_value=
On the flex nodes, this value is not exposed.

I can't tell if this is an SDMC/FSM limitation, or a flex node limitation.
I know that IVM sees it, but am not sure about HMC.

So, when LPAR profiles are copied over, they will bring the VEMBV,
and there is no way to change it short of deleting and re-creating.

All in all, it may just be easier to use mksyscfg from the start.
An example might be:

mksyscfg -r lpar -m Server-8205-E6D-SN10FFFFF -i profile_name=DefaultProfile,\

But there's already reference online for this sort of command.

Also, while working on a p740 via IVM, I ran into more differences from HMC/SDMC.
When you add a client LPAR with virtual SCSI, IVM automagically creates the VIO server virtual scsi server adapter. In addition, +1 from that slot it creates a virtual serial adapter for mkvterm.

If you're used to adding virtual scsi adapters in order, and you don't skip a slot on the mksyscfg lines, then you'll get this error:
[VIOSE01050173-0290] Cannot create virtual serial adapter in the management partition in the virtual slot number specified 20.

I couldn't find this error anywhere else on the internet, and it was a little confusing since I wasn't making a virtual serial adapter.
xaminmo: (Logo IBM CATE)
[personal profile] xaminmo
I always run into issues when I work in a multiple VLAN environment, because it's not *that* common for my builds. This is a reminder for me.

The magic is when using multiple VLANs:
1) Don't use the real VLAN ID for the trunk PVID unless you know for certain that was set on the switch. It is stripped off of all packets, and who knows what the PVID of the switch is, if any.
2) Any mismatch between PVID on the SEA and the trunk will cause packets to be dropped.
3) Don't use IEEE VLAN mode for the client adapter unless you're going to add VLAN interfaces from AIX. When not in VLAN mode, the PVID is ADDED to all packets on client adapters.
4) When using multiple trunks on one SEA, they all have to be the same trunk priority. ha_mode=sharing balances not using trunk priority, but based on the order of the virt_adapters field.
xaminmo: Josh 2016 (Default)
[personal profile] xaminmo
Redbooks, SDMC docs, blade docs, etc give info assuming everything works.

Everything does NOT work by default.

Major caveats:
1) The BladeCenter Blade IP config is for the service processor. Configure that. It's necessary. You cannot set an IP for it on both ports, so if you lose the blade switch, you lose LPAR management capabilities for the blade. Also, make sure your customer doesn't put the same IP on the AIX full system partition as was used for the service processor. You'd think this would be an easy one, but it has happened.
Ref: https://www-304.ibm.com/support/docview.wss?uid=nas1057800440ee4dae4862573e9005ad197&wv=1

2) The minimum firmware levels are for late may, early june of 2011. If you log in as admin/admin to the FSP and have NO OPTIONS, then you are not at the right level. Just use "update_flash" on the .img file you pull down. FixCentral does NOT provide firmware for blades. The only place I could find the list was from an AS/400 techdoc:
POWER7: http://www-912.ibm.com/s_dir/slkbase.nsf/ibmscdirect/5DDD545112AD21218625776500573780
POWER6 http://www-912.ibm.com/s_dir/slkbase.nsf/ibmscdirect/1C94AF8A9EAC7C138625764A006A426F

3) The p7 blade has to be initially discovered by service processor IP address. If you discovered it through the BladeCenter, then you have to remove the POWER Blade from director before you can make it work. You can't simply rediscover it.
Ref: https://www-304.ibm.com/support/docview.wss?uid=nas79b91491123ade0f88625786f0067dec5&wv=1

After all that, go to Navigate Resources, click on the server, pull down Actions, System Configuration, Edit, or Create Virtual. After that, it's vaguely similar to the HMC.
xaminmo: Josh 2016 (Default)
[personal profile] xaminmo
According to the EMC Connectivity Guide for AIX, when using PowerPath greater than, the devices will get a UDID, which is what we want for VIO.

So for whatever reason, this is not the case at one of my customer sites. Powerpath was installed at 5.1, but is now VIO was 1.5, is now 2.0 on some, and 2.1 on others.

So, before you say, "Oh, old vio.. blah blah blah" no, that's not it. Even on after removing VTDs, removing hdiskpower, removing hdisk, update powerpath, reconfig everything, there's still no UDID in ODM. Even for NEW devices, there is no UDID.

My workaround has been to manually create the appropriate UDID in ODM, but I was hoping someone, somewhere had run into this and found a solution.
xaminmo: Josh 2016 (Default)
[personal profile] xaminmo
Migrated from P6 to P7 via mksysb clone then updates.
cfgmgr now gives this:
Method error (/usr/lib/methods/cfgpager -l pager0 ):
        0514-040 Error initializing a device into the kernel.
Method error (/usr/lib/methods/loadkclic -l):
        0514-038 Error loading kernel extension.
loadkclic: load kernel extension failed.

Method error (/usr/lib/perf/cfg_pmsvcs load):
        0514-068 Cause not known.
setup_branchtable: Processor not yet supported.

The loadkclic is IZ78017 but I can't find anything about the other two.
The fun thing is that current VIO is AIX and you have to be at to fully exploit POWER7 performance.


I'm going to apply SP01, make a backup, and then make a frankenvio by updating with AIX and see what happens.


eserver: (Default)
IBM POWER servers

June 2017

45678 910


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 21st, 2017 10:37 am
Powered by Dreamwidth Studios