Vswitch problem on z/VM 5.2.0?

[Brian Nielsen]

This weekend I upgraded from z/VM 4.4.0 to 5.2.0 and ran into an
unexpected problem with connectivity through vswitches to the external
network. I'm trying to figure out if the problem is in my vswitch
definitions or if it's a reportable issue.

At the moment it appears that private IP address ranges (eg
192.168.xxx.xxx, 10.xxx.xxx.xxx, 172.16.64.xxx) which worked fine through
a vswitch in 4.4.0 do not work at all through a vswitch on 5.2.0.

Private IP addresses in the same subnet, on the same VLAN, on the same
vswitch couldn't ping each other or their external gateway. The public IP
addresses on the same vswitch had no trouble pinging their external
gateway. Private IP addresses with direct OSA conections had no problems
either.

I was able to confirm it was a vswitch related problem through testing
with my z/VM TCPIP stack. Normally it has a dedicated OSA connection and
a private IP address of 172.16.64.3 on VLAN 7. On zVM 5.2 I was able to
PING its external gateway at 172.16.64.1. When I changed the TCPIP stacks
network connection from a dedicated OSA to a virtual NIC on the vswitch
the ping failed.

My vswitch was defined in z/VM 4.4.0 with:

DEFINE VSWITCH SWITCH02 RDEV 0500 F804 PORTNAME PT0500 PTF804

and in 5.2.0 with:

DEFINE VSWITCH SWITCH02 RDEV 0500 F804 VLAN 7 PORTNAME PT0500 PTF804

For testing the TCPIP NIC on the vswitch it was authorized with:

SET VSWITCH SWITCH02 GRANT TCPIP VLAN 7

In both cases, PROFILE TCPIP includes:

LINK ETH0 QDIOETHERNET ***@0503 VLAN 7


Changing TCPIP's GRANT to include PORTTYPE TRUNK, and verifying the change
with Q VSWITCH ALL DETAILS, had no effect.

On the exact same switch, SWITCH02, other Linux guests on VLAN 2 with
public IP addresses (eg 164.165.57.xxx) had no trouble communicating with
the network outside the z/890. They are authorized to the vswitch with:

SET VSWITCH SWITCH02 GRANT <userid> VLAN 2

The problem also manifested itself on the other vswitch carrying traffic
on private IP addresses for other VLANs (3 and 4).


Is there something I've overlooked in the changes to vswitches from 4.4.0
to 5.2.0? Or is this a reportable problem?

Brian Nielsen

P.S. The problem has been temporariliy circumvented by connecting the
userids with priviate IP address on VLAN 7 to a guest LAN and using the
zVM TCPIP stack to route their traffic to the OSA. (Yuck, but it works.)

[David Kreuter]

had a similar problem on 5.2. See if ptf UM31613 APAR VM63895 are on your system. It corrected our problem.
David

-----Original Message-----
From: VM/ESA and z/VM Discussions on behalf of Brian Nielsen
Sent: Mon 3/6/2006 3:17 PM
To: VMESA-***@LISTSERV.UARK.EDU
Subject: Vswitch problem on z/VM 5.2.0?

This weekend I upgraded from z/VM 4.4.0 to 5.2.0 and ran into an
unexpected problem with connectivity through vswitches to the external =

network. I'm trying to figure out if the problem is in my vswitch
definitions or if it's a reportable issue.

At the moment it appears that private IP address ranges (eg
192.168.xxx.xxx, 10.xxx.xxx.xxx, 172.16.64.xxx) which worked fine through=

a vswitch in 4.4.0 do not work at all through a vswitch on 5.2.0.

Private IP addresses in the same subnet, on the same VLAN, on the same =

vswitch couldn't ping each other or their external gateway. The public I=
P
addresses on the same vswitch had no trouble pinging their external
gateway. Private IP addresses with direct OSA conections had no problems=

either.

I was able to confirm it was a vswitch related problem through testing =

with my z/VM TCPIP stack. Normally it has a dedicated OSA connection and=

a private IP address of 172.16.64.3 on VLAN 7. On zVM 5.2 I was able to =

PING its external gateway at 172.16.64.1. When I changed the TCPIP stack=
s
network connection from a dedicated OSA to a virtual NIC on the vswitch =

the ping failed.

My vswitch was defined in z/VM 4.4.0 with:

DEFINE VSWITCH SWITCH02 RDEV 0500 F804 PORTNAME PT0500 PTF804

and in 5.2.0 with:

DEFINE VSWITCH SWITCH02 RDEV 0500 F804 VLAN 7 PORTNAME PT0500 PTF804=


For testing the TCPIP NIC on the vswitch it was authorized with:

SET VSWITCH SWITCH02 GRANT TCPIP VLAN 7

In both cases, PROFILE TCPIP includes:

LINK ETH0 QDIOETHERNET ***@0503 VLAN 7


Changing TCPIP's GRANT to include PORTTYPE TRUNK, and verifying the chang=
e
with Q VSWITCH ALL DETAILS, had no effect.

On the exact same switch, SWITCH02, other Linux guests on VLAN 2 with
public IP addresses (eg 164.165.57.xxx) had no trouble communicating with=

the network outside the z/890. They are authorized to the vswitch with:

SET VSWITCH SWITCH02 GRANT <userid> VLAN 2

The problem also manifested itself on the other vswitch carrying traffic =

on private IP addresses for other VLANs (3 and 4).


Is there something I've overlooked in the changes to vswitches from 4.4.0=

to 5.2.0? Or is this a reportable problem?

Brian Nielsen

P.S. The problem has been temporariliy circumvented by connecting the
userids with priviate IP address on VLAN 7 to a guest LAN and using the =

zVM TCPIP stack to route their traffic to the OSA. (Yuck, but it works.)=

[Brian Nielsen]

I am already running with that APAR.

Brian Nielsen


On Mon, 6 Mar 2006 15:20:38 -0500, David Kreuter <***@vm-
resources.com> wrote:

>had a similar problem on 5.2. See if ptf UM31613 APAR VM63895 are on your
system. It corrected our problem.
>David

[Brian Nielsen]

It is my understanding from the manuals that the VLAN ID on the DEFINE
VSWITCH only sets up the default VLAN ID to be used by guests that do not
include a VLAN ID on the SET VSWITCH GRANT command. All of the guests
GRANTed access to the vswitch include the VLAN parameter for their
specific VLAN ID.

The external gateway (a CISCO 6509) is expecting (and restricting)
communications with the z/890 on VLANs 2, 7, 3, and 4.

Brian Nielsen


On Mon, 6 Mar 2006 15:28:40 -0500, Rick Barlow
<***@Nationwide.com> wrote:

>I think the problem is your VLAN ID on the DEFINE VSWITCH. That value
>needs to match the management VLAN ID for the switch to which your OSA is
>connected. The normal default is 1. Your installation may have changed
>the value. You will need to talk to the group that configures your switch
>hardware.
>
>____________________________________
>Rick Barlow
>Systems Engineering Consultant
>Nationwide Services Co., Enterprise Business Intelligence Services
>Mainframe, z/VM and zSeries Linux Support
>One Nationwide Plaza 3-25-02
>Columbus OH 43215-2220 U.S.A
>Voice: [phone] Fax: [phone]
>mailto:***@nationwide.com

[Brian Nielsen]

On Mon, 6 Mar 2006 18:40:11 -0500, Alan Altmark <***@us.ibm.com>
wrote:

>From the book, "The defvid defines the default VLAN ID to be associated
>with untagged frames received and transmitted by the Virtual Switch." This
>is also known as the "native VLAN id" in the switch. They must match.

I will check to see what the "native VLAN id" in the real switch is and
make my vswitch default VLAN ID match it. I didn't see anywhere in the
zVM manuals where this was stated as a requirement. Did I miss it or does
the doc need updating?

How has the vswitch behavior changed from zVM 4.4.0 as a result of the
introduction of the defvid? I find it confusing that the public IP
addresses on VLAN 2 work fine with my current definitions.

>Do not use DEFINE VSWITCH VLAN xx to avoid SET VSWITCH GRANT VLANs.

I had no intent to. As I posted earlier, every SET VSWITCH GRANT includes
the VLAN parameter.

>And unless you want VM TCP/IP to route between VLANs on the same OSA
>trunk, I recommend not to use a VLAN-aware configuration in PROFILE TCPIP.

The Network folk insist on using VLANs to the z/890. Since I give TCPIP
dedicated OSA addresses I don't think I have a choice but to make TCPIP
VLAN-aware at least for that link. Am I wrong? Are you recommending that
I connect TCPIP to a vswitch with a GRANT for the VLAN ID rather than
having TCPIP VLAN-aware directly to the OSA?

Brian Nielsen

[Brian Nielsen]

On Mon, 6 Mar 2006 18:40:11 -0500, Alan Altmark <***@us.ibm.com>
wrote:

>From the book, "The defvid defines the default VLAN ID to be associated
>with untagged frames received and transmitted by the Virtual Switch." This
>is also known as the "native VLAN id" in the switch. They must match.

I checked with the Network group and according to them they do not have
any native VLAN id set up - they do not want any untagged traffic in the
network. What would I use in my DEFINE SWITCH for the defvid in this
case? Is an unused VLAN number acceptable?

Brian Nielsen

[Brian Nielsen]

If I do that I can't do GRANTs for specific VLAN ids required by the
various Linux guests.

I did a test, and using a defvid of "1" on a vswitch definition allowed a
VLAN 7 private IP address to ping the external gateway. Whereas when my
test with a defvid of "7" on the vswitch definition the private IP address
could not ping the external gateway.

So the problem is the vswitch interfering with the packets when it thinks
VLAN 7 is for untagged traffic. I'll be taking Alan's advice and changing
my vswitch definitions to use a defvid of "1". Presumably any unused VLAN
would work in my case since there is no untagged traffic on the network.

Brian Nielsen


On Tue, 7 Mar 2006 14:44:44 -0500, Rick Barlow
<***@Nationwide.com> wrote:

>You should be able to just leave the VLAN parameter off and let it default
>to VLAN UNAWARE.
>
>____________________________________
>Rick Barlow
>Systems Engineering Consultant
>Nationwide Services Co., Enterprise Business Intelligence Services
>Mainframe, z/VM and zSeries Linux Support
>One Nationwide Plaza 3-25-02
>Columbus OH 43215-2220 U.S.A
>Voice: [phone] Fax: [phone]
>mailto:***@nationwide.com
>
>
>VM/ESA and z/VM Discussions <VMESA-***@LISTSERV.UARK.EDU> wrote on
03/07/2006
>02:40:29 PM:
>
>> VMESA-***@LISTSERV.UARK.EDU
>>
>> On Mon, 6 Mar 2006 18:40:11 -0500, Alan Altmark
<***@us.ibm.com>
>
>> wrote:
>>
>> >From the book, "The defvid defines the default VLAN ID to be associated
>> >with untagged frames received and transmitted by the Virtual Switch."
>This
>> >is also known as the "native VLAN id" in the switch. They must match.
>>
>> I checked with the Network group and according to them they do not have
>> any native VLAN id set up - they do not want any untagged traffic in the
>> network. What would I use in my DEFINE SWITCH for the defvid in this
>> case? Is an unused VLAN number acceptable?
>>
>> Brian Nielsen
>=========================================================================

[Brian Nielsen]

On Tue, 7 Mar 2006 16:26:00 -0500, Alan Altmark <***@us.ibm.com>
wrote:

>On Tuesday, 03/07/2006 at 02:58 CST, Brian Nielsen
><***@SCO.STATE.ID.US> wrote:
>> So the problem is the vswitch interfering with the packets when it
>thinks
>> VLAN 7 is for untagged traffic. I'll be taking Alan's advice and
>changing
>> my vswitch definitions to use a defvid of "1". Presumably any unused
>VLAN
>> would work in my case since there is no untagged traffic on the network.
>
>I recommend that the default VLAN id on the VSWITCH to match the native
>VLAN id on the physical switch since trunked switches are supposed be
>configured to use the same native VLAN. On Cisco switches, VLAN 1 is the
>default native VLAN id.
>
>Alan Altmark
>z/VM Development
>IBM Endicott

Understood, and in my case the Network group says there is no native VLAN
defined. I have nothing to match against.

Brian Nielsen

[Brian Nielsen]

On Tue, 7 Mar 2006 18:00:16 -0500, Alan Altmark <***@us.ibm.com>
wrote:

>In general, I recommend not sharing
>an OSA being used by a VSWITCH since our eventual adoption of protocols
>like GVRP will cause a mismatch if others are using the OSA with different
>VLAN IDs. In your case, I recommend putting TCP/IP on a different OSA
>chpid (on an access port).

Thanks. Since separate OSA's are scarce, would the next best thing be to
put TCPIP on the VSWITCH? Are there any serious negatives to doing so?

Brian Nielsen