UCS Networking Adventure: A tale of CoS and The Vanishing Frames

The Problem

This week I had to connect an additional NetApp Storage System to my existing UCS environment through a different path than a similar shared storage platform that we utilize here. This shouldn’t be a big deal but there were a few caveats:

  • The Storage System was attached to a dedicated Nexus 5k for this customer
  • The VLAN configured on the customer switch collided with one configured in UCS so VLAN translation was necessary through access ports.
  • The traffic takes a different switching path from normal NFS traffic in this environment

I configured everything as one normally does when connecting to IP storage, jumbo frames and all. There was only one problem:

Standard Frames

~ # vmkping -s 1400 -d
PING ( 1400 data bytes
2508 bytes from icmp_seq=0 ttl=255 time=0.232 ms
2508 bytes from icmp_seq=1 ttl=255 time=0.198 ms
2508 bytes from icmp_seq=2 ttl=255 time=0.265 ms

--- ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.198/0.232/0.265 ms
~ #

Jumbo Frames

~ # vmkping -s 2500 -d
PING ( 2500 data bytes

--- ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
~ #


Now, normally this is just a simple issue of the MTU being set incorrectly somewhere along the traffic path but as I dug deeper into this issue that turned out to not be the case, it was something much stranger and more interesting. The IPs, VLANs and hostnames have been changed or obscured to protect the innocent. Onward!

Quick Tip: Bonding, LACP, and VLANs in Linux

I have been doing a lot of tinkering with linux based storage (more to come on that!) over the past few weeks and I had to hunt and peck around the internet to find all of the information on using bonding/lacp and vlans in linux so I want to bring it all to one place. All of these configuration files are from Ubnutu but the format should be similar in other distros. All of the switch configurations were on a Cisco 2960 running IOS 12.2-lanbase which is a fairly old and basic switch.

Quick Tip: Register all VMs on a datastore

Today I had an entire datastore of VMs to register, probably about 30 in total, and I didn’t want to go through the GUI and register each VM manually.

I came up with this quick unix one-liner:

# NOTE: My datastore path is /vmfs/volumes/5317a80e-add165f6-ada9-001517599f73
# replace this with whatever datastore needs searching

find /vmfs/volumes/5317a80e-add165f6-ada9-001517599f73 -name "*.vmx" -exec  vim-cmd solo/registervm {} \;

find /vmfs/volumes/5317a80e-add165f6-ada9-001517599f73 -name "*.vtmx" -exec  vim-cmd solo/registervm {} \;

Tutorial: 802.3ad Port Configuration with ESX

Just wanted to post a quick post about how to configure 802.3ad port consolidation with a Cisco switch and VMware ESX (vSphere was used for this example). I was using an HP DL380 with 2 onboard NIC and 2 24 port Cisco 3750G connected with stackwise cables.

Switch Configuration:

interface GigabitEthernet1/0/1
description ESX NIC 2
switchport trunk encapsulation dot1q
switchport mode trunk channel-group 1 mode on
interface GigabitEthernet2/0/14
description ESX NIC 2
switchport trunk encapsulation dot1q
switchport mode trunk channel-group 1 mode on
interface Port-channel1
description ESX PortChannel
switchport trunk encapsulation dot1q
switchport mode trunk


BeyondVM is a personal blog is about virtualization, system administration archetecture and the business of IT. I post research that I do into better management of virtualization and infrastructure, as well as things that I learn along the way.

The views expressed anywhere on this site are strictly mine and not the position of any employer, vendor or provider including but not limited to my employer, VMware or any of its companies. Any solutions that I offer are 'use at your own risk.'
Twitter Follow Me!
Recient Comments