Alternate Method for Converting VMware vCenter Server from Physical to Virtual (VM)

At some point you are going to figure out that the advantages of having your vCenter Server as a VM outweigh the risks, and you're going to want to convert your existing physical vCenter into a virtual one.

There are a few ways of going about this, including just migrating vCenter to a fresh VM.

However, if you're stuck using only the VMware Converter standalone, you may find it frustratingly hard to make the conversion.

If you throw caution to the wind and just convert the vCenter Server on the fly while it's running (using the sync command to try an clean up after), you may get lucky, but more than likely you will have database corruption and other fun.

If you follow VMware's procedures for converting the machine and shut down all the services, the SQL databases, etc. then the VMware Converter can't connect to your vCenter to add the machine to the cluster.

One way around that would be converting to a seperate ESXi host that isn't in your cluster, and later join the host to the cluster, bringing vCenter with it.

However, that's a bit difficult when you are remote to a network (like I was) so there is no oppourtunity to provision physical hardware.

The answer? Double-Conversion.

Shut down all of the necessary services on your vCenter server, and run the Standalone VMware Converter.

Once there, simply convert your vCenter server to a VMware Workstation/Fusion .vmdk image.

Once that is complete, reboot vCenter so all the services are functional again, and convert that .vmdk into a VM on your cluster with the Standalone VMware Converter. vCenter treats this like any other vanilla VM, nothing special is needed.

Yes, you have a period of time where the old vCenter is running and controlling the network, and you're going to be 'going back in time' so to speak to the time of the conversion, but it's no worse than restoring vCenter from a backup.

I connected my new vCenter VM to a dummy network, and did all the final cleanup on it (installing vmxnet3, pvscsi, etc).

Make sure vCenter is running. If you have 5.1 with the Single Sign On (SSO) service, it will probably keep you from running vCenter.

It's not an error in the conversion process. When enough hardware changes, it breaks the SSO service, which still starts normally. However, vCenter won't start. The full article from VMware about this condition is here:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2036170

The set of commands I had to use was:

set JAVA_HOME=C:\Program Files\VMware\Infrastructure\jre
cd C:\Program Files\VMware\Infrastructure\SSOServer\Utils
rsautil manage-secrets -a recover -m MASTER_PASSWORD

(Where MASTER_PASSWORD is your master password when you installed vCenter/SSO)


When you have everything on your vCenter VM running smoothly, shut down your physical vCenter server, and connect your vSphere Client to the raw ESXi host holding your vCenter VM. Once you start it, you'll be able to connect to vCenter as per normal.

Make sure you've read the various forums on how to protect yourself from a bad situation with a virtual vCenter server.

Oh, and if you can keep the old physical server around for a bit, you have a quick emergency backup in case you run into a problem cold-starting your datacenter. 

FreeBSD 10 ZFS Boot Hang with USB Boot Drive and Non-USB Keyboard

Here's an interesting one.

If you create a FreeBSD 10 ZFS system that boots from a USB drive, you may hang if you don't have a USB keyboard.

I've noticed this on a few different hardware systems, and it may even be present in FreeBSD 9.x - I haven't had much chance to look into it.

The hangs are always the same ; The zfs boot loader can't find a bootable disk.

When I manually type in zfs:rpool (in the case of the manual ZFS on root setup here) or zfs:zroot/ROOT/default (in the case of the new FreeBSD10 ZFS installer) I'll get one of two things;

1) kernel panic and crash. Note that this is often from pressing 'return' at the boot loader prompt, but typing 'zfs:rpool' won't cause an issue.

2) a small move forward to set the host UUID and hostid, then nothing - just hung.

It looks like when the USB keyboard is not present, then da0 (my USB Flash Stick) doesn't exist.

At some point I'll look into this further, as you shouldn't need any keyboard (USB or other) connected to boot FreeBSD, but for now just make sure you have a USB keyboard hooked up for boot.

Additionally, the order of the USB devices seems to be important. If I have a USB Keyboard on my USB0 Root, and the boot device on my USB1 Root, then I have this symptom as well. If I boot with the boot device on USB0, and keyboard on USB1, then I'm okay. 

Using MessagePack as Transport Beween a Python Tornado Server, and a JavaScript Browser Client

First of all: Good Luck. :-)

MessagePack is a great binary serialization format, allowing you to transport globs of variables across whatever transport you like. It's very similar to JSON, but leaning towards a tighter packet size , as shown in the graphic I took from the msgpack.org site above.

MessagePack also has a great variety of languages supported. If you code it, you can probably find a msgpack.X file on their site.

Now the problems:

1) MessagePack has evolved over the years, and updated the specifications for the protocol. This isn't well documented, nor is there a "Version 2.1" or similar tag on the specification documents. If you look around, you can see that some links refer to this as v5, but how do I know that we're not at v8 and that linking site is stale? This makes it less obvious about what version a msgpack library supports, which means it's more difficult to choose compatible msgpack libraries when you're using different languages. 

2) No CHANGELOG, because there are no revisions/versions. Good luck seeing what msgpack was 3 years ago without detective work.

3) Because the specification has changed, some of the MessagePack libraries are up-to-date, others are years behind. The pure Python u-msgpack.py is up-to-date as of Dec 2013, but the JavaScript msgpack.js (v1.05) is not.

Through trial and error, I was able to get the JavaScript version working with the u-msgpack.py version, but not the standard msgpack.py (which is a wrapper for the C library).

However, since u-msgpack.py is following the new specification, it can create msgpacks that the JavaScript version can't decode, particularly when Python is sending a long string.

I've patched msgpack.js, and have contacted the author (uupaa) to see if he has an updated version. If that dosen't pan out, I'll fork and update the code so others can have access to it.  I'm also interested in patching it to work with JavaScript Blobs, so I can send binary data to it.

There is one small error in the u-msgpack.py file:

The problem is in _pack_string. It's calculating the length of the string before it's encoded to UTF-8.

I think you must encode the string before you find the length of it, as some characters need to encode as double-byte or longer.

An example would be the French name Allagbe, or the French word precedent , where the 'e' is with Acute (http://www.fileformat.info/info/unicode/char/e9/index.htm)

  

Python's encoder makes this b'Allagb\xc3\xa9', which is one byte longer than than the original string.

u-msgpack encodes this as b'\xa7Allagb\xc3\xa9'  - notice how it's only 7 bytes long - it's trimming the \xa9 char from the msgpack.

When you feed this trimmed string through a .decode('utf-8') method, you'll crash with a python error : 'utf-8' codec can't decode byte 0xc3 in position 6: unexpected end of data

The solution is to encode to UTF-8 before calculating the string length, as detailed below:

def _pack_string(x):

x = x.encode('utf-8')

    if len(x) <= 31:

        return struct.pack("B", 0xa0 | len(x)) + x

    elif len(x) <= 2**8-1:

        return b"\xd9" + struct.pack("B", len(x)) + x

    elif len(x) <= 2**16-1:

        return b"\xda" + struct.pack(">H", len(x)) + x

    elif len(x) <= 2**32-1:

        return b"\xdb" + struct.pack(">I", len(x)) + x

    else:

        raise UnsupportedTypeException("huge string")

With this patch in place, I'm able to pass plenty of French and Italian names through u-msgpack.py to msgpack.js

Now.. if you're trying to make all of this work across Tornado's WebSockets, keep in mind that what you get from Tornado will probably not fit into u-msgpack.py's .unpackb() function properly, but I'm out of time to detail that fun today.

In the end: I'm happy with MessagePack, but I wish it was a little easier to get into.