A Day in the Life of...

I've been having to run VMware Workstation 6.5.4 on Linux kernel 2.6.30 because the modules will not compile on any higher kernel. VMware was offering a 30% discount on upgrades, so I thought I'd upgrade and see if the module compile problems had been fixed. Well, they hadn't. The errors occur in iommu:

CC [M] /tmp/vmware-root/modules/vmmon-only/linux/iommu.o
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function ‘IOMMU_SetupMMU’:
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:156: error: implicit declaration of function ‘iommu_map_range’
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function ‘IOMMUUnregisterDeviceInt’:
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:216: warning: ignoring return value of ‘device_attach’, declared with attribute warn_unused_result
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function ‘IOMMU_VMCleanup’:
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:403: error: implicit declaration of function ‘iommu_unmap_range’
make[2]: *** [/tmp/vmware-root/modules/vmmon-only/linux/iommu.o] Error 1
make[1]: *** [_module_/tmp/vmware-root/modules/vmmon-only] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.35-4.slh.11-aptosid-amd64'
make: *** [vmmon.ko] Error 2

I could have saved myself $70 since I got module compile issues, i.e. fatal make errors, for free with either 6.5.4 or 7.x. Fortunately, there is an easy fix documented at www.rrfx.net.

If you don't wish to visit the link, the fix by Robert Reid is documented below:

cd /tmp
tar xvf /usr/lib/vmware/modules/source/vmmon.tar -C /tmp
perl -pi -e 's,_range,,' vmmon-only/linux/iommu.c
tar cvf /usr/lib/vmware/modules/source/vmmon.tar vmmon-only

To get VMware Workstation 6.5.4 to work on kernel 2.6.32 or above on Ubuntu 10.04 LTS, you'll need to do two things, get the installer to bypass a hang condition and patch vnetUserListener.c and pgtbl.h by adding two include statements, #include "compat_sched.h" and #include "<"linux/sched.h">", to each file:

1. chmod u+x VMware-Workstation-6.5.*.bundle
./VMware-Workstation-6.5.*.bundle --ignore-errors

in a separate terminal run:

while true; do sudo killall -9 vmware-modconfig-console; sleep 1; done

Original documentation is here.

2. Patch vmci and vmnet documented here and here:

tar xvf /usr/lib/vmware/modules/source/vmnet.tar -C /tmp
tar xvf /usr/lib/vmware/modules/source/vmci.tar -C /tmp

cd /tmp

perl -pi -e 's,("vnetInt.h"),\1\n#include "compat_sched.h",' vmnet-only/vnetUserListener.c
perl -pi -e 's,("compat_sched.h"),\1\n#include "<"linux/sched.h">"",' vmnet-only/vnetUserListener.c
perl -pi -e 's,("compat_page.h"),\1\n#include "compat_sched.h",' vmci-only/include/pgtbl.h
perl -pi -e 's,("compat_sched.h"),\1\n#include "<"linux/sched.h">"",' vmci-only/include/pgtbl.h

tar cvf /usr/lib/vmware/modules/source/vmnet.tar /tmp/vmnet-only
tar cvf /usr/lib/vmware/modules/source/vmci.tar /tmp/vmci-only

note: "<" and ">" should just be < and >, but blogspot.com's servers make things like header files disappear in source code.

Now run vmware-modconfig --console --install-all to finish the VMware Workstation installation.

Labels: vmmon compilation failure VMware kernel 2.6.35