how to rebuild Plone when you upgrade to a 64-bit kernel

how to rebuild Plone when you upgrade to a 64-bit kernel

ah 32 bits I hardly knew you

Why in (Early) 2016 I Was Still Running a 32-bit Kernel

I've been running Plone on Linode VMs (linode.com) for years now and have been very happy with them and the service.

At UW Oshkosh we run RedHat EL (the stable enterprise version) and we used to run SuSE EL before that. Above all, we prize reliability and stability over having the latest and greatest system packages that the young punks (er, kids) are using (I'm talking about *you*, Ubuntu!).

So when I started running Plone on my Linode VMs I started to use SuSE as well, and then when it became clear that SuSE was slowly dying, I switched to CentOS 5.5 (the RedHat EL clone) for the stability and longevity I knew it would have.

Why I Switched to a 64-bit Kernel

Because I have been running Plone sites on this particular Linode for years now, I hadn't had the opportunity to set up a new Linode VM with a shiny new kernel (or new distribution - still looking at you, Ubuntu).

The wonderful folks at Linode.com had been offering a *free upgrade* to VMs using their SSDs (solid state disks), which are so much faster than the old spindle drives. However, to take advantage of that free SSD upgrade, I first would have to switch to a 64-bit kernel.

I resisted this for months, not wanting to mess with Something That Works.

Eventually, I followed the documented procedure for switching to a 64-bit kernel and it went perfectly smoothly. Once that was done, I was able to take advantage of the SSD upgrade, and after a reboot my Linode VM and all its Plone sites were up and running flawlessly.

Why I Regretted Switching to a 64-bit Kernel

Today I made the mistake of making a small change to my Plone buildout, to upgrade one add-on package. (I won't say which package, in order to protect the innocent).

To my surprise, buildout pulled down a number of eggs that I wasn't expecting. After all, this was a small version upgrade to just one add-on, so why did it seem like the bad old days when Plone people didn't pin versions and spent hours tearing out their thinning hair in despair (hey, that rhymes!)?

I let buildout run, despite a growing sense of dread. When I restarted my clients, everything seemed to run just fine, except for errors in the log like

XSLTApplyError: xsltValueOf: text copy failed

...and as it turned out, none of the Diazo themes were being rendered - those sites looked pretty bare. I had to fix this (or restore from backup) in a hurry.

Googling around revealed that the problem was related to incorrect versions of lxml2, yet this particular Plone installation had been running for years without a hitch.

After looking over my *.cfg files carefully, I was sure I hadn't changed them except for this one version bump. It was when I happened to look through my buildout-cache/eggs folder (I used the Plone unified installer) and noticed that I had two eggs for lxml2: the usual one, and the x64 version.

The problem was that the new 64-bit kernel was causing buildout to build and install 64-bit versions of Python packages....into a Plone that had been built with 32-bit packages and binaries. Oops!

The 64-bit Light at the End of the Tunnel

I decided that the easiest way to fix my problem was to re-run the Plone unified installer in a new target directory (in this case /opt/Plone-4.3.2-x64) so it would build and include a fully 64-bit Python.

  • I copied the old install's *.cfg files from /opt/Plone-4.3.2
  • modified in buildout.cfg the zeoserver and client1 and client2 ports so they wouldn't clash with the still running processes
  • reran buildout
  • tested that the new zeoserver and clients 1 and 2 started up ok, then shut them down
  • copied over the old var/filestorage and var/blobstorage directories
  • restored in buildout.cfg the zeoserver and client1 and client2 ports to match what was in the old install
  • reran buildout
  • killed the old zeoserver, client1, and client2 processes
  • started the new zeoserver, client1, and client2 processes
  • renamed /opt/Plone-4.3.2 to /opt/Plone-4.3.2-x32-dead
  • symlinked /opt/Plone-4.3.2-x64 to /opt/Plone-4.3.2 to sneakily make everything look like nothing had changed