After four months of work, we are finally ready to migrate from our old server infrastructure to our shiny new cloud infrastructure. This will happen over the weekend of May 12th-13th, and as a result all ned Productions Limited websites will go read-only from Friday 11th May until the end of the weekend.

In other news, I have accepted a job offer from Research In Motion in their Native SDK division, and as a result from now on I will be closed to new business. As a result I have disabled the products and prices page.

To those who have provided me custom over the years, my thanks!

Due to a major security breach in Plones before v4.0.8, tomorrow we'll be upgrading to v4.0.9. We've already run a dry run on a local copy and it went very smoothly, so ned Productions sites shouldn't be down for more than an hour. Of course, as always, read-only content will continue to be served from the caches during the downtime.

After two days of prep and an awful lot of testing on a local copy to try and remove as many of the bugs as possible, ned Productions Limited hosted websites are finally migrated to Plone v4.0.5!

This is a big upgrade - there are a whole load of new features across all our hosted sites, especially in commenting and database features never mind a very significant performance increase and reduction in server load. Given the size and complexity of the upgrade, I had been putting it off for more than six months now, but as we're about to start a new large R&D project we needed a fully capable and up to date CMS to back its public facing website so the upgrade definitely needing doing. Now it's done, I am glad as I can move ahead with the R&D.

Unfortunately one of our sites didn't make it - neocapitalism.org which was felled by the lack of a working ZWiki. The old ZWiki, v0.61, won't work with Zope 2.12 while the new ZWiki, v2.0, won't work with Plone 4. However, the site's content has been saved off and when ZWiki gets itself sorted out we will complete the port to Plone 4. Other that that, all our sites migrated fairly well though some took a lot of work indeed :(

Total downtime was about twelve hours, though varnish continued to serve up read-only versions of our sites during that time. Thanks for the patience!

Finally got the time to migrate the servers over to Plone 4 which is a fairly hefty undertaking. Servers will be read only 16th to the 17th April 2011.

If you're seeing this, then you are looking at the new server! It will take 24h for the DNS changes to propagate to these new servers, then the old server infrastructure will be shut down and securely wiped.

My apologies for the ten minutes of downtime earlier today - it turns out that the documentation for making a ZEO based Plone site read-only is severely out of date, so I have submitted updated docs to the Plone FAQ.

Enjoy the considerably faster new server cluster - it's about 200% as fast for cold page loads, 600% faster for warm page loads (see page load times archive here) and can handle several dozen times more maximum simultaneous users! We would also like to announce that due to reliability problems we have disabled the Los Angeles cluster which will shortly be decommissioned - all global traffic now goes to Amsterdam. In case you are wondering about the effects on ned Productions Ltd hosted websites:

The first graph shows the current and former european latency profiles whereas the second graph shows the former geo-targeted profile (note that the two graphs can't be directly compared as their test locations and times are very different, however the blue line is for the same top quality London Docklands based server in both despite the up to 100ms difference ). As you can see, the new Amsterdam cluster isn't as well connected outside Europe as it could be with its 50-100ms extra latency, but sometime in early 2011 we hope to open a data content caching node in Phoenix, Arizona for much improved global data read latency.

This is just a note to let visitors and customers know that the primary ned Productions Limited servers are being migrated tomorrow (1st July) onto much more powerful hardware with considerably improved international network connectivity. Read access ought to be entirely unaffected so there will be no site downtime at all, however write access will be completely disabled across all ned Productions Limited hosted websites for much of tomorrow until the migration is completed.

We hope that this does not inconvenience you too much. Remember to run portal_migration and portal_quickinstaller upgrades on July 2nd or thereafter!

nedproductions.biz has been on IPv6 since its beginning, however its geo-targeting domain name resolution server ns1.nedproductions.biz (i.e. the server which converts names such as www.nedproductions.biz into a unique IP address) was until today queryable in IPv4 only because it runs in a tiny commodity OpenVZ VPS. And unfortunately the host for that VPS doesn't have any IPv6 configured which as it's on OpenVZ means that one cannot configure the usual trick of using an IPv6-over-IPv4 tunnel.

Well I managed to find a workaround, so I'm going to document it here. Firstly as always you need an IPv6 tunnel from www.tunnelbroker.net and then you'll need to do the following as root:

cd /root
apt-get install iproute gcc
wget http://tb-tun.googlecode.com/files/tb-tun_r14.tar.gz
tar zxf  tb-tun_r14.tar.gz
gcc tb_userspace.c -l pthread -o tb_userspace

tb_userspace is a handy little userspace TUN/TAP forwarder program whose author wrote specifically to work around the problem of getting IPv6 into a IPv4 only OpenVZ VPS.

Now to make it go (and assuming you're on Ubuntu/Debian), nano /etc/network/interfaces and append:

iface he-ipv6 inet6 v4tunnel
        endpoint <IPv4 tunnel endpoint>
        address <Your IPv6 address>
        netmask 64
        up      ip route add default dev he-ipv6
        down    ip route del default dev he-ipv6
        mtu     1480

auto tb
iface tb inet6 manual
        pre-up  setsid /root/tb_userspace tb <IPv4 tunnel endpoint> > /dev/null &
        up      ifconfig tb up
        post-up ifconfig tb inet6 add <Your IPv6 address>
        post-up ifconfig tb mtu 1480
        post-up route -A inet6 add ::/0 dev tb
        down    ifconfig tb down
        post-down       route -A inet6 del ::/0 dev tb
        post-down       killall tb_userspace

I have left in the standard he-ipv6 config which tunnelbroker suggests so you can see how the tunnel IPv4 and IPv6 addresses map themselves onto the tb_userspace forwarder. You can safely leave in the he-ipv6 config for the time when your VPS provider does implement IPv6 whereupon you can add an auto he-ipv6 and disable the auto tb.

While this means that everything under your control can now be made IPv6, the last remaining step to making everything IPv6 from start to finish is the addition of IPv6 glue records by your domain registrar. Obviously this is out of your control unless your registrar happens to support IPv6 glue records.

Anyway I hope that was useful. I know it will be for me in the future!

We would like to announce that the upgrade of Plone to v3.3.4 will occur between tomorrow (Friday 19th Feb) and the end of this weekend (Sun 21st Feb). After this upgrade, for each of your websites you will need to do the following:

1. Enter portal_migration in your ZMI and upgrade.
2. Enter portal_quickinstaller in your ZMI and reinstall any products which have been upgraded. We strongly recommend that you do one at a time if you get any errors by trying to do all at once.
3. For each product upgraded, reenter its settings in Site Setup. For example TinyMCE will return to default settings.

During the upgrade your website will still be present, but any attempts to login may return a 502 Service Unavailable error. In the highly unlikely situation that your site dies after the upgrade, please contact us.

Details behind the upgrade:

We have been waiting for v2.0.5 of the varnish web cache to be released before we performed the upgrade of our Plone installation to v3.3.4. This is because that release of varnish finally adds support for the HTTP If-None-Match header which is used by web browsers on Plone sites to tell the web server which version of a web page they already have a copy of by sending its last known ETag. Unfortunately, varnish used to always bypass its cache when it saw such a header which effectively meant that all visitors to any ned Productions Ltd. website who had been here before bypassed the cache (not that this caused much load nor delay, as Zope simply returned a 304 Not Modified).

What that meant in practice was that if we took down Plone for any reason, every visitor who had been before instantly got a 502 Service Unavailable error. Other than stripping If-None-Match support and therefore greatly increasing page load times while visitors navigated our sites, there wasn't much we could do.

With this new support however, we can now take down Plone for up to 24h and varnish will continue to serve out its cached content and so long as the cache has a copy of the page, no one should notice a thing. Obviously enough because logged in users always bypass the cache they will not be able to log in at all during the upgrade. Right now the cache is being populated, so we should be ready to start work tomorrow.

Obviously enough this new feature considerably improves load scalability and load times of changed pages on all ned Productions Ltd. websites. Whereas before Zope always had to be asked to verify an ETag and if it didn't match then Zope  had to deliver a copy of the page, now varnish understands ETags and will handle everything. This should reduce page load times for updated pages by a few hundred milliseconds.

Other new features resulting from this upgrade are mostly bug fixes. Firefox users will see a very substantial improvement in page load times as a bug invoking Java with every page has been fixed. There has been a very rare XSS vulnerability fixed in Zope (not an issue in Plone), and there have been many more bug fixes to TinyMCE and a few other components.

We finally got around to implementing the geo-directing part of the ned Productions Ltd web services platform, so finally all our hosted websites are now being served to their closest geographical server. This has very substantially improved page load times and browsing speeds for our customers:

Our server locations are as follows:

View Larger Map

We have also added a new product: you can now have your own sites be geo-directed for a low low price of just €12/year!

We strongly believe in being transparent about how we implement our products and services so you can see for yourself their quality - or indeed, how to implement them yourselves if you so chose. We have hence opened a new Wiki section which contains detailed implementation notes as a public service.

While most compilers implement the C99 feature typeof(), unfortunately the Microsoft C++ compiler does not as yet. Here's an implementation which can be just dropped in:

#if defined(_MSC_VER) && _MSC_VER>=1400
namespace msvc_typeof_impl {
	/* This is a fusion of Igor Chesnokov's method (http://rsdn.ru/forum/src/1094305.aspx)
	and Steven Watanabe's method (http://lists.boost.org/Archives/boost/2006/12/115006.php)

	How it works:
	C++ allows template type inference for templated function parameters but nothing else.
	What we do is to pass the expression sent to typeof() into the templated function vartypeID()
	as its parameter, thus extracting its type. The big problem traditionally now is how to get
	that type out of the vartypeID() instance, and here's how we do it:
		1. unique_type_id() returns a monotonically increasing integer for every unique type
		   passed to it during this compilation unit. It also specialises an instance of
		   msvc_extract_type<unique_type_id, type>::id2type_impl<true>.
		2. vartypeID() returns a sized<unique_type_id> for the type where
		   sizeof(sized<unique_type_id>)==unique_type_id. We vector through sized as a means
		   of returning the unique_type_id at compile time rather than runtime.
		3. msvc_extract_type<unique_type_id> then extracts the type by using a bug in MSVC to
		   reselect the specialised child type (id2type_impl<true>) from within the specialisation
		   of itself originally performed by the above instance of unique_type_id. This bug works
		   because when MSVC calculated the signature of the specialised
		   msvc_extract_type<unique_type_id, type>::id2type_impl<true>, it does not include the
		   value of type in the signature of id2type_impl<true>. Therefore when we reselect
		   msvc_extract_type<unique_type_id>::id2type_impl<true> it erroneously returns the one
		   already in its list of instantiated types rather than correctly generating a newly
		   specialised msvc_extract_type<unique_type_id, msvc_extract_type_default_param>::id2type_impl<true>

	This bug allows the impossible and gives us a working typeof() in MSVC. Hopefully Microsoft
	won't fix this bug until they implement a native typeof.
	*/

	struct msvc_extract_type_default_param {};
	template<int ID, typename T = msvc_extract_type_default_param> struct msvc_extract_type;

	template<int ID> struct msvc_extract_type<ID, msvc_extract_type_default_param>
	{
		template<bool> struct id2type_impl; 

		typedef id2type_impl<true> id2type; 
	};

	template<int ID, typename T> struct msvc_extract_type : msvc_extract_type<ID, msvc_extract_type_default_param> 
	{ 
		template<> struct id2type_impl<true> //VC8.0 specific bugfeature 
		{ 
			typedef T type; 
		}; 
		template<bool> struct id2type_impl; 

		typedef id2type_impl<true> id2type; 
	}; 


	template<int N> class CCounter;

	// TUnused is required to force compiler to recompile CCountOf class
	template<typename TUnused, int NTested = 0> struct CCountOf
	{
		enum
		{
			__if_exists(CCounter<NTested>) { count = CCountOf<TUnused, NTested + 1>::count }
			__if_not_exists(CCounter<NTested>) { count = NTested }
		};
	};

	template<class TTypeReg, class TUnused, int NValue> struct CProvideCounterValue { enum { value = NValue }; };

	// type_id
	#define unique_type_id(type) \
		(CProvideCounterValue< \
			/*register TYPE--ID*/ typename msvc_extract_type<CCountOf<type >::count, type>::id2type, \
			/*increment compile-time Counter*/ CCounter<CCountOf<type >::count>, \
			/*pass value of Counter*/CCountOf<type >::count \
		 >::value)

	// Lets type_id() be > than 0
	class __Increment_type_id { enum { value = unique_type_id(__Increment_type_id) }; };

	// vartypeID() returns a type with sizeof(type_id)
	template<int NSize>	class sized { char m_pad[NSize]; };
	template<typename T> typename sized<unique_type_id(T)> vartypeID(T&);
	template<typename T> typename sized<unique_type_id(const T)> vartypeID(const T&);
	template<typename T> typename sized<unique_type_id(volatile  T)> vartypeID(volatile T&);
	template<typename T> typename sized<unique_type_id(const volatile T)> vartypeID(const volatile T&);
}

#define typeof(expression) msvc_typeof_impl::msvc_extract_type<sizeof(msvc_typeof_impl::vartypeID(expression))>::id2type::type
#endif

I hope you find this useful. For this and loads more useful C++ stuff, try the Boost libraries or my own TnFOX portability toolkit.