Aside from initial installation and commissioning, what’s typically one of the most important, and carefully managed tasks that operators have to tackle on a regular basis? The answer is simple: system upgrades. The process however, is nothing short of magic.
Guest blog by Ron Breault.
Think about it: if you’re an operator, you’re delivering services in a very tight market, with aggressive competitors constantly looking to lure away your customers. Your business users and consumers demand no downtime, 7×24 availability, and yet want all of the latest features and capabilities ASAP. You need to move forward, upgrading your systems, but you can’t disrupt operations.
How do you accomplish that? Thankfully, after decades of work between operators and telecommunications equipment providers, network upgrades have become a well understood science.
Detailed processes and features are in place to deliver ‘rolling upgrades’ to every node in the network. Operating systems are upgraded; new telecom applications are deployed; databases are upgraded; data is migrated to new formats; protocol versions are up-reved, etc. all without service impact, so that the entire process is transparent to businesses and consumers – they don’t see a thing. It’s almost magic, really.
But wait- now we’re operating in a new paradigm, with virtualisation and cloud technologies underpinning telecommunications services. NFV has moved from concept to reality, with OpenStack seemingly everywhere. Does that impact the upgrade process? Should operators be concerned? The answer is an emphatic yes.
OpenStack came into existence for the information technology and enterprise world. This is a world where downtime can be carefully planned and systems can be shut down for several hours at a time while new systems deployed and activated.
Think about how many times you’ve received emails at your office telling you that some important system (accounting, email, payroll) will be down over the weekend for upgrades – we’ve all seen these. And yet this same technology is now being used to host virtualised network functions, functions such as call servers, packet cores, routers, firewalls and session border controllers (to but name a few). To make matters worse, OpenStack is not some small project with a long, slow lifecycle. It’s a massive open source undertaking, literally evolving and spinning out new versions twice year.
If OpenStack included a nice, smooth, in-service process for system upgrades, operators using it could breathe a sigh of relief – unfortunately, the industry isn’t there yet. If you’re looking for a ‘fun’ way to spend your weekend, go to the OpenStack website and browse through the numerous documents, wikis and webinars describing the various upgrade processes for the individual projects comprising OpenStack (Keystone, Nova, Neutron, Cinder, etc.). Much more work is clearly needed to achieve the level of process maturity necessary for standard OpenStack upgrades in a telecom environment.
It is a pleasure to say that operators who have deployed the Wind River Titanium Server network virtualisation platform are in for a completely different experience when they upgrade to a new release.
Titanium Server was designed and built by the same telecom engineers who created the ‘magic’ that’s behind the upgrade process used in traditional, non-NFV networks. These engineers understand the constraints operators are faced with in performing upgrades. They understand that upgrades can’t impact service – zero downtime – and that the process itself must be easily understood and fully automated.
Recently there was a demonstration of such an upgrade in person, live. Using the upgrade tool that’s an integral part of Titanium Server, I watched as a complete upgrade of a running system took place.
Our framework automates the entire process from beginning to end: VNFs are live migrated from one node to another (even those using DPDK); the system operating system is upgraded; OpenStack components and services are upgraded; the virtualisation infrastructure manager is upgraded.
During the process, messages are displayed, and logs and alarms are generated to inform operators of the progress of the upgrade. Rollback points are created for controlled fallback, if required, and all system data is reformatted and migrated as necessary. In short, Titanium Server makes system upgrades in the world of NFV behave just the way they did in the non NFV world – an important, but well managed event.
Courtesy of Wind River.