Friday, January 28, 2011

Scalability of Systems

When confronted with the redesign of a huge DoD system the issue of adequate scalability for the handling of transactions will always come up. When planning for the deployment of large enterprise-level programs Program Executive Officers (PEOs) are apprehensive about the scale of what is proposed.

PEOs will argue that the capacity and the inflexibility of the DoD infrastructure will limit the scalability of any central application. If the network, its switches and routers have limited capacity, then any central computer service will not scale. Distributed processing, with local operations, will more likely deliver the desired level of service.

The organization of the application code and databases will also influence how much centralization is possible. If local operations, in the Services and the Agencies, retain control over versions of the proposed software, a Service Oriented Architecture (SOA) will be hard to implement. DoD-wide management of an over-arching SOA design will be too difficult to implement.  Excessive variability cannot be scaled without added costs for maintenance manpower.

  Disregarding likely objections, systems scalability must be first defined in terms of its transaction processing capacity. If a system does not have the power to handle customer transactions, PEOs will be justified to break up programs because it is not “scalable”.

The metric for evaluating scalability is the number of transactions per minute processed against a database (usually Oracle) dated December 2010.  A report on "Oracle Databases on vSphere Workload Characterization Study" shows that a relatively small and cheap virtualized server configuration  (<$200,000) will handle 505,000 transactions per minute. *  Such capacity exceeds by more than a multiple of 100 the average Oracle database processing capacity.

 Transaction-processing power is now available at a very low cost. Therefore, PEOs will have to start looking for other ways to deliver more effective results.

Availability of a relatively inexpensive system with the capacity of processing 500,000 transactions/minute suggests that it could accommodate ten thousands of administrators engaged in conducting related activities simultaneously.  Even with multiple redundant operations for fail-safe operations, it is unlikely that DoD can generate demand for unaffordable transaction-based systems in DoD.

Concerns about the scalability of DoD systems to handle application workloads are misplaced. Any restrictions on processing power will arise from the inability of DoD networks to support traffic, or from small-scale contracts that will result in an excessive variability in design.

New virtualized technologies and the availability of inexpensive multi-core servers favor the centralization of information processing. Concerns about the practical limits on scalability have not merit.
The economics of data processing favors consolidation. Arguing against that is the fear that central operations will result in unmanageable delays as well as in budget over-runs. Questions about scalability are therefore matters that concern how systems are managed rather than which technology is deployed.