Scale or scale: How and why?

There are two scaling strategies: scale-up and scale-out. How do they differ and when is which variant used in IT? Two examples.

«Nine women can't deliver a child in a month either»

Author: Filka Yaneva

The term «scalability» is on everyone's lips in the IT sector and is often formulated as a basic requirement. But when is a scalable system really necessary and how does scaling actually work? Is there a universal recipe for this?

In IT, scalability means the ability of a system to proportionally increase its overall performance under increased load (more data, more user requests) by adding resources (typically hardware). A system scales optimally when performance increases linearly with the added resources - but in practice this can only be achieved with a specially adapted software and hardware architecture.

Two examples

Let's take a look at two concrete situations that call for a scaling solution:

  1. «Big Data Analytics»: Sale Corp. is eager to understand what their customers think about them and what could be attractively offered to them. This should be revealed by continuous analyses based on posts in social media, for example. To do this, Sale Corp. needs a high-performance architecture that can be equipped with additional hardware on an ongoing basis to handle the storage and processing of ever-increasing amounts of data. 
  2. «Video Analysis»: The company Sauber AG uses an optical sorting line for waste separation. By analyzing the video data from the waste conveyor belt, a sorting system decides in real time whether a waste item is to be filtered out or not. Now Sauber AG wants to upgrade its sorting plant so that more waste can be sorted even faster.

How can the two companies best scale their systems? And what determines what a well-scaled solution must look like?

Scaling strategies

In general, two types of scaling strategies can be distinguished: The horizontal scaling (scaling-out) and the vertical scaling (scaling-up).

Scale-up.jpg
The basic principle of the scaling strategies «Scale-up» and «Scale-out». Source: Ryan Strehlke, Turbonomic

Scaling-up (vertical scaling)

Scaling-up refers to the addition of resources within a logical unit.

Examples of this are: Deploying more powerful memory (such as flash storage), adding CPUs or GPUs to an existing server, or upgrading memory.

The software to use multi-core architectures in the context of a single application is usually implemented by parallel programming, e.g. by multithreading and interprocess communication. Various libraries and compiler extensions are available for this purpose, such as OpenMP, CUDA and OpenCL, etc.

Typical application examples of vertical scaling are real-time applications for visualization or image processing such as virtual reality, computer games, sorting systems or inline quality inspection on the production line. The calculation result, e.g. in interactive applications, often has to be fixed in real time and is usually composed of partial results, which can also be interdependent. The partial results are calculated on separate threads for optimal scaling, which often requires synchronization of data between several threads. These all therefore need access to the same, fast memory (for example, main memory or GPU memory).

Scaling-out (horizontal scaling)

Scaling-out refers to the addition of several logical resource units (nodes) to a system, which are connected to each other. With the aim that they function like a logical powerful unit.

Examples of this are: Increasing the number of available servers in a data center (where, for example, one server serves as a proxy or load balancer), or cloud architectures that have clusters of virtual nodes.

Special software tools for distributed programming are required to operate such a distributed system. Typical implementation patterns for this are MapReduce, Worker/Master Schema orBlackBoard.

Horizontal scaling is usually used for «big data» applications that store and evaluate huge amounts of data in clouds, e.g. to calculate real-time statistics in social media, to serve real-time user requests (Google search), to predict outbreaks of diseases, or to analyze consumer behavior

The elementary calculations within distributed applications usually have a low to medium complexity. The partial results produced in them are already greatly reduced in volume and dimensions before they are combined to a complete result. An example is the calculation of the number of mentions of relevant terms in social media to determine trends.

Assets and drawbacks

Both scaling methods have their advantages and disadvantages.

Significant advantages of vertical scaling are easier implementation and maintenance. Both main memory and data memory remain in one logical unit. This makes data exchange by passing memory references and using caches very efficient in many cases. In addition, no changes generally need to be made to the software to benefit from the increase in performance.

However, vertical scaling is limited upwards by the maximum hardware equipment of the computer. For example, it is not possible to install an unlimited amount of RAM in a computer. More powerful hardware also quickly becomes very expensive.

Adding standard hardware to a cluster is much cheaper and not limited to the capacity of a single unit. However, the lack of (fast) shared memory when scaling horizontally makes communication and data exchange within the cluster more difficult and makes sharing, forwarding and updating data much more complex. Only with special software architectures and maintenance tools can this be implemented efficiently and in an error-resistant manner.

Is it possible to choose scale-up or scale-out at will?

No, in many cases this is already predetermined by the existing scaling problem. With regard to the two companies, Sale Corp. and Sauber AG, this can be explained more precisely:

  • If Sale Corp. decides to purchase a very powerful machine, this would be a very expensive investment. If the load is increased further, it can nevertheless quickly reach its hardware limits. In this case, vertical scaling is rather unfavorable.
  • If the Sauber AG company wants to run the conveyor belt twice as fast, even two individual optical sorting systems will not help to maintain waste separation properly. Both systems will miss the waste to be sorted before the decision for or against sorting is made. However, a second conveyor belt (scale-out) would be much more expensive than a faster sorting system (scale-up). In this case, only vertical scaling makes sense. 

When deciding whether scale-out or scale-up, or even whether both approaches are required at the same time, the nature of the respective problem and the properties of the data play a decisive role:

If a real-time response is expected from the system and the final calculation result can only be determined by frequent synchronization of intermediate results, vertical scaling is generally advantageous. However, if a large number of small data sets are involved that can be easily partitioned, horizontal scaling can offer great advantages. However, complex software adaptations must first be made in order to benefit from the additional computers.

In the special field of numerical simulation, both many and very large data sets often have to be processed. In this case, scaling in both vertical and horizontal directions is necessary. This task is usually handled by supercomputers.

Comparison and use cases for scale-up and scale-out

The following table compares the two scaling strategies and common use cases.

Scale-up-2.png
Fig. 2.: The scaling strategies «scale-up» and «scale-out» in comparison

Conclusion

Scalability refers to the ability of a system to grow in order to cope with increasing demands. How the concrete solution for this looks like can be very different. The above examples show that the decision between scaling-up and scaling-out is by no means trivial and depends very much on the analysis problem, the data and the required response times. However, the costs of the respective solution are also decisive. A new application that is to allow substantial performance improvements in the future is best designed to support scale-out.

For both companies, Sale Corp. and Sauber AG, it is important to find a scaling strategy that suits them to achieve a good cost/performance ratio. For Sale Corp., the scaling-out strategy has proven to be the most favorable; for Sauber AG, on the other hand, the scaling-up strategy.