The pyramidal computing resources infrastructure

Supercomputers are often referred to as forming the top of a pyramid of resources, which then in turn are referred to as mid-level systems and basic resources at the bottom. Although the various pyramids may yet differ in height or broadness of base, this picture makes much sense. The picture usually reflects at the same time different layers of responsibility, such as – going from bottom to top – to individuals and research groups, universities of institutes, national and/or European responsibilities. Again at the same time these responsibilities reflect usually also the cost in terms of real money of investment in any of these layers. The higher the direct investment level, the more cooperation and the more strategic visions are required to get the funding and the broader context in which the funding takes place. But the pyramid is not just a series of resources ranked after performance and stacked on top of each other. The pyramid is essential to a solid research computing environment and should be kept in tact at all times.

For the foreseeable future there seems no limit in wanted computer resources/cycles for science and research. This insatiable need can only be satisfied if at all levels of the pyramid the required resources are available and sustainable: desktop facilities, local clusters (or a gridified cluster environment) for direct and on demand access, higher level systems for (very) demanding codes at the national level (or an equivalent gridified environment) and national/European level supercomputers for “qualified” science/research. The pyramidal structure is essential for this scheme, because building on the top of the pyramid only makes sense if the base is well founded. If the middle layer, that is due to be formed by national HPC systems, is absent, the work permeates one way or another to either other layers. This leads to a cost ineffective use of those resources if not directly to a shortfall in scientific production and quality. The disappearance of the middle layer may readily result from a funding scheme for the top where effectively the funding for the middle layer is used to pay for the top. Such a scheme would not yield a continuum of resources and certainly not more cycles for scientists but merely form another bureaucratic complication for the access to the still urgently needed cycles by scientists and researchers.

The grid infrastructure is the vehicle that glues all resources in the pyramid to one transparent environment and that has to guarantee that all resources in the pyramid will be optimally used for their very purposes. This should not be under utilized nor be infinitely queued up. The layers in the pyramid are admittedly not very sharp. A system based on a given technology may be in the basic layer if the number of processors is small, in the middle layer if the number is significant and at the top if their number is abundant. But in addition to these layers based on performance of “size” it is also the access possibilities that co-determine the layers: at the basic level the access is instantaneous, at the middle level there may be queues or appointments with colleagues or other user groups about sharing policies. At the top an additional hurdle must be taken, which usually is peer review. But without local and institutional /national and easily accessible resources it will be difficult to make one’s case for access to the peer reviewed environment.

The form of the European-wide pyramid may, in addition to the elements already mentioned, become more dependent on the total cost of ownership/cost per cycle than has been the case. This is due to a few parameters that were around already but have become differently valued more recently. Such parameters include energy cost, heat production, support cost.

It is important to provide the suitable level of resources in the pyramid for the research groups in order to guarantee cost efficient usage and still reserve the high-end resources only to those applications that can really benefit from such a system.

Figure 1: Schematic view on the integrated performance with an 18-months investment cycle, a doubling of the performance of each new system and an 6-year depreciation period (y-axis is logarithmic)


, , , , , , , , ,

  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: