Credit: NATL INST. FUSION SCIENCE

On the outskirts of Yokohama, just outside Tokyo, a very special computer has been assembled. Housed in a building the size of an aircraft hangar, the Earth Simulator boasts some impressive statistics. Some 220 tonnes of cables link its 5,120 processors. Its combined memory is tens of billions times larger than that of a new desktop machine. And when running at full steam, it should be able to perform 40 trillion calculations per second, making it the most powerful computer in the world.

The Earth Simulator has been designed to model the climate and geophysics of the Earth with unprecedented accuracy. Initial tests of the machine began earlier this year, but serious questions about its operation remain unresolved. Getting the most out of such a machine requires new computer software and improved theoretical climate models, and Japan lacks the appropriate expertise. Collaborations could provide a solution, but the project's backers have yet to decide on how much access foreign researchers should be given. And at a price of about US$370 million — plus up to $50 million in annual maintenance costs — some observers suggest that the machine is costing the Earth, rather than simulating it.

The project has its roots in politics as much as in science. The machine uses vector processors, the manufacture of which is dominated by Japanese companies. These chips, which simultaneously perform the same mathematical operation on arrays of data, or vectors, were the mainstay of scientific computing at the beginning of the 1990s.

But the vector computer industry did not fare well during that decade. The performance of the conventional processors used in desktop machines increased rapidly throughout the 1990s. Although each can only perform calculations on one data point at a time, hundreds or thousands of these relatively cheap chips can be wired up to create powerful 'parallel' supercomputers. In 1997, the import of Japanese vector machines into the United States was banned following a trade dispute between the two countries. So researchers in the United States — the biggest single market for supercomputers — turned increasingly to parallel systems.

Climate of change

Spaghetti junction: hundreds of tonnes of cables link thousands of vector processors to make the Earth Simulator the most powerful supercomputer in the world. Credit: NATL INST. FUSION SCIENCE

By the late 1990s, Japan was looking for ways to help its struggling vector super-computer manufacturers. A vector-based climate machine was an obvious choice. Climate researchers had not willingly abandoned vector machines, as it is difficult to adapt some climate models so that they run efficiently on parallel systems. And with a relatively weak base in climate science, the Japanese government was aware that Japan was contributing little to the global effort to understand climate change. Plans for the Earth Simulator were launched in 1997, the year Japan hosted the climate conference that launched the Kyoto protocol on limiting greenhouse-gas emissions. The Japanese information technology company NEC, based in Tokyo, was given the contract to build the simulator.

By that time, large parallel systems had become considerably faster than single-processor vector supercomputers. To match the speed of these new systems, the Earth Simulator's designers took a hybrid approach. The result is a parallel machine built with vector processors, and wired with extremely fast interconnections.

This speed brings various advantages. Some researchers, for example, simply want to perform multiple simulations with existing climate models. Some outcomes of these models, such as the size of the temperature increase that greenhouse gases are expected to cause, are very sensitive to the details of the model. Running slightly different versions of the same model gives researchers an idea of the range of possible increases. A similar process is routinely used in weather prediction, says Thomas Stocker, a climate researcher at the University of Bern in Switzerland: “It must be done for climate predictions as well.”

Another advantage is the possibility of higher-resolution models. Climate models divide the atmosphere into cuboid cells. The smaller the cell, the more realistic the modelling process can be, but smaller cells need more time to process. Time constraints mean that existing climate supercomputers use cells with sides about 100 kilometres long when running global models. The Earth Simulator, in contrast, should be able to run simulations using cells with 20-km sides.

Smaller cells should help to capture features that existing models miss. John Mitchell, a modeller at the UK Meteorological Office's Hadley Centre for Climate Prediction and Research in Bracknell, west of London, says that features such as squall lines — rows of storms that form in the tropics — could be represented by the Earth Simulator.

The same holds for ocean simulations, where important currents in the Atlantic and at the Equator are not represented well in today's models. “Higher resolution requires increased effort,” says Stocker. “But these investments are fully justified by the increase in the realism.”

But moving to higher resolutions can be difficult. Mitchell cites the example of modelling convection in thunderstorms. The storms are driven by the heat released as warm, moist air rises and the water vapour condenses. Existing models simplify the process, as it occurs on a much smaller scale than the cells used in the model. But there is no guarantee that the same simplification will work for cells with 20-km sides, which are close to the scale on which the convection occurs. Unless current models are adapted, says Mitchell, they could start to produce unrealistic results.

Large parallel systems pose other problems. Simulations only run efficiently when the computational load can be spread evenly around all the processors in the system. In the case of climate simulations, each processor handles a set number of cells. But if one chip takes longer than others to complete a step in the model, the rest of the system must stop and wait. Even small delays can have dramatic effects on the system's performance, and such effects are impossible to eliminate completely.

Computer scientists in the United States have accumulated considerable experience in developing the software needed to minimize such problems, but the relevant expertise is rare in Japan. “With all the vector systems around, there aren't that many people in Japan who have experienced the sheer difficulty associated with large parallel computers,” says Yasumasa Kanada, director of the University of Tokyo's computer centre.

Tetsuya Sato: wants foreign researchers to get involved.

Collaborations with foreign groups could help, but the Japan Marine Science and Technology Center in Yokosuka, close to Tokyo, which manages the Earth Simulator, has yet to draft guidelines for the machine's use. The Japanese government has only said that any work done on the system has to address the country's policy needs. This could, for example, mean focusing on climate changes specific to Asia, or the risk of earthquakes in and around Japan.

Officials at the Japan Atomic Energy Research Institute in Tokyo, the research agency that made the biggest contribution to the simulator's set-up costs, originally suggested that use be restricted to scientists based at Japanese institutes. But theoretical physicist Tetsuya Sato, the new director of the Earth Simulator Center, has been lobbying for considerable access to be given to foreign teams. Sato also suggests that time could be allotted to simulations in other disciplines, such as nuclear physics, nanotechnology and life sciences. With the system still undergoing testing, and some of the software perhaps years away from completion, the debate could continue for some time.

Whether or not they are granted access to the Earth Simulator, climate researchers in Europe and the United States are already debating whether to lobby their paymasters for similar machines. Although most agree that fast machines are useful, some are asking whether their limited budgets would be better spent in other ways.

Elementary interaction

Keep it simple: Didier Paillard works with cheaper machines.

Didier Paillard, a climate modeller at the French Atomic Energy Commission's research centre in Saclay, near Paris, says that very simple models, which can be run on relatively cheap computers, can provide results different from, but just as useful as, those produced by high-resolution simulations. Rather than improving the resolution of the oceanic and atmospheric components that make up the models, Paillard argues that attention should be focused on understanding the interactions between the two systems.

In the simple models favoured by Paillard, both the ocean and the atmosphere are represented by just a few cells. Although they cannot produce accurate climate simulations, these models offer insights into ocean–atmosphere coupling that more complex models miss. They can also reveal mistaken assumptions. “Ten years ago, the ocean was still assumed to limit climate change,” says Paillard. “Now it is seen as a possibly very active system. Increased computing power didn't contribute much to elucidate this point.”

Detailed outlook

But others defend the importance of resolution. “I used to be sceptical of the blind pursuit of resolution,” says Richard Rood, a climate modeller at NASA's Goddard Space Flight Center in Greenbelt, Maryland. “But it brings phenomena closer to the scale on which observations are made.” This, Rood argues, will allow observations to have a greater impact on models. “This is a critical bridge that must be crossed.”

Whatever decisions other nations make, financial constraints in Japan mean that the Earth Simulator is likely to mark the end of the country's tradition of big computer hardware projects. “We are now looking at what kind of computing performance scientists really need, rather than at what is technologically feasible,” says Masashi Shinozaki, who is responsible for high-performance computing at the Ministry of Education, Culture, Sports, Science and Technology.

The Earth Simulator is, nevertheless, a remarkable achievement. Regardless of the politics behind its inception, the result is a machine of unprecedented capability. And if Sato gets his way, its impact could be felt beyond the limits of nationality and disciplines that some of its original backers proposed. It is not too late for the Earth Simulator to have a truly global impact.

→ http://www.es.jamstec.go.jp