Design

The Square Kilometre Array (SKA) is the World’s largest mega-Science project of the next decade. It represents numerous firsts for New Zealand, being the World’s biggest Big Data project, the largest Science project in which NZ has ever had substantial lead roles, and possibly the largest NZ involvement in an international ICT collaboration. Once the receivers, infrastructure and supercomputing systems have been built scientists in the member countries, including NZ, will have access to the World’s largest and most sensitive radio telescope. With it they will make fundamental advances in our understanding of the Universe over the next 50 years.

Megaprojects are well known for providing substantial innovation spinoffs. Like past megaprojects, the SKA is seen as a key driver of technology innovation, particularly in computing hardware and software technology due to its unprecedented data requirements.

The New Zealand SKA Alliance (NZA) consists of three universities (AUT, Auckland, Massey) and four companies (Catalyst IT, Compucon NZ, Nyriad, Open Parallel). The collaboration focuses on the challenging computing design for the SKA. Established in 2013 and aiming for SKA phase 1 construction 2018-2023 it will be one of the longest and largest academic-industry collaborations in NZ.

The SKA is the unprecedented as a Big Data project, requiring the processing of data at rates an order of magnitude larger than the entire global internet traffic. In phase one alone the SKA will receive data at 160 Terabytes/second, which is almost six times the entire global internet rates in 2015, and 700 times the entire data processed by Google worldwide. NZA is one of the largest computing groups in the SKA worldwide and accounts for around 90% of NZ’s involvement in the SKA. It is at the blast furnace end of these data rates, designing next generation solutions to data challenges at a scale the ICT industry won’t see for another decade.

Besides a computer system being designed and prototyped that can process data at unprecedented rates, it also needs to be staggeringly energy efficient. In partnership with other international SKA teams, NZA are developing next-generation platforms, firmware and software that will replace the computing capability of supercomputers such as the Chinese Tianhe-2, at 30 PetaFLOP and consuming 40 MegaWatt power, with an SKA high-data throughput design capable of 100 PetaFLOP while consuming only 5 MegaWatt power. This has required significant advances in computing technologies, from the processor level through to entire supercomputer clusters.
Working at the scale of the SKA requires significant innovations. There have been numerous technological and engineering innovations made by NZA to meet the SKA computing requirements, including:

  • Novel high performance computing using low power processors more typically found in embedded devices,
  • Significant implementation improvements in signal processing algorithms such as FFT and gridding,
  • New memory chip architecture for better energy efficiency when using streaming data,
  • Novel high-level approaches to employing efficient FPGA devices for high performance signal filtering,
  • Advanced middleware design for the SKA using open source components,
  • Revolutionary streaming data compression schemes that adaptively optimise the compression format to store and process data,
  • Progress on high performance computing cluster management platform to meet SKA needs as well as industry needs for Big Data and Internet of Things,
  • First end-to-end signal processing model for the SKA to optimise system design,
  • New high-throughput data communication standard AXIoE,
  • New VLSI correlator designs to handle enormous data rates,
  • Development of and progression of open tools and standards for next-generation computer systems,
  • Business model for spinning off the benefits of high performance computing to small and medium sized organisations giving power performance gains without relying on Moore’s Law.

These innovations revolve around solving the unprecedented Big Data and Exascale computing challenges posed by the SKA, and range from new hardware and improved ways of utilising existing hardware, innovative algorithms and highly optimised implementations, through to system level software that more efficiently orchestrates large computer systems. Power efficiency has been a driving concern, which will also emerge as the dominant issue in data centres and cloud computing facilities in the future.

The design and construction of the SKA is an enormous Engineering feat, with just phase one requiring four years of design and six years construction. NZA is providing about 10% of the required 350 personnel worldwide to design the system, and one of the largest groups in its computer design, so facing many of its unique challenges. Project management has had unique challenges to overcome, developing ways to coordinate a combination of academic and industry partners, coming to the project with quite diverse motivations and expectations.

The SKA has been a catalyst for bringing NZ universities and local industry together to work on what will be their largest and longest term collaboration. As a result all are gaining early exposure to new ICT technologies and helping drive future directions of global technology developments. Each organisation has been able to build internal capability, recruit new expertise, and some have already generated new IP for commercialisation.

The IP generated by NZA is advancing. Massey University has developed advanced low-power integrated circuit designs which NRC Canada is looking at for array beamforming outside the SKA. A startup company (Nyriad) evolved from NZA’s SKA data compression work and is building a very successful business model around liquid data, already growing to 30 employees and securing millions in VC funding. Open Parallel has also secured seed capital to develop IP associated with its SKA work.

The team’s work has raised the profile of the NZA organisations through TV, radio and print media articles. Nationally it has attracted talented people drawn to interesting work, and encouraged students to study Science, Technology, Engineering, Maths subjects, critical to NZ’s growth as a knowledge economy.

NZA has successfully put NZ on the world stage through its achievements in the SKA. Its work has forged an international reputation for NZ as a can-do country in challenging megaprojects, giving us a foot in the door for involvement in future big projects. It has fostered international relationships, particularly with Australia (CSIRO), Canada (NRC Canada, MDA), Netherlands (ASTRON), South Africa (CHPC, SKASA), and UK (Cambridge, Manchester, Oxford, STFC), through very close collaborations. This has opened the door to business ventures beyond the SKA.

Via AUT as the legal entity and lead organisation NZA has contractual agreements to work with an international consortium led by NRC Canada in the SKA Central Signal Processor, and with another international consortium led by the University of Cambridge in the Science Data Processor. These agreements have been supported by MBIE funding via NZA to AUT, Catalyst IT, Compucon, Massey University, Open Parallel and the University of Auckland across four grants totalling $1.9 million since 2014. NZA member organisations have contributed approximately double this themselves in co-funding to ensure NZ participates in the SKA at an internationally significant scale.

Over the past two years NZA has produced research publications in journals and conference proceedings, over 20 refereed technical reports detailing aspects of the SKA computer design, two keynote and three invited addresses at conferences, and organised five NZ-based SKA conferences and colloquia.

The NZA successes in the SKA show that NZ punches above its weight in the international arena and sets the stage for our country’s involvement in future megaprojects. The profile has helped stir interest amongst multinationals over NZ as a significant hub for technology development, and encourages aspiring youth by showing we can push technology boundaries from within NZ.
Its innovations span major technological and engineering advances, with generated IP resulting in commercial spinoffs and Big Data applications outside the SKA, new industry standards, and progress benefiting the open source software community. These innovations address many of the Big Data challenges that the IT industry would have been facing in future. It sets a successful example of science translation to industry within NZ and how academia and industry can openly collaborate.

Perhaps most excitingly are the innovations that are still lie ahead for NZA, as the SKA project will further push boundaries when it moves into phase two next decade, with data rates increasing an additional two orders of magnitude.
Lastly, it is helping build a system that will answer fundamental questions about the Universe over the next 50 years, which is a very stimulating worldwide goal for NZ to contribute toward.

Tech Blog

Restructuring a RAM Multiplexer for Performance in an Intel Stratix10 FPGA

William H. M. Kamp, Ph.D, High Performance Computing Research Lab, Auckland University of Technology, New Zealand. Context The SKA Mid-frequency Correlator dumps 20100 visibility products from its Matrix style cross correlator function every 190 microseconds. It does this over 20 buses each approximately 312 bits wide, with a total instantaneous bandwidth of over 500 GB/s. This dump of data must be serialised and processed by the long term accumulator to an external DDR4 interface that limits the bandwidth. This is achieved with a debursting buffer that has 20 independent write-ports and a single independent read port. The design requires that the Read more …