
Gi's a job
I'll stroke some of their servers if they want.
Amidst the paroxysms of coronavirus and Brexit, the United Kingdom on Wednesday found time to ratify the Convention that formally establishes the SKA Observatory (SKAO), paving the way for the giant radio telescope to be built. The Convention governing the SKAO requires that at least five nations, including the three host …
If you think the Kalahari / Karoo is a nasty, hot place, just be glad you're not going to Johannesburg or Pretoria. Just last week my mother's elderly neighbour almost was hijacked at the local supermarket. When she got into her vehicle a man climbed into the passenger seat and started giving her directions where to drive to. Lucky for the neighbour, a car guard saw what happened and was able to rescue her. This is in broad daylight in what is presumably the better side of Pretoria. A few days later two elderly people were assulted and robbed at gunpoint at a garden center not far away from the supermarket. Nasty place indeed.
That was part of a diplomatic slap fight between building it in Oz or Za.
Australia expressed "health and safety concerns" for staff in South Africa, to which J'burg replied along the lines of "you're aving a larf - what about giant venomous spiders setting up home in the dishes if you put it in Oz?"
In true diplomatic fashion it is now in both countries.
Large parts might be putting it a bit optimistically. Australia and South Africa both have some test kit set up on site. Out of the planned 3000 dishes and 130,000 attenae, they have 100 dishes between them, all significantly smaller than the ones that will be used in the actual SKA. It's an important part of the design and prototyping process, and obviously can do plenty of useful science in its own right, but it's a very, very long way from being a large part of the final thing.
It's always been like that with radio telescopes. I remember being shown a Manchester University network map 30+ years ago, one link on which was labelled "M6Net". Of course I asked what this was - it turned out to be a graduate student driving from Jodrell Bank to Manchester every Friday afternoon with a boot full of 1/2" magnetic tapes containing that week's observations.
Manchester in those days had one of the national supercomputing centres, in large part to process Jodrell Bank data.
is lacking an entry for computer storage...
Final annual output is expected to be 130 petabytes a year, still a hard-to-wrangle quantity of data for the world’s astro-boffins.
https://www.theregister.com/Design/page/reg-standards-converter.html
Yet another over the top pi**ing contest multi-govt project - 8.8 terabyte of data per second from each dish x 130,000 of the beggars - how much of this data is actually going to be professionally analysed/used ? Surely it would be better to go for lower amount of quality based data rather than going for an all out scatter gun approach and trying to grab all the information in the observable universe. Heck, for the humongous amount of money potentially to be spent on this project (think hardware, data management, staff, logistic and political costs), what about about being truly forward thinking and setting up a radio dish array on a smaller scale on the far side of the moon as has been proposed for a while - probably would work out cheaper and get proper research quality data on a useable scale - better get Elon Musk in on the job !
The data will probably be hardware filtered in the same way that CERN reduces the petabyte of data per second from the LHC. They have banks of FPGAs that bring the data rate down to something acceptable.
Just because you can generate 8.8TB/s, does not mean that is what you have to operate with. Change the filter to change the dataset so you have a 'lower amount of quality based data' but you can change the focus.
That number is a bit "marketing speak". Imagine listening to radio1 on a software defined radio, that's 100Mhz at 16bit = 200Mb/s. But it quickly cooks down to much less "information" / second.
The reason the data rate from each dish is so high is that the 'raw' data can now be digitised and then a bunch of correlation / filtering /frequency selection etc that we used to do in HW can now be done more efficiently and cheaper in software,