I understand the maths involved in the reconstruction to create a virtual telescope, but what boggles me (and betrays my real experience) is the sheer scale of the organisation and admin needed to get all of this together and functional.
So you've 'seen' the black hole. Now for the interesting bit – how all that raw data was stored
The black hole image released yesterday needed over a thousand helium-filled disk drives to record it. Regular drives were tried and tested but failed due to air pressure at the high altitudes of the radio telescope sites. Eight radio telescopes at six locations formed the Event Horizon Telescope (EHT) array, which …
COMMENTS
-
-
-
Friday 12th April 2019 11:10 GMT Anonymous Coward
Re: *shakes head in bewilderment*
"[...] procuring, configuring and implementing storage solutions is not usually a core competency of a researcher – nor should it be."
*gets green with envy*
Head of a R&D lab here. Just yesterday I spent an hour trying to find furniture that could fit in our monthly budget.
-
Friday 12th April 2019 17:51 GMT Anonymous Coward
Re: *shakes head in bewilderment*
If you've seen the difference in price between SMOHO storage for ~100TB+ range, like a Synology, and Enterprise storage in the same range, you'd appreciate WHY a hard pushed for cash R&D lab head would choose a "comedy home-brew system" when the university's offering is non-existent.
-
Monday 15th April 2019 11:24 GMT imanidiot
Re: *shakes head in bewilderment*
Find furniture that fits in your overal budget and spread booking the payment over more than 1 month? Or buy a few tables and chair each month? This is mostly a problem that calls for creative bookkeeping to fit whatever arbitrary rules the organisation has put in place.
-
Monday 15th April 2019 14:06 GMT Anonymous Coward
Re: *shakes head in bewilderment*
Or buy a few tables and chair each month?
Local guvmint rules (I'm in South Bananistan) asks for three separate estimates for each unique item. Must get several documents for each seller. Lowest bidder wins. Allow 90-120 days for payment. Bidders know that so all proposals already are 50-100% overpriced.
I've mentioned to our troll in charge of buying stuff that the time we spend on this is worth more than the chairs and desks. Got a lecture on "doing things the way they are supposed to be done".
15 more years until retirement... then I'd have to face the HR trolls.
-
-
-
Friday 12th April 2019 13:19 GMT Jamie Jones
Re: If I might quote from one university's draft Research Data Management document...
<hippy mode>
I was just thinking how great the human race can be when it works together.
It makes Trumps walls and Moggys threats to throw tantrums in the EU even more pathetic.
</hippy mode>
Oh, and Checkmate, flat-eathers!
-
-
This post has been deleted by its author
-
-
-
-
-
Friday 12th April 2019 14:10 GMT rg287
I understand the maths involved in the reconstruction to create a virtual telescope, but what boggles me (and betrays my real experience) is the sheer scale of the organisation and admin needed to get all of this together and functional.
Probably why it took Shep Doeleman more than a decade to make it happen! Years and years of proposals, talks and not a little sucking up to the right people to get the funding and observing time. Huge amounts of preparation just to get the infrastructure in place for 5 days of observations.
And all that is before you get into the intellectual feats to do anything useful with it and develop the mathematics and software to wrangle the raw data into a meaningful result!
-
-
-
Friday 12th April 2019 12:39 GMT A.P. Veening
Re: Flat Earth
My kit is slightly below sea level, just like the ground floor of my house. More importantly, computer manufacturers have been asked in the past to recertify their products to guarantee it from somewhere like 10 meters below sea level to their current maximum height above sea level, just to secure an order from Schiphol (Amsterdam Airport) as it is at minus 7 meters. Luckily for most pilots, air planes mostly use radio altimeters now.
-
-
Friday 12th April 2019 16:57 GMT A.P. Veening
Re: Flat Earth
Only near the ground I think. Many planes don't have radio ones, just barometric. QFE/QNH etc.
Near the ground is the only time it is relevant. Planes used to fly and land at negative heights using barometric altimeters upon an approach of Schiphol, enough pilots got scared shitless the first time this happened to them.
-
Saturday 13th April 2019 14:18 GMT Anonymous Coward
Re: Flat Earth
"Planes used to fly and land at negative heights using barometric altimeters upon an approach of Schiphol, enough pilots got scared shitless the first time this happened to them."
Not a pilot, but ISTR that pilots reference altitude relative to sea level at cruising altitudes and reference ground level during landing/takeoff phases. For barometric altimiters don't they set 0' to correspond to the pressure reading at the airport?
Even if that's not the case, knowing that ambient pressure is always changing, landing at a negative altitude for a roughly sea level airport wouldn't be discomforting. If you're relying on a barometric altimiter reading, it'd be just as unnerving to land at a 3000' high airport on a day when the altimeter was reading 2900'. There's no magic about 0', the hard limit you're worried about is where the runway is supposed to be.
-
Saturday 13th April 2019 15:40 GMT Andre Carneiro
Re: Flat Earth
Above a transition altitude you use 1013.2 hPa as the altimeter reference. Below the transition level you use QNH (altimeter setting corresponding to the air pressure which makes it so that the altimeter will read the aeroplane ALTITUDE) or QFE (altimeter setting that makes it read the HEIGHT of the aeroplane in relation to the aerodrome).
So if you're using QNH and your aerodrome is at 150' above mean sea level (MSL) then your altimeter will read 150' when you're on the ground.
If you're using QFE then your altimeter will read 0 when you're on the ground.
QNH is the setting used most commonly, as radio altimeters are more precise and are the source of data below approximately 2500'.
-
Monday 15th April 2019 13:12 GMT Bill Michaelson
Re: Flat Earth
"For barometric altimiters don't they set 0' to correspond to the pressure reading at the airport?"
They do. In the US, the first part of an airport condition report such as ATIS contains the phrase "altimeter XX.YY (in/Hg)" so that a pilot can appropriately calibrate the pressure altimeter via the Kollsman window prior to approach.
But the pilot is typically quite relaxed anyway because they have another window in front of them that they can look through to see the runway. As long as the altimeter is calibrated reasonably, there will be adequate error margin for a precision approach with 200' AGL minimum to be clear of clouds, and a precision approach supplements vertical guidance via continuous glide slope indication. A non-precision approach will like have 500' minimums or higher. That's about a half inch of Hg slop.
-
-
-
-
-
-
-
Friday 12th April 2019 20:55 GMT Jamie Jones
Re: Flat Earth
Hmmm. Ok, a very small membrane!
https://www.pcmag.com/util_get_image/19/0,1468,i=194580,00.jpg
I was led to believe it was like a tiny pin sized thing made of a balloon-like material. My bad.
-
-
-
-
-
Friday 12th April 2019 10:28 GMT Tom 7
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
The start of streaming is indeed delayed but the end of streaming is a lot lot lot quicker. Which in this case is quite important. In this case latency can suck as much as it likes, In fact 'latency sucks' sucks.
-
Friday 12th April 2019 22:49 GMT JJKing
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
… latency sucks, though.
Not if you get high enough to catch a Jet Stream. ☺
Your Vint Cerf reference in the Title was actually referring to a motorcycle with a sidecar and a new Unix release. Mine's the one with the 300 baud modem in the pocket.
-
Monday 15th April 2019 10:17 GMT Adrian Harvey
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
Title reference is to Tanenbaum in “Computer Networks”. Not Cerf.
Quote details in the link. Tanenbaum was likely paraphrasing others here. Lots of people have made similar statements in various forms.
Pedant icon as we’re both being pedantic!
-
-
-
-
-
Friday 12th April 2019 17:14 GMT Ian Michael Gumby
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
Not really.
While the cost per TB is getting closer, there is still a significant gap.
Its going to be a while, although... I'd say go w SSDs because they will last longer under the stress imposed on them. (Less likely to fail)
I tend to use Samsung pro drives and M.2 cards, New Egg has a 10pack of 3.8TB drives for $9,750USD
That's ~975.00 a drive for 3.8TB which isn't bad. (Never tried them so I don't know how well they will last. ) Samsung has 4TB for around $1,140 USD.
And Intel has a 7.6TB drive.
YMMV
-
Monday 15th April 2019 09:06 GMT Trollslayer
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
Flash memory including the type sued in SSDs are less reliable.
Flash relies on storing an electric charge and that can leak away over time.
Spinning rust requires energy to flip the bit.
The best example was when courts had USB sticks with evidence on and stored them without making certified copies.
2-3 years later they found the USB sticks were unreadable because the charge had leaked away in too many places.
-
-
-
-
-
Friday 12th April 2019 20:06 GMT Jon 37
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
Yep, one of the big challenges of making helium-filled drives was figuring out how to seal them so the helium doesn't escape before the warranty runs out. Of course, if you have a helium-filled drive, I would suggest replacing it sharpish when the warranty does run out, because it might be about to run out of helium.
-
Friday 12th April 2019 23:04 GMT JJKing
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
helium has tiny molecules and leaks through everything.
So that is why my farts stink the up the neighbourhood. They are made of helium and are not filtered by my grundies.
Mine's the one with air freshener and gas mask in the pocket.
-
Sunday 14th April 2019 01:15 GMT J.G.Harston
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
Hmmm? A molecule of hydrogen is two atoms, 2 protons + 2 electrons = 2.001ish. A molecule of helium is one molecule 2 protons + 2 neutons + 2 electrons = 4.002ish. Doesn't that make helium bigger than hydrogen? Certainly I remember doing diffusion experiements in school, and H2 escaped faster than He.
-
Sunday 14th April 2019 08:03 GMT Stoneshop
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
A helium atom is surrounded by a single, fully populated, electron shell; a hydrogen molecule consists of two atoms, each with their shell (which get intertwined to some extent). As the diameter of these shells are by and (not very) large equal, simple geometry would posit that a hydrogen molecule is larger by the distance between their nuclei. Except that at the atom scale simple geometry doesn't quite apply. But anyway, empirical data has shown that He is indeed a bit smaller than H2, 28pm (2.0248024e-13 brontosaurus) versus 31pm (2.2417455e-13 brontosaurus).
-
Sunday 14th April 2019 17:48 GMT Anonymous Coward
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
Helium is not only smaller being monoatomic, it has the advantage from a diffusion point of view that it isn't sticky, like hydrogen (hydrogen bonds), which is why helium is used in leak detectors.
But my solution for keeping the helium in the drives is simple: include some Pu-238, which is an alpha (helium nucleus) emitter. Not only is the supply of helium being constantly restored but even at altitude, where it's cold, your drive stays nice and warm.
I can't see any obvious objections, and for some reason it isn't patented, so feel free.
-
Monday 15th April 2019 09:37 GMT eromana
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
You are correct a H molecule is lighter than a He atom, so the H2 sound speed is greater that of He by a factor of root. Diffusion on the other hand depends on 2 materials and their state. So for example, in air H2 has a longer diffusion length than He. It turns out that the molecular diameter of H2 is slightly larger than that of He, 297 vs 258 (picometer) [1]. Because He is an nonreactive noble gas the diffusion of He through a solid material barrier is much but much greater than the molecular diameter would insinuate. "At 512°C the rate of helium diffusion [in silica glass] is approximately 45 times that of hydrogen, and it is believed that chemical forces delay the progress of the hydrogen molecule through the silicate network"[2]
[1] http://www.kayelaby.npl.co.uk/general_physics/2_2/2_2_4.html
[2] https://aip.scitation.org/doi/pdf/10.1063/1.1750133?class=pdf
-
-
-
-
Friday 12th April 2019 21:51 GMT dfsmith
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
Pfft. Spinning rust is terrible for PB of data. It would take almost 100 million "spinning rust" drives to store 14PB. Luckily, spinning cobalt alloy media is far superior. (The last iron oxide drives were sold in the early 1990s.)
-
Sunday 14th April 2019 15:34 GMT Ian Michael Gumby
Re: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
huh?
Pfft. Spinning rust is terrible for PB of data. It would take almost 100 million "spinning rust" drives to store 14PB. Luckily, spinning cobalt alloy media is far superior. (The last iron oxide drives were sold in the early 1990s.)
Oh, while you're getting to be a word nazi, the term spinning rust is meant as a reference to hard drives.
But to your point spinning drives at 12TB per drive. 100 drives is 1.2PB. 1,167 drives is 14PB.
Now I thought that there were larger drives... like 14TB so that would be 1000 drives.
-
-
-
-
-
-
-
Friday 12th April 2019 13:05 GMT Anonymous Coward
Re: BBC: what?
"You misunderstand the BBC. They were trying hard to explain why this meant brexit was a good idea."
Using their standard technique of having 4 Remainers and 1 Brexiteer for balance? And when questioned over balance declaring that any government minister in-spite of any views said minister then chooses to express (or later follows up with by voting for in Parliament)?
-
Saturday 13th April 2019 17:14 GMT Anonymous Coward
Re: BBC: what?
Yet they permit 1 UKIP activist to appear on and ask questions on at least 4 different question time shows in one year and then plead ignorance when they are called on it, along with cutting the reply from a Scottish Govt minister to make her look incompetent.
This along with quoting UKIP at every chance they get, failing to challenge blatant falsehoods by Brexiteers, having at least one senior writer hauled up for being biased against the main opposition (yet said writer is still in post)
The BBC is pro-Brexit pretending to be impartial, look at the amount of times they've had Brexiteers on TV and given them soft ball questioning and allowed them to air nonsense and then act like its undisputed fact
Read articles from BBC News from late 90s early 2000s and compare the standard of writing with the red top gutter "journalism" they emit today, where everything is dumbed down and their contradictory biases are apparent - Pro-Brexit, Anti- disabled, Pro-radical feminist, Pro religion, Anti-sex, pro-nostalgia (what products from the good old days are due a comeback as an example, all of it was stuff that anyone born after the mid 60s would have only seen in the grandparents homes)
-
-
-
-
-
Sunday 14th April 2019 07:48 GMT Richard 12
Re: "Mostly wrong on both counts..."
I've done the legwork, and the only possible inaccuracy is the word "the" instead of "some of the" or "most of the" or "one of the", as there were of course several different algorithms and many different "codez", all of which were needed to get to the moon/assemble the image.
However, in this case, Katie was personally responsible for the algorithm and software which made it possible to run any of the others, so yes, she did write the thing that made it possible.
Just not the one thing you looked at, scross. Do the legwork properly next time.
-
Sunday 14th April 2019 22:49 GMT Anonymous Coward
Re: "Mostly wrong on both counts..."
Reference the other article about this on The Register right now, which includes a link to Katie Bouman's own Facebook page, which states the following:
"I'm so excited that we finally get to share what we have been working on for the past year! The image shown today is the combination of images produced by multiple methods. No one algorithm or person made this image, it required the amazing talent of a team of scientists from around the globe and years of hard work to develop the instrument, data processing, imaging methods, and analysis techniques that were necessary to pull off this seemingly impossible feat. It has been truly an honor, and I am so lucky to have had the opportunity to work with you all."
https://www.theregister.co.uk/2019/04/12/astronomer_schools_sexists/
https://www.facebook.com/photo.php?fbid=10213326021042929&set=a.10211451091290857&type=3&theater
So at least Katie has the class to try and keep the record straight, despite both mainstream and social media's attempts to try to spin things differently.
Similarly inaccurate statements were made about Margaret Hamilton's role in the Apollo program, but right off the top of my head I don't recall whether or not she tried to correct these statements. I specifically noted that the captions on Hamilton's photos usually tried to claim that the huge stack of folders (program code listings) were all her own personal work, while the articles that those photos were attached to usually pointed out (buried somewhere in the text) that this was in fact the product of a relatively large team.
Now don't get me wrong here. I don't mind folks calling out these ladies' contributions, and maybe even playing them up a bit. But I do mind them just flat-out lying about the situation, as they are so wont to do these days.
At this point I have a completely different question to ask, though: Why are we being shown the output from data collected back in 2017, when apparently there is much more current data available now?
-
Monday 15th April 2019 12:07 GMT Antron Argaiv
Re: "Mostly wrong on both counts..."
I think the object here is to show that women can make significant contributions too. And to remind us that they are under-represented, as a group, in the cutting edge workforce (those that do important and interesting things, like image black holes, design computers, and develop new medicines, etc). We need more women in science and engineering, and this serves as a reminder that we need to do more to encourage and support them in these fields (and others). It makes no sense to discourage half your potential talent pool because of their gender (or race, for that matter)
I don't think caption accuracy is tremendously important here, and Dr. Bouman seems somewhat taken aback by all the publicity, to which she has reacted, quite correctly, by pointing out that it was a team effort, and she was only part of a much larger team. I find it hard to fault her. She is an attractive, smart, young lady, working in a male-dominated field, which is why the photo is being circulated so widely by the media. Most of them are still using their lizard brains and are thinking first of what sells papers (or bits) and not too worried about the accuracy of their captions.
-
Monday 15th April 2019 15:33 GMT Anonymous Coward
Re: "Mostly wrong on both counts..."
Well, in general it's not just captions but also clickbait headlines and such. A lot of people probably don't read much beyond that stuff, and I doubt that many of them read entire articles through and through, where (as I said) the truth of such matters is often buried down somewhere near the middle or bottom. (I confess that I don't throughly read many articles today, either; rather I often just skim them looking for exactly these types of situations.)
As for Margaret Hamilton (and since social media is all the rage these days), compare these two things: A recent tweet from MIT (emphasis mine) vs. a tweet from the BBC. One is far more accurate than the other. You'd think that a place like MIT would at least try and get this right, and certainly her work is old enough now that there really should be no questions about it. But as I said earlier, I don't recall whether she ever tried to correct the record here; I'm guessing not.
MIT - Right: MIT computer scientist Margaret Hamilton w/the code SHE WROTE that helped put a man on the moon.
BBC - 1969: Margaret Hamilton alongside the code that got us to the moon
-
-
-
-
-
-
-
-
-
-
-
-
-
Friday 12th April 2019 13:30 GMT Anonymous Coward
Re: Bigger, better
If you can put telescopes in heliocentric orbits, and solve the data & synchronisation issues (of which I guess the data volume one might be severe), then can you get away with less sensitivity because you can make very long observations? My guess is that perhaps you can't, for this purpose, because the accretion disk may be changing significantly on timescales of a few days, so really long observations won't work.
-
Monday 15th April 2019 08:14 GMT MJB7
Re: Bigger, better
"the accretion disk may be changing significantly on timescales of a few days, so really long observations won't work."
Exactly right. This is why they couldn't image the BH at the centre of the Milky Way galaxy. It's less massive (and hence smaller radius) than the one they did image, but it's a *lot* closer, so it would have twice the apparent size. The problem is that because it's physically so much smaller, the variations happen on a time-scale of hours, and they just couldn't accumulate enough photons to get the necessary sensitivity.
-
-
-
-
-
-
-
Friday 12th April 2019 10:29 GMT Baldrickk
To most weather, yeah.
I imagine thunderstorms will add noise, and I've heard stories of nearby microwave ovens* but also people and birds on the dish causing noise too.
*iirc, the issue with the microwave wasn't in it's use - the shielding was adequate, but the physicists hungry for their lunch would open the door while it was running instead of stopping the oven first. This led to microwave leakage as the door opened before it properly cut out. Not anything to worry about normally, but the dish cared.
-
Friday 12th April 2019 12:32 GMT Anonymous Coward
From what I understand, they needed to capture the same "image" at the same time across the radio dishes. They would need to capture the difference in timing depending on the location (so the ones closer to the equator would receive the same image as the south pole dish slightly ahead). Atmospheric conditions can slow light by measurable degrees, so you aren't able to identify the timing accurately.
Plus the wider weather aspect was they couldn't ship out the south pole data until winter had passed.
-
-
Friday 12th April 2019 10:36 GMT Tom 7
Re: He filled hard disks
On the TV program I saw on the evening of the release they said that it was incredibly difficult to back up the drives. If you consider that a drive may fail on downloading the data then its as likely to fail when making a backup so that gets you nowhere* and costs a shitload more**.
*unless you have an MBA in the loop who (by definition) doesnt understand the process.
** and the backup disk money would have been spent on the MBA anyway.
-
Friday 12th April 2019 10:46 GMT ibmalone
Re: He filled hard disks
I'm not sure that's entirely true, the drives are being moved, so backing them up prior to that would escape any damage caused in transit. Also simultaneously writing to two sets of drives would avoid the fail-on-read problem (and also any fail on write issues, well, not avoid, but do (1-p)^2 on the failure rate). But with this amount of data some of it will be missing or corrupted anyway, so their algorithm for combining it likely has to deal with the missing observations and a corrupt hard-disc in petabytes of data may well just be another source of noise.
-
Friday 12th April 2019 10:48 GMT Doctor Syntax
Re: He filled hard disks
"the backup disk money"
Tape?
It was probably a financial decision. The bigger risk would have been loss in transit rather than H/W failure during transcription. Anything from rough handling via over-zealous customs official to a crash, and not forgetting the packages that disappear into a warehouse and are never found again due to theft or incompetence.
-
-
This post has been deleted by its author
-
-
-
-
Friday 12th April 2019 12:40 GMT Yet Another Anonymous coward
Re: Helium filled at high altitude.
It's was a pain keeping equipement working on Hawaii at 14,500ft -can't imagine how hard it is for ALMA.
CPU fans don't work because there is only half as much air as they expect, we tried sealing cabinets and pressurising them but its expensive, tricky to get cables in/out and where does the heat go ?
This was before water cooling was easily available. We generally just used slightly lower than cutting edge machines, speed wise, and put extra case fans everywhere we could
-
-
Friday 12th April 2019 18:08 GMT Yet Another Anonymous coward
Re: Not so high
>The operations Support Facility (OSF) where the primary observations are done and where the data are stored, is located at 2900 meters.
But the support controls, power drives and all the detector and front end electronics needs to be on the telescope - in a place where even plastics break down in months. The UV levels were causing laminated windshields in cars to fog.
-
-
This post has been deleted by its author
-
-
-
-
Friday 12th April 2019 18:04 GMT Yet Another Anonymous coward
Re: Helium filled at high altitude.
> fans are commodity components and readily available at higher speeds?
On your home made PC maybe, on a Sun workstation in the 90s they weren't, and every failure had to be shipped back to the mainland for repair.
Plus a fan running at higher speed in lower pressure doesn't necessarily generate the same air flow - it tends to just burn out it's own motor - especially before they were speed controlled brushlless DC motors like today.
-
-
-
-
Friday 12th April 2019 12:36 GMT Annihilator
Re: Helium filled at high altitude.
An untethered helium drive will naturally drift up to higher altitudes over time anyway, so I suspect that they had an abundance of drives that had piled up against the radio dishes, having long since drifted away from an inattentive child who was carrying it around on the end of a string but sadly let go.
-
Friday 12th April 2019 12:55 GMT Andy The Hat
Re: Helium filled at high altitude.
When my mate was working for GRO, I believe Canary Islands scopes (at around 2200m) used standard drives. I'm not sure what Mauna Kea have done b.h. (before helium) but they're at 4200m and I'm almost certain they had standard pcs with drives on the desk ... Perhaps it's a potentially high failure rate issue with low flying heads in modern drives rather than drives not working at all?
-
Friday 12th April 2019 16:02 GMT Mystic Megabyte
Re: Helium filled at high altitude. @Annihilator
Firstly; LOL!
I once spent a day selling Helium balloons at an exhibition (please don't ask!). I must have done the squeaky voice thing a hundred times that day. And yes you were correct, the ~20mtr high ceiling was populated with loads of balloons. They must employ someone with an air rifle to get them down.
-
-
Friday 12th April 2019 13:37 GMT O RLY
BBC stole that tweet about the two scientists without correct attribution.
Here is the original tweet from MIT CSAIL:
-
Friday 12th April 2019 13:41 GMT John Savard
Tape
Using tape instead of hard drives to ship the data sounds like a good idea. But I suppose they could have had reasons. All the sites would have to be using the same kind of high-capacity tape drive. Plus, isn't tape capacity lagging behind hard drive capacity these days? And the tapes would only be useful for shipping data, whereas the hard drives could always be used later as more hard drive capacity.
Still, for many of us, using the helium drives to only ship the data as far as down the mountain to the nearest city would make sense - as it might be easier to get tapes instead of disks through customs for ordinary people like you and I, and we wouldn't have petabytes to ship. Of course, the site in Antarctica didn't have that option.
-
Friday 12th April 2019 14:55 GMT ibmalone
Re: Tape
Tape capacity is catching up again, and will probably pass HD soon as HD are at the limits of what's physically possible while tape spent a long time not being developed much, LTO-8 can take 12TB, though in 2017 they'd have likely had to go with LTO-7 and 6TB, or one of the proprietary formats which can take more. Bandwidth is comparable, though not as good, about 350MB/s (uncompressed, tape figures often quote compressed due to use as backup), vs about twice that for 12TB helium drives. Drives are expensive, but individual tapes are a lot cheaper than really big HDs of the same size.
However I'd guess the need for a drive makes it less attractive for this application where you want to be able to maintain high bandwidth and continuous writing (also there's the wraps thing which I don't know how it plays out in practice, the wraps are written in alternating directions, but not sure if there's a delay at the change-over). Likely cheaper overall to have an array of HDs than an array of tape drives. That 350TB/day is about 4GB/s, so you'd need over 10 tape drives with two changes (three tapes) a day with LTO-8 to keep up (probably more, since you'd only just keep up), while about 30*12TB HDs will provide the capacity and easily have the bandwidth. A 12TB helium drive is about £400, a basic LTO-8 drive is about £2000 (tapes for it are ~£60), so the break-even cost-wise is about 6 tapes to one drive, which obviously has much less bandwidth. (Doesn't account for the extra hardware to host the HDs though, which will start to balance it out once you've got more drives than can connect to a single server.)
-
Friday 12th April 2019 18:44 GMT Yet Another Anonymous coward
Re: Tape
You don't need to keep the raw signal data for very long once you have analysed it. Some of the intermediate results are worth archiving for future improvements.
The problem with tape is the time to read/write. The disk storage system is sized to barely keep up with the data rate, so having enough spare capacity to buffer it so that it could be written to tape at the same rate - together with the number of tape drives you would need - is expensive.
Then you have the same problem at the processing site. The processing for these signals is read the data once into the correlator and then the drives can be sent back for reuse.
Funnily enough the detectors seem to have kept the same rate of improvement as the drives. 20 years ago we shipped 6Gb scsi drives home with a few nights data, now they ship racks of 8Tb drives.
For real data firehose nightmare - look at the Square Kilometer Array !
-
Saturday 13th April 2019 20:21 GMT ibmalone
Re: Tape
This is why I was looking at the data rate and talking about multiple tape drives, you can actually get decent bandwidth. Back of the envelope a tape drive has about half the write bandwidth of an HD (is there a longer write latency? anyone care to chip in?), compression can bring it up, but I'm guessing raw observation data is not very compressible. From the description they've had to do work to make sure the data is being distributed across the discs, it's certainly more than one or two could handle (while you could RAID-0 it instead that's unattractive because one missing disc would corrupt all the measurements), but it looks like the resulting bandwidth of the system is more than they need. There's two things determining the size of the disc array, the total capacity needed and the bandwidth needed, the difference with HD is adding capacity automatically adds bandwidth, with tapes you need to increase the number of more expensive drives.
Correction to what I posted previously though, the bandwidth is 8GB/s (64Gb/s from the article) I've just realised why going from 350TB/day to 4.4GB/s is missing a factor of two... The 32 drives in one of those DBE should be able to handle something like 2-3 times that (may well be bottlenecks elsewhere), but it changes the numbers for my back of the envelope for tape as that's based on the minimum number of drives you can get away with, 24 would only just manage on paper, so 32 tape drives should have the bandwidth, but would cost more than the 128 HD for a site. Hadn't accounted for the read at the other end, though you don't need the same bandwidth there. The cost calculation may change for longer observation periods at inaccessible sites; the longer you need to keep the raw data before you can ship it the more storage you need, while your bandwidth requirements stay the same.
By bizarre coincidence, the SKA was one of the projects I applied to do a PhD on, but didn't get that project (maybe just as well, hadn't realised they still haven't built it yet!).
-
-
-
Friday 12th April 2019 15:07 GMT bodi_thung
The usual exaggeration
"locate an orange on the surface of the Moon from Earth"...
You could presumably "see" an orange on the moon but find one? Not likely - as magnification increases, the field of view decreases. This would be like trying to find a needle in a haystack, in the dark, with a laser for a flashlight.
-
Friday 12th April 2019 16:40 GMT Deryk Barker
Careless?
Apparently somebody involved with this article cannot read.
The image from the BBC clearly says that the large pile of fanfold listings is the *code* that got the Apollo astronauts to the moon, whereas the HDDs contain the *data* for the black hole image.
Comparing apples and elephants.
-
Saturday 13th April 2019 04:00 GMT Uncle Ron
More?
I may sound like an old coot here, but please tell me that there is more benefit to this huge effort than just a fuzzy picture of a bagel that appears to be getting too toasted on one side. I'm not a physicist or a mathematician or even a photographer, but I'd like to see bullet points as to what is being learned from whatever else this Mount Everest of data shows. Like, why do they keep referring to it as a "shadow" and not an actual view into the gaping maw of eternity itself. What's going on here?
-
Saturday 13th April 2019 21:50 GMT ibmalone
Re: More?
It's a good question. The data is going to be mined for years I'm sure, to test different theories on black holes and accretion discs. For one thing, just the shape of that halo confirms theories about how accretion discs behave around black holes, before we just had models, now we know they are right (sometimes), it also allows confirmation of the mass of the black hole, again this was previously only known indirectly, so we can be more confident about those methods. One thing that's gone largely unmentioned, they've got images from four days (nights?), showing the structure is stable over that time period. The ring structure confirms a lot of things about gravitational lensing and how light behaves near the event horizon.
The horse's mouth is a good place to start: https://eventhorizontelescope.org/. If you really want to get into the details, the first few papers are at IoP (open access). Papers V and VI are the most interesting for what they can actually tell from these images.
Like, why do they keep referring to it as a "shadow" and not an actual view into the gaping maw of eternity itself..
In the simple sense we can never see into a black hole, because no light can escape. There is the phenomenon of Hawking radiation, but it's produced outside the event horizon. So all you can see (if that's the right word), is nothing, a shadow. There's slightly more to it though, the shadow, the region from which you don't see light, is distorted by lensing effects and is bigger than the horizon, if the black hole was rotating significantly it would be noticeably oblate. Slightly disappointingly, M87 doesn't spin much, but they can at least confirm its shadow looks as expected.
But the simplest bullet point? Ever since they were first hypothesized, people have thought black holes couldn't exist for one reason or another, it's only really in the last three decades they became widely accepted, and now you're looking at one. (Quasars were discovered about 1960, but it was a long time before black holes were accepted as a possible explanation.)
-
Tuesday 16th April 2019 00:21 GMT ibmalone
Re: More?
An important point I missed (if you're feeling generous you can say 'lensing' covered it): the bright ring is due to accretion, but more fundamentally it's the exterior of the photon sphere https://astronomy.stackexchange.com/questions/30317/m87-black-hole-why-can-we-see-the-blackness/30360#30360. That is to say, we are actually looking at the effect of light orbiting a black hole, basically confirming the Schwarzschild or Kerr equation holds outside the event horizon (there's a bit in one of the papers about testing the Kerr nature of M87).
-
-
Saturday 13th April 2019 07:48 GMT Paul Johnson 1
1/3 mm total error
For me the most stunning thing is the required accuracy. The observations were taken at a wavelength of 1.3mm. A telescope works "perfectly" if it is accurate to 1/4 of a wavelength (the Rayleigh Limit, not to be confused with the equally important Ralyleigh Criterion which is about the aperture). So the total error in the system had to be kept down to 0.4mm across the Earth. If I mention that the moon creates tides of up to 1m in solid Earth every day, and one of the telescopes was on a moving ice sheet, you will have some idea of the practical difficulties.
Timing was equally important. 0.3mm is 1 light-picosecond, so clocks accurate to 1ps had to be used to time the incoming signals.
-
Saturday 13th April 2019 12:48 GMT elgarak1
Back when they started doing VLBI, they used video tape. Storing the data diagonally got them higher capacity, and was more error-correctible than any other storage available at the time.
I gave a seminar talk on it in the late 1990s, and back then video tape was still pretty much state-of-the-art.
-
Monday 15th April 2019 07:03 GMT Bob.
Re: I was just musing the other day that M$ might do this and low and behold!
Turing got it right [near enough] in the 40s, when he pre-supposed Moore's law and forecast Gigabit memories/storage by the end of the century.
Tape and HDDs have their quirks and amusing stories.
They first did amazing things with paper tape loops and magnetic ferrite core arrays.
And the Manchester William's Tube CRT phosphor dot thing. What was all that about?
I remember seeing an IBM 5 platter[?} removable 5MB 'cassette' at school. Our teacher brought it in from a SysAdmin friend to show us.
Tapes suffer 'Print Through' and stretching and Wow and Flutter.
Ever seen a fingerprint or speck of dust on a hard drive platter?
Probably only after the head has crashed to the surface. They don't like those, which is why they are sealed.
I remember early reports of HDDs popping their seals in unpressurised aircraft holds. Dunno if true.
The technology doesn't really matter in the end. It will always evolve.
Backup, backup, backup. Multiple copies in multiple places and the data should survive. At least for a while.
Whether you can find it in 10 years time is another matter.
-
Monday 15th April 2019 13:04 GMT IJD
Re: I was just musing the other day that M$ might do this and low and behold!
How about mercury delay line memories, as used on UK's MOSAIC computer? (which my father-in-law Bill Chandler designed and built with Allen Coombs after Colossus). Five tons of mercury in a hundred steel tubes, kept temperature stable to better than one degree...
https://mraths.org.uk/?page_id=582
-
-
Monday 15th April 2019 09:34 GMT Milton
One optimist, one cynic ...
One optimist, one cynic ... are combined, somewhat unprepossessingly, in this tired old carcass.
The optimist is delighted by this achievement. The world seems to be in full-on 'Lunatics are taking over the asylum mode', and this amazing effort is a grand antidote. It's good to be reminded of what the best of humanity can achieve.
The cynic, on the other hand, observes that the world is full of dumb-shit politicians, liars, hypocrites and soulless greedmongers making life worse for everyone except themselves; and that the only people doing constructive, worthwhile, decent, positive things for the species are the generally humble, badly-rewarded ones: teachers, nurses, artists, writers and above all scientists.
While the teacher who educates your child has to visit a food bank (maybe because she buys lunch for some of the most deprived kids from her own pocket) your useless, evasive, podgy, halfwitted, gob-on-a-stick Member of Parliament is bagging a £250k profit selling a house the taxpayers paid for.
I am a believer in the importance of every human life, but I cannot pretend that some humans appear to be way better for our species than those who seem to be ... just a waste of oxygen. I've never forgotten the independent analysis done after the 2008 global financial crisis, comparing the social usefulness of top bankers with the people who cleaned their offices, taking into account their netted financial contributions and costs to society. The minimum-wage cleaners came out a clear winner; the bankers balanced out as highly costly parasites.
If you asked me whether I'd sacrifice the entire contents of both houses of Congress in the US, and the Commons and the Lords in the UK, to save the life of a single Einstein-calibre scientist (let's say, a Fields or Nobel nominee)—hell, I'd really have to think about that. The difference, by both character and intellect, is a grim, depressingly vast gulf.
And I can't help feeling that a species that hasn't the vision, wit or fortitude to recognise and rid itself of an existentially damaging parasitic infestation, in the shape of its most selfish, dishonest and foolish members, may deserve the fate that those people are bringing upon it.
Yep: cynic.