AMD: Why we had to evacuate 276TB from Oracle DB to Hadoop AMD has migrated terabytes of information from an Oracle Database installation to an Apache Hadoop stack, claiming Oracle's pricey software was suffering from scaling issues. The chip maker's chief information officer, Jake Dominguez, revealed further details of the transfer in a chat with The Reg. "Within the common Oracle …
they didn't know what they were doing to begin with. Or perhaps weren't able to do the right thing to begin with and had an organically growing system that nobody seriously managed and they just let it grow until it hit a major breaking point.
Not that I'd use Hadoop to replace Oracle, rather go with something like Vertica instead especially for a small data set like 276TB.
> "276 TB is small in the database world."
>
> I'd wager that more than 99.9% of databases in existence are smaller than 276 TB
That just means there's been a proliferation of a lot of small databases as the technology has become cheaper to use. The big jobs continue to get bigger all the time and people aren't terribly impressed when you start throwing around non-trivial numbers.
Perhaps I was exaggerating (or should have said "276TB isn't overly large") but to me how many databases are smaller is less relevant than how unusual a database larger than that size is. Having consulted on a project a couple years ago where one database utilized several fully loaded VMAX arrays, perhaps I'm jaded.
Probably 99.9% of all programs are shorter than 10K lines, but that doesn't make a 10K line program large.
...they're doing data mining, which is dumb to put in a relational database. It's not that the Oracle DB failed, it's that Oracle failed to come up with practical distributed computing systems for non-relational data. I know Oracle created some distributed systems in the past but I don't think anyone made it to the end of the installation instructions.
Oracle usually shits a brick well before that mark. Depends in your query rate and how much horribly written PL/SQL you're throwing at your doddering old mainframes. I've lost count of the number of places that have come to us wanting to replace their Oracle boxes, thinking they'd pick up something like Teradata or Vertica, only to end up settling for Hadoop.
Between the completely hassle free configuration and administration, the resilience and fault tolerance, the as-good-as-free hardware and licensing and the massive toolkit you get, it's pretty much a no brainer. Especially now the major distributions come with their MPP SQL engines (Impala, Stinger etc.). Throw in Parquet and Snappy and you literally have a carbon copy of your garden variety Teradata architecture, but open source and really, really sodding cheap.
Those in traditional data warehousing should be worried.
> Please, don't forget to email corrections@thereg is you spot something odd
Couldn't you set up an online form instead of email? I would occasionally send the odd correction but I'd rather not divulge my "useful" email addresses and I really cannot be arsed to log into a throwaway one for this.
>>So let's say you have a report that takes 6 hours to run... When your boss says "and I need it yesterday!", you can say "You've got it, literally!"
Only if they get you the request before 6:00pm that day.
And typically that's when the sadists let you know there's something urgently needed right away, ime. :( >:|
never seen this limit, (going back as far as version 7.3.4) so not sure what they are talking about, there is a limit of 1000 columns in a table although that would indicate your doing something really stupid.
Possibly they have performance problems with data > 100,000 rows being returned but there is no hard limit.
I can believe it.
I can believe almost anything after watching Oracle "specialists" fail to get Oracle RAC running cleanly on a Sun/Solaris stack in a reasonable time, or to be able to say "yes, we know exactly why <very popular document manager application> hosted by our Weblogic Application Server running in Solaris 10 on a dedicated 4450 is acting as though it is running in treacle on wet tar, and here is how to fix it".
I was approached by an Oracle afterservice drone a couple of years back who asked "How do you like Weblogic on that 4450?" and I had the great pleasure of telling him we liked it so much we were now running the application in webSHPERE on an LPAR in a crowded P7 - and had seen an order of magnitude pick up in response time as a result".
The fun when doing that is to keep a straight face.
"Even if it does Oracle may end up being Wang a few decades later."
It looks as though Oracle have been firmly wanged in the short and curlies by AMD + Open Source of all people. If a sufficiently detailed white paper is published I suspect this will develop into a Tsunami of woe for Larry.
Yep, I eventually got FreeBSD used in a previous place of employment - ultimately, it's ability spoke for itself - however, in the initial stages, 'suits' weren't impressed with the word 'free' in the title.
And I''m sure we've all heard of religious American nutjobs who are against it because it 'glorifies satan' etc.
That would be Management from the 1980s. There are a lot of silly names out there that dominate the industry now. Example: Google. Puppet. Red Hat. Git. Linux. Any versions of software with "X" in them (makes it "edgy"!). Android. I mean, comeon, the computer industry is rife with quirky names. In fact, it's pretty much a tradition.
True, I am going back to the early 90's - when open source and free software weren't generally known, and there was still a big 'Nobody gets fired for buying Microsoft' ethos..
(Yes, I know the phrase originally mentioned IBM, but it was used for Microsoft in the 90's)
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
