* Posts by David Norfolk

100 publicly visible posts • joined 29 Mar 2006


Service Design - ITIL v3

David Norfolk
IT Angle

Don't bother with metrics?

Yes, there are a lot of intangibles, but that doesn't mean that pursuing *appropriate* metrics isn't worthwhile...

Have you noticed that IT has a reputation for delivering Late, Over Budget and Wrong? As it happens, I believe that this is often unfair to IT, and that IT is often blamed when the business doesn't really know what its own business process is - but, without metrics, how do you prove this?

And, in its unmanaged, chaotic way, craft-IT does often build systems "right" (until they get maintained); the trouble is, they are not always the "right systems" when the business starts to use them... The IT status quo isn't held in universal respect - outside of IT.

Comments are disabled

David Norfolk

Disabling comments?

In general, I don't censor comments. Amanfrommars does get rejected quite often, if he makes no sense at all, but I confess that I have a certain tolerance for the surreal...

How can you possibly test modern software fully?

David Norfolk


Now corrected (I hope). You obviously can't rely on inspection for QA...

How to counter premature optimisation

David Norfolk

Not optimising doesn't mean doing it wrong....

True, if premature optimisation is bad, so is careless, bloated programming – as Dave Jewell’s “related article” points out. But it’s not a binary option – most professional programmers follow “good practice” and write good quality, reasonably efficient code (I hope).

Bad programming is NOT the only alternative to premature optimisation.

Data modelling layers: do you wanna get logical or physical

David Norfolk


I know what Jason means when he says "I'm uncomfortable with the implication that 'data modeling' implies a RDBMS and E/R tool". He makes some interesting comments and I'm pretty much in agreement.

But in the context of the article, only the first part worries me a bit. The second part doesn't so much. I've used E/R modelling to help me build (hierarchical) IMS databases. I see no reason to believe that it wouldn't be useful when designing (hierarchical) XML databases or even understanding the data in an object. I think the reader must distinguish between the logical properties of the data (including, but not limited to, E-R properties) and the physical implementation in a so-called RDBMS.

The first part is trickier. Most commercial databases claim to be RDBMS (often with not very much justifification - see Chris Date) and I imagine that most Embarcadero customers use ER tools for RDBMS. So perhaps the article does rather assume RDBMS. But this is realistic, and it does mention XML alternatives.

But I think that E-R modelling is an important part of data modelling (or, rather, data analysis) and that data analysis is an important precursor to any kind of application design. The point of RDBMS is that it should help you implement your data analysis more directly. I don't think Donna's views, as reported, are inconsistent with any of this.

What's the future of the UK IT industry?

David Norfolk

Too pessimistic?

The Government is blind? Quite possibly; but that pessimistic view isn't really supported by the report I was commenting on (feel free to disagree with it, of course).

Off-shoring is no panacea - managing a remote workforce is harder than managing a local one and a successful user of outsourced services needs a certain organisational maturity. Firms doing it well seem to concentrate on a mixture of local and global resources.

However, using the best, most qualified, people for the job, regardless of where they happen to live, has a certain attraction. And English workers seem to be competing much better as "knowledge workers" than they do as plumbers, say, these days.

Teradata Universe in Warsaw

David Norfolk

Fancy new product?

Teradata has been around for almost as long as I've been in IT (I think Teradata started in 1979); I'm surprised our commenter hasn't heard of it before.

It's sold as a datawarehouse and scalabilty is at least as good as, probably better than, mainframe DB/2 and Oracle. Price? You probably need your "clueless exec" and his/her checkbook; it targets the biggest companies/applications in the world. It's not really playing in the MySQL mass-market space but I suspect that Teradata experts drive very nice cars.

More information on its website; I guess the interesting competition is appliance or packaged datawarehouse from, eg, Netezza, DATAllegro and Kalido.

The Economist de-rails Microsoft media love in

David Norfolk

I always did respect the Economist!

These "heretical" remarks make a lot of sense to me - although I have been involved in a paper magazine with great (and independent) content which couldn't make any money from it....

Specialist technology magazines on paper are struggling today and online content (obviously) has its place - if it isn't "advertorial" and if it's fun to read - but I predict a return to paper magazines as premium, paid-for, adjuncts to online info sources in a year or so!

Paper will probably be marketed as "new technology" by then, of course.

Adding mobility with Antenna Software

David Norfolk

CRM system changed without users noticing

I said in the article that someone had done this.

Well, Gregg Plekan (SVP Product Development at Antenna) tells me that it was EMC:

“For us, EMC is the perfect example of why a Service Oriented Architecture and Web Services approach to mobility is so critical in today’s business environment,” he says. "In one weekend, the company needed to migrate from its previous CRM system to a new one and do so without disrupting its business operations and employee productivity. However, since EMC had been using the Antenna Mobility Platform -- in an on-demand, hosted environment -- the migration was transparent to end-users, who likely had no idea that the plumbing on their mobile CRM solution had been completely replaced. In other words, the user interface looked the same as always, but the pipes to the back-end systems were brand new.”

It's a nice trick if you can manage it, I guess!

Mainframe now obsolete for data warehousing

David Norfolk

Why do we post comments we may not agree with?

I think the mainframe still exists - although I also wonder if the distinction is worth making. A mainframe is simply a computer using an architecture that doesn't change every month; and which handles multiprocessors and I/O properly.

I remember my first job in journalism after I'd walked out of a bank - the news editor refused to believe that mainframes still existed, despite me being sure I'd been using one a few days before.

But we don't post comments to attract mockery. We post them to invite discussion (as has happened in this case). We don't censor comments generally - except when they don't make any sort of sense; are dangerously wrong or spamming irrelevant URLs; or when they libel someone...

Microsoft calls on UK public to raise the Office standard

David Norfolk

Take two petitions into the shower...

As a conspiracy theorist, I assume there are two petitions in order to halve the number of opposition signatures people have to quote in the press releases...

Spreadsheet security? What spreadsheet security!

David Norfolk

Spreadsheet errors are an integrity issue

I think spreadsheet errors are a very serious problem - see http://panko.cba.hawaii.edu/ssr/.

But I also think they are also a "security" issue - Security is Confidentiality, Integrity and Availability, not just Confidentiality. OK, I know that coding errors aren't usually considered security risks (although the fixes often are) but I think that our habit of treating security-related defects as something different to system defects in general tends to reduce security - things fall into the gap between the security assessment and normal testing...

New vulnerability strikes heart of Web 2.0

David Norfolk

A couple of points...

I'll point out that I did reference Grossman in my article but thanks for the extra reference, it's interesting corroboration. I think the Fortify advisory cites the GMail example too and the comments in Grossman's blog reference Fortify. But Brian Chess's advisory adds to what's there, to my mind.

I'll also point out that the availability of tools to make applications secure is not the issue. Windows has lots of tools to make it secure and some people even write secure Windows code. Does that mean that Windows security is a "done thing"?

FUD is a danger. But so is complacency - and security awareness is always good (back when I was paid to do this stuff for real, I rated security awareness training higher than security technologies, for delivering real security).

Oh, and BTW, this isn't Reg - it's Register Developer...

CMMI, practically speaking

David Norfolk

You don't have to have a mature development process...

"Only bean counters think that coding is a science that can be reduced to a process of any kind."

Well, if you really think that software development is a process-free area, can I recommend an El Reg teeshirt: http://www.cashncarrion.co.uk/products/16262/0/

The trouble is that those beancounters are employed by the people who pay programmers; and these paymasters expect predictable deliveery of working software that meets a business need - and seem unhappy about our ability to do this in practice.

But CMMI doesn't say that you must have process or that Maturity Level 1 (ML 1) organisations must fail. It just says that ML 1 is high risk. Probably the best example of a (very) successful low-maturity development organisation is Microsoft in the 1990s. Microsoft now seems to be interested in increasing its development process maturity, however.

Fortran developer John Backus dies

David Norfolk


Thanks for that link to Backus the man... My memory of IBM techies generally, even in the 1980s, is that they were nowhere near as stuffy as the "grey suits" legends would suggest. In IBM, technical excellence excused a lot!

Fortify and the Java open review project

David Norfolk

Yes, Java is a popular OSS language...

Matt, I must admit that I always take these "my code base is bigger than your code base" stats with a pinch of salt. It often depends on scope, on what code you reckon is important enough to count and how you count it, and I'm not sure how I'd go about coming up with anything definitive myself. Which is why I was vague on the exact numbers - and I don't think that the popularity of Java vs. C++ was the main point of the article anyway.

However, the source is http://www.google.com/Top/Computers/Programming/Languages/Open_Source/, if that helps. I think that this provides some metric for popularity, but I agree it doesn't necessarily equate to "most used". However, according to Fortify, it's very difficult to figure out how many OSS packages there are of any 'flavour' (SourceForge, I'm told, doesn't list major packages such as Apache, MySQL or Linux).

To put "popularity" in context, at the end of 2005 an Ovum analyst said "We estimate that there are over 200 billion lines of COBOL in production today, and this number continues to grow by between three and five percent a year” . Would that make COBOL a "popular" computing language?

As for what is an OSS language, Java and PHP feature in that Google directory reference bur COBOL doesn't. Is there a lot of OSS software written in COBOL? Probably not (but I'm now waiting for something like IBM to point out that there's a major COBOL OSS project on Aphaworks).

Microsoft offers Architects a view

David Norfolk

Why not OOXML - well it's a horrid name...

Very misleading name, potentially. I must say I share your concerns about Open XML, especially if it is some 6000 pages. And some of the issues with date formats and country codes in the references you cite horrify me - as an ex DBA/data analyst, I see tears before bedtime if people actually try to use this "standard", especially in a non-MS environment.

But I haven't read and analysed the whole 6000 pages - I do wonder if all the people I saw filling the Microsoft petition have.

What I have done, is to commission an article from somebody competent, looking at this issue dispassionately....

Putting the graphic artist in the loop

David Norfolk

Fair point, but

I did try to imply, by mentioning ILOG, that this wasn't an MS invention.

I too can remember separating business logic, data access and presentation layer (as I'd probably put it) in the 1980s.

But I don't remember having a graphics artist in the loop then and I'm not sure what s/he could have done if s/he had been....

Blog: End-user experience with Citrix

David Norfolk

It's a blog

As it's a blog posting, it reflects my honestly-held personal opinion, Andy.

What do you disagree with? With measuring end-user experience? With the idea that EdgeSight does this? With the idea that Citrix sells EdgeSight? Or the idea that Citrix has repackaged its technologies into its latest release? Or, do you just dislike Citrix?

I think you should be more explicit!

Lights, camera, action

David Norfolk

Style challenge

"with Mr Clarkson’s jalopy you get to travel with a bit more style"

Sorry, "Clarkson"... "style"...

I'm having real trouble getting my head round that sentence...

Are founder-driven SW vendors good for enterprise buyers?

David Norfolk

Customer focus rather than press focus?

Well, I certainly follow Information Builders (http://www.regdeveloper.co.uk/2007/02/11/search_webfocus_magnify/, for example) and I know that Martin Banks follows SAS. We don't write about them constantly because there are a lot of other firms out there to write about too!

But I'd certainly agree with the thrust of this article. One reason these privately-held companies don't feature so much in the press, is that they can concentrate on customers (I was at an Info Builders summit a year or so back, and its customers are certainly both loyal and happy) and longterm strategy - instead of "wasting time" on PR.

While publically-quoted companies have to manufacture press interest in order to keep their share prices going in the right direction - and if journalists are deluged with Microsoft "news" it can be hard to just ignore all of it.

However, there's a risk associated with having a charasmatic leader, even if they're as bright as Gerry Cohen and Jim Goodnight, as they have been known to lose touch with where things are going (again, remember Ken Olsen of DEC and the PC).

But, in my book, there's an even bigger risk associated with making short term fashion-driven decisions in order to move the share price up, at the behest of venture capitalists.

For me, then, private ownership is a good sign, all else being equal. A business doesn't want a major technology supplier changing its technology strategy just to keep some know-nothing venture capitalists happy!

Vista: fit for purpose...

David Norfolk

Open Source

Some people have taken me to task for being unkind to altruistic Open Source Software (OSS) developers providing free software in their spare time - as I'm apparently expecting them to provide a "fitness for purpose" warranty too.

Well, no, I'm not. And there's a comment in here as to why the CDDL might have that disclaimer (although I'm still not happy with it). However, although my primary target was conventional vendor software - I would include also any OSS used in a commercial business.

I should have made it clear that if you take on OSS for free and don't take on a support contract, you can hardly expect a "fitness for purpose" warranty - sorry.

But, then, you probably shouldn't use software like that in any sort of business critical systems where you'd need to invoke a "fitness for purpose" warranty - unless you accept the whole risk of supporting the software in-house, which avoids the issue.

If you pay for support for industrial strength OSS like JBoss, I still think that you should expect it to be "fit for purpose". Of course, defining what "fit for purpose" means in detail won't be trivial, but I'm sure that software companies could cope....

David Norfolk

Goods or services

I am most definitely not a lawyer but I think that Vista will be supplied as a service not a good and that the "fitness for purpose" provisions of that Act won't apply.

Now, I'm not at all sure that that would stand up in court but I don't think it has ever beem tested in UK or European law (no doubt someone will tell me if I'm wrong) - and it is quite possible that MS won't want to test it in court anyway.

But I'd hate to take on Microsoft's lawyers if they do, even if I did win in the end...

Is old code automatically good code?

David Norfolk

Not just language

Well, I rather agree with Torben - I like SPARK (http://www.regdeveloper.co.uk/2006/09/20/high_integrity_software/). But you could argue that the machine/OS architecture should distinguish between code and data - as in the AS400/iSeries - and keep users isolated from system functions in separate address spaces - as in z/OS mainframes.

But, for whatever reason (cost?) we are currently using less secure architectures and languages for non-safety-critical systems, in practice - and we need to get defects out of this code.

Whether we should also use more inherently secure systems and languages (and I think that Moore's Law says we can probably afford to do so more often these days) is another article....

EJB 3.0 Based Web Services

David Norfolk


for those corrections. Don't know how JBoss and JBos got used so inconsistently, my fault for subbing late at night...

Is Solaris really a bright choice for developers?

David Norfolk


A reader has suggested Blastwave as a good source of Solaris freeware:


Or, on good ol' Wikipedia:


David Norfolk


I've been taken to task about my thow-away remark about possible (I emphasise) "gotchas" in the GPL - by people that would like it thrown away. Well, fair enough - I think that some of the concerns I alluded to are overstated (and possibly have been exploited by over-interested parties - see http://www.theregister.co.uk/2001/06/02/ballmer_linux_is_a_cancer/).

But the fact remains that there are concerns, that parts of some versions of the GPL have not always been clear and unambiguous and that, even now, there are concerns about the wording of GPL v3 (alluded to here: http://www.regdeveloper.co.uk/2006/08/18/gpl_3_torvalds/).

No developer wants to get involved with IP lawyers - but sometimes they have to (it's a continuing concern of the JCP, for example). Open Source Software is not License Free Software and the IP questions around it aren't trivial. Which is really to say that I don't want to add a detailed critique of the GPL to an article on Solaris. Which uses CDDL.

David Norfolk


One reader wrote in with another alternative: NexentaOS is a GNU-based open source operating system built on top of the OpenSolaris kernel and runtime and using Ubuntu to glue things together.

Take a look at http://www.gnusolaris.org/gswiki


David Norfolk

Missing classics?

DeMarco and Lister's "Peopleware", perhaps?

myers' "The Art of Software Testing"?

Microsoft picks IBM top brain

David Norfolk

But the point is...

The point is, surely, that IBM develops these people - while Microsoft can only pluck them, fully formed...

Well, that's going a bit far, perhaps; but it does seem that IBM has a culture that understands Enterprise computing, while Microsoft is trying to develop one.

Microsoft has plenty of brains and is acquiring the right people - but I wonder if building the "culture" needs even more than that?

Alice in Business

David Norfolk

Feedback on Nick's blog

Nick talks about the feedback he got on this article at:


The Screwpole Emails - Project Types

David Norfolk


I rather like Mr Screwpole's classification of projects.

However, I can take it personally, because I have a personal interest in building a house extension ATM. We have plenty of vision and plenty of experiencer on the teams involved and no inclination for micro-management.

All the quotes are fairly imprecise and all the workers very plausible and, if it all works we'll no doubt be very pleased that we just set things in motion and let them get on with it - and the listed building people provide a degree of 3rd party inspection to keep everyone honest.

However, there are times when I'd quite like to have a formal and detailed specification of exactly what we're getting and when. Even though I realise that this would cost more and, even then, is negotiable in the light of what actually happens when work starts.

If I was building a mission critical business system, I'd probably place an even higher value on comfort. Screwpole's manager on the "voyage of Discovery", say, has to have lots of people skills and dispense lots of comfort to all the stakeholders on the voyage - just delivering eventual suuccess may not always be enough

Malware: Windows is only part of the problem

David Norfolk

Employing hackers?

Well, I did an "ethical hacking" course once, but that wasn't about employing hackers. Some of my best friends are hackers - but their exploits were long ago, when "experimentation" was a valid excuse.

Consider this. Why should the hacker (and sorry about misusing what was once the term used for clever UNIX coders) you employ do any serious testing? You'll be happy if he tells you you have no vulnerabilities and pay him anyway...

Or this. If the hacker does discover vulnerabilites, why should he tell you about all of them? Perhaps they have a market value..

Or this. How do you know your hacker is any good? A successful hacker only has to get lucky once; but the "ethical hacker" has to produce a complete risk/vulnerabity assessment.

There are real issues in employing "reformed" hackers as security testers - you have to work out how you're sure they've reformed. As an IBM security consultant said to me once: "we usually find it easier to teach ethical systems programmers about hacking techniques than to teach hackers about ethics".

Although you do have to pick systems programmers with the right sort of "left field" mindset...

Working in the COBOL mine

David Norfolk


Inspect does indeed look like an interesting product, but when I approached Piercom in order to learn more about it, Piercom's CEO said: "Thanks for your interest but we have no interest in this unless there was a tangible commercial opportunity. If you have money we can talk"...


I wonder how it now compares with the modern tools from Micro Focus etc?

Test, test and test again

David Norfolk


> A definition of Done: The coding of an object is not done until all tests pass.

Well, yes; and that's lots better than some of the definitions used in the non-agile world ("it's done when we reach the unrealistic deadline the boss negotiated months ago, in return for his latest promotion/bonus").

But it does imply that you have a complete set of tests and how do you define that? You can't really say simply that having some tests for every interface is enough...

As someone else has pointed out, you have to go back to requirements and (I think) a requirements model that can be mathematically tested for completeness and consistency; and validated by the users in business terms,

So, while test driven development is good, requirements driven development is, ultimately, king!

David Norfolk

Cobbler's children

Ooops, yes, bad typo in the circumstances - but it is now fixed and documented (thus making the comment into an "error" or at least an anachronism).

Finnbar makes some good points - especially about error handling. And even if you have some, how thoroughly is it tested? Not as huge a problem in the old days of ACID transactions, perhaps; but I think that missing or untested "compensating transactions" in the SOA world may well be a killer...

As for his last point - yes weight of documentation is a rotten compliance metric, and "documents" poor process. It is usually easiest and cheapest to do the job properly - with automated tools that don't produce piles of paper but link tests through to requirements, possibly?

Adopt and adapt better than rip and replace...

David Norfolk

Mainframes are still here, still in use

Irony indeed. When I first worked as a journalist, my News Editor refused to believe that the bank I'd just walked out of still depended on mainframes - and was likely to do so for a long time (despite our IT director promising to migrate it all to NeXT in a year or so).

And, iirc, the first non-Microsoft compiler for NT was FORTRAN - but that's another story (and I really do prefer COBOL - or Rexx).

And, as an aside, Web html programming is a lot like mainframe 3270. But, then, the web browser model is really 3270 terminal in software... :-)

Vintage Databases

David Norfolk

Aston Tate

dBase? Well, I did mention FoxPro...

I have an issue with dBase, since, iirc, it was the first database to seriously attempt to devalue the term "relational database"....

Uncle Mac takes A Cup of Cold Cocoa...

David Norfolk


> Nexties definitely arrived at Apple in their own RDF and have shown little signs of looking outside

NeXt people were always a bit like that, IME. I once worked for the biggest purchaser of NeXT workstations in the City and NeXT was very strategic (according to the IT director, all our mainframe stuff was soon going to be running on NeXT - obviously he hadn't looked at the source code, as far as there was any, or the viability of NeXT itself).

But when I met it, NeXT technology was unable to integrate with any of the existing communications/email systems effectively, which hardly worried any of the NeXT enthusiasts. You couldn't, AIRI, actually get an email from the London back-office to someone with a NeXT workstation in London unless you knew about an undocumented little back-door in Chicago...

Access isn’t a relational database

David Norfolk


It's only a personal opinion, but I've always thought that recovery to the point of failure (ie, not taking the database back in time and not requiring manual resubmission of successfully completed transactions after failure) is an important business-oriented goal for a DBMS. So, while ACID isn't actually required for basic transaction support, I think that not offering it in a DBMS claiming to support transactions is potentially dangerous.

However, with loosly coupled composite services these days, perhaps recovery to the point of failyre is moot and we must now always assume a need for compensating transactions to undo the consequences of failure...

David Norfolk


Actually, I think that one meaning of "relational" is rather well defined - by Codd, Date et al - and is to do with set theory.

The problem is that lots of people ignore this and come up with various less precise and less useful meanings - and, as you say, often for marketing purposes. Was it Ashton Tate that decided that "relational" referred to the fact that you could relate two different files open at the same time?

Building a data warehouse on parallel lines

David Norfolk


Sorry about White Cross - but it is Kognitio now, and your comment supplies the omission.

As for Greenplum, I almost included a reference but decided not to - the article was really about Kognitio and I could have referenced many other products (probably even including Teradata). Also, Greenplum is covered somewhere in the related articles at the bottom of the piece.

Escape from Access DB

David Norfolk

Loosly coupled DB access

A reader commented via email that there's no need for pre-configuring a DSN for each database you wish to use, you can put the path to the MDB right in your connection string. This saves users from having to go into the ODBC control panel or Sysadmins from configuring the DSN on each machine, and its pretty much the only way to go if you want to choose a database from an Open dialog:

String dbUrl="jdbc:odbc:Driver={Microsoft Access Driver (*.mdb)};DBQ="+mdbFile.getCanonicalPath();

Pan, who's away from his usual PC and can't remember his Reg dev password :-), reponds:

"True enough, but there's one advantage to using the DSN... it means that if the location (or name) of the MDB file changes then it's only the DSN that needs to be updated. The Java code is loosely coupled and refers only to the DSN, so no recompilation is necessary.

The same decoupling could be achieved by using a properties file rather than hard-coding the MDB file name and location in the Java application.

So, have a good Christmas everyone - and we should really all switch off our PCs for the duration - David

Borland throws down Gauntlet to software defects

David Norfolk

Reusing "best practice"? About time!

If "everything here has been established as best practice some years back" then Borland is to be congratulated!

It sometimes seeems that the norm in IT product development is to ignore everything experienced practitioners know; and then re-invent the wheel, to produce products which aren't really usable in practice until the second or third release.

Take established good practice and make it more accessible? It'll never catch on....

Database in depth

David Norfolk

Depends what you mean by success

>it is little appreciated how much relational theory stands on top of very solid mathematical foundations, far more so than the heuristic approach of object modelling. Then again, consider the success of mathematically sound formal languages against the programming languages that have been succesful.

Well, I certainly agree with the first part of that statement. As for the second part, it really does depend on how you define success. Praxis points out that although the "mathematically sound formal language" Spark probably isn't as popular as C#, say, Praxis is still in business and still has important customers (such as the NSA) that choose Spark in preference to other languages - for safety critical (and other) projects, where defects simply aren't tolerated. That's a sort of success.

The trouble is, I haven't met a "mathematically sound formal" relational database recently, so I don't know if it would be successful. I don't even think that one has been tried in commercial practice (at least since memory and CPU constraints have been relaxed) and found wanting.

Oracle, DB2 and SQL Server are very good databases, just as IMS (which was my area of expertise once) is a very good database, but they are all more or less "heuristic" databases; and perhaps a "true" relational database with a proper optimiser would support a leap forward in system reliability and ease of programming.

SQL Server and the 7.5-day MTBF

David Norfolk

But the point...

The point is, surely, that Mark was criticising shoddy PR figures - which often don't get questioned - not commenting on the reliabilty of any databases per se.

When you are looking at real-world reliability then, certainly, a requirement for "planned downtime" in the operating system must be factored in. And MTBF often doesn't mean what you think it means anyway. Perhaps, if one server fails, the other 12 are more likely to fail at the same time; as, if the failure is due to faulty hardware, a whole batch of hard disks or chips are likely to be faulty.

And how a database fails is important. If a database fails without courupting the database in any way, you can probably tolerate more database failures than if you have to repair the database after a crash. And, yes, mainframe DB2, say, really doesn't fail often - and probably doesn't corrupt itself if it is forced to fail either - but that isn't the point of the story, as I see it.

Where do you store master data?

David Norfolk

Master Data

An Active Repository, whether Oracle or not, certainly sounds like a good place for Master Data. But I think that the real issue is defining exactly what Master Data is, and what it is used for, before deciding where to put it.

One definition I've seen is "master data is a set of data elements along with their associated attributes, such as customer, product, employee, vendor, etc." Well, in the bad old days, that was simply the customer, product etc databases in the transactional corporate database, plus (probably) bits of the Data Dictionary (aka "Repository") we drove this from.

If Master Data is something people have suddenly discovered, then I wonder how our banking systems (eg) have been working for so long (yes, I know, they use legacy COBOL and IMS databases, back from when we did this stuff properly).

So, coming up to date, if "Master Data" is always going to be treated as transactional, then a Datawarehouse is a poor place for it - keep it in a "real" database.

However if it not frequently updated, then the Datawarehouse is a reasonable place for it, as long as transactional systems can get at it as and when necessary.

And if people want to know how the Master Data changes over time, then it does probably belong in a Datawarehouse.

But there's a hidden assumption here - and I don't really see a Datawarehouse as fundamentally different from any other sort of database (it stores historical data, however, so it might get very big; and historically it has been optimised for retrieval rather than update).

Making a Datawarehouse a special kind of database is simply a result of technology limitations - there's no a priori reason why historical data and BI cubes shouldn't live in the transactional database, it's just that most current technology wouldn't cope too well with this (and DBAs, sensibly, are a conservative lot). As an aside, IMS could probably cope (to an extent - it'd need extra access functions), because you could physically partition off non-transactional data within a database - and modern transactional databases such as SQL Server are starting to catch up.

So, if Master Data is needed for transactional processing, it must be available on demand (with low latency) to the transactional systems; and if it is needed for historical analyses, it must be accessible to the BI analysis programs.

Where you put it physically depends on the characteristics of the technology you have available and the relative usage frequencies of the various access paths and their associated required latencies.

In other words, where you put Master Data isn't a generic vendor-driven decision, Oracle vs IBM, say; it's a matter of physical database design. Do we still do that?

Aspect oriented programming with Java

David Norfolk

Is exception logging better?

And here is John's reply:

Presenting a new concept and a concrete real world example of how to use that concept in a short column is always a challenge. The logging example was selected as almost everybody has had to put some form of logging into their code at some time or another - and thus it is a common reference point.

In real world applications, it is more likely that the specifics of an application will determine what an appropriate Aspect might be - hence the (very) brief section on generating Aspects. I also suspect that it is only once something has been required a number of times that its use as an Aspect may become obvious.

I find it interesting that the one use of AOP you mention does end up being logging of exceptions - and I personally think this is an excellent example of what can be done with AOP. I also believe that Spring makes quiet extensive use of AOP.

As far as logging method entry and exit goes I have found that with customers’ systems, where it can be difficult to go in and add extra logging, actually being able to turn on entry and exit information (including parameter listing) can be an extremely useful feature - although this can depend on the environment in which one is working

David Norfolk

AOP - a bad example?

Richard emailed in anonymously with this comment:

A few months ago I decided I was fed up with hearing all the hype about AOP and thought I'd better actually look into it to actually understand it. Now, what really annoyed me about trying to get started in understanding AOP is that the most prevalent example on using AOP concerns logging. In my opinion (and experience) logging is NOT a good area for using AOP. In fact, I have never seen an decent example of how to use AOP. Not to say that one doesn't exist. I'm lead to believe that Spring using AOP behind the scenes for transaction handling, etc.

I don't log entry and exit into methods. If I have a function that I'm not sure is behaving properly I will add debug statements at various points during the method. When I'm logging, it's some value that probably isn't supplied as an input parameter to the method. I'm also concerned that writing AOP-based logging statements has the knock-on effect of doubling the size of your code base.

Having said all that, I have found a suitable application for AOP-based logging in my application. I use it for logging exceptions at the DAO layer with all the information I need. AOP has been very useful in that I have been able (through Spring AOP) to write a short class that can log a method call signature and the generated stack trace. Unfortunately I don't think I can apply it elsewhere right now.

I suppose what I'm trying to say is that it would have helped me to have seen a really useful application of AOP methods to solving a problem because, to me, logging ain't it.

Migrating Access to SQL made (almost) easy

David Norfolk

Download SSMA?

Tried putting "ssma download microsoft" into Google?

Google is your friend...