back to article A bad day for DBAs: MIT boffins are replacing you with a mere spreadsheet

MIT boffins reckon they've cracked one of the tough nuts of usability, creating an easy-to-use SQL interface for non-database administrators. The tool, SIEUFERD, has been in gestation since at least 2013 (Oracle mentions it here), but considering the problem it addresses has been around since the 1970s, let's not be harsh …

  1. London 75

    Hmm

    I'm no longer a DBA so maybe it's changed but I bet it's brilliant right up until it's not brilliant one day and then you need the DBA back but unfortunately you've let them go and just have one grumpy sod left who actually spends most of the day in the pub as you never invite him to meetings any more.

  2. Christian Berger

    Actually in many cases

    A simple text file may be a much more efficient and simpler. Databases, or more precisely, relational databases, are made for the case when you have lots of tables that "link" together forming some sort of relation.

    However there are other applications where you want to handle data. One example I came across recently was the "Portierungsdatenbank Festnetz", which is a large table storing what operator handles what number(-prefix). It has over 100 Million datasets. Using this with a traditional database is incredibly slow, as even with indexes, looking up a data set will take a good fraction of a second. However if you simply sort it once, then sort the updates and simply mix them into your main file, you can work with it faster than your harddisk can handle the data... and since those are linear reads, not seeks, your harddisk will be at peak speed. And all that with text files you can simply edit with a text editor.

    It often makes sense to look at alternatives off the hype.

    1. 27escape

      Re: Actually in many cases

      I'm thinking that this solution is not targetted at you, but at all the others who have different requirements and abilities to solve them

    2. Charlie Clark Silver badge

      Re: Actually in many cases

      Even such single query data sets can benefit from being in a database, providing you set it up correctly which would presumably require a degree of normalisation.

      For your single use, keeping things in a flat file might suffice. Though your "simply sort" notion might not always be that simple in practice. Performance is also directly related to the sort order.

      I regularly import large data sets into a db and have to take the necessary measures to do this fast enough (drop and recreate indices). But seeing as I'm running random queries, using a db is the only sane way to do this.

      1. Christian Berger

        Re: Actually in many cases

        Well in this case there is just a single table with 3 columns, number/prefix, start date and operator. There is only one "index" that would make sense, and that's the prefix. To query that database, there already is a highly optimized pseudo in memory database (it uses mmap on a prepared read-only file). That's actually several hundred times faster than MySQL. That's actually the only part of the whole process which is not in plain text.

        The rest of the operation, essentially adding updates to that dataset, is much quicker to do with plain text. Even our first prototype, done by an inexperienced programmer can do the task in about half an hour.

        Again if you are doing random queries, or if you have multiple interconnected tables, a relational database system is right for you. If you are just storing rows and rows of numbers (e.g. temperatures, statistical data, etc...) it might be wise looking into something else. It's always better to use the right solution, not the only one you can think of.

        1. Trixr

          Re: Actually in many cases

          And in the example given in the article, that's precisely the use-case they have here. Multiple interconnected tables. Relational database.

  3. Charlie Clark Silver badge

    What's a DBA?

    I've always considered a DBA to be the person who helps the domain expert setup and tune the DB to their needs. I don't see this tool replacing them at all.

    1. werdsmith Silver badge

      Re: What's a DBA?

      DBA stands for "Does Bugger All" :)

      Actually, as an ex-DBA I read this article and straight away thought "what's this got to do with DBAs?"

      1. Anonymous Coward
        Anonymous Coward

        Re: What's a DBA?

        Yes I read the head line for this article and concluded that the author doesn't really know :

        • what a DBA does
        • the various user friendly ad-hoc access tools already available for databases
        • the history of user friendly ad-hoc access tools, the first I heard about was called COmmon Business-Oriented Language, and then another Structured Query Language (**1)

        The long and the short of this being that either: the tool is fairly easy to use in which the case the DBA (or more likely database developer) will end up writing the more complex requirements; or it isn't in which case it will spaff off another layer of SIEUFERD developers (or maybe the DBA or developers just pick this work up).

        (**1) Look at the capitalization to work out what we tend to call these user friendly tools these days, and these were first "touted" as not requiring programmers.

  4. Kevin McMurtrie Silver badge

    A = Administrator?

    Maybe you meant developers for relational databases or data mining. The new GUI won't keep the database running.

    I suspect that the GUI won't replace relational database developers either. It looks like an excellent prototyping tool, though. Keeping SQL commands in sync with evolving diagrams on a whiteboard is exactly zero fun.

  5. xeroks

    I can see this being very commercial

    Great piece of work.

    In our work we have a lot of people who use spreadsheets to accomplish things that should really be done on databases. At least part of it is being able to "see" the data as they're using it.

    Sometimes they do move it to a database, but usually only once the spreadsheet is horrifically complex

    Having a tool like this might well bring them over from the dark side sooner.

  6. Novex

    What users want...

    ...is to be able to click a button once and get their report, all formatted nicely and ready to go. They don't want to have to understand tables, fields, field types, functions, joins, primary keys, indexes, etc. That's what they (PHBs) ask developers to know and work with.

    What is true is that this could be some kind of simpler tool for developers to be asked to understand (alongside all the other ones!) The amount of Excel spreadsheet systems in use as de facto database alternatives, and the problems inherent in keeping the VBA (or whatever) code that keeps it all working properly in some kind of decent shape, tells me that developers of underlying systems are, and in some way will always be, needed.

  7. Wensleydale Cheese

    SIEUFERD

    It'll not sell well until it gets a pronounceable name.

    1. Rich 11

      Re: SIEUFERD

      Suffered?

      1. frank ly

        Re: SIEUFERD

        I'd go for a SIEGFRIED or maybe a SIGMUND backronymn. They sound helpful and reliable.

        1. Fibbles

          Re: SIEUFERD

          Sie Ferd.

          If you speak German and Norwegian I guess you could take it to mean 'they journey'. It describes what your queries will be doing through the database.

          My consultancy fee will be with your office momentarily.

          1. allthecoolshortnamesweretaken

            Re: SIEUFERD / Sie Ferd

            Not really. "Sounds a bit like", but...

            "Sie fährt" = she drives (somewhere)

            Seepferd = sea horse

            SIEGMUND: "Tell me about your mother."

            SIEGFRIED - maybe if you love the smell of databases in the morning. However, it's complicated and probably won't end well.

        2. Down not across

          Re: SIEUFERD

          I'd go for a SIEGFRIED or maybe a SIGMUND backronymn. They sound helpful and reliable.

          Definitely Freudian. You may be onto something there.

    2. The Packrat
      Go

      Re: SIEUFERD

      How about YART - "Yet Another Reporting Tool"?

  8. BigAndos

    Not really DBAs

    So it still needs a database to connect to in order to run queries? In that case DBAs will still be just as vital to keep it humming along nicely, and of course handle the query load this tool will generate.

    What it really seems to be is a halfway house between writing SQL and using Business Objects, which of course needs a developer to set up the universe first. I think it would have a fairly niche audience in large companies in that case, but might come into its own for smaller companies that can't afford a fully fledged BI team.

  9. Anonymous Coward
    Anonymous Coward

    What's really new?

    I didn't read the papers yet, but what's really new against most "query generators"? The UI may be interesting, but how good is the query generator behind? Can it generate the most performing query for a given RDBMS?

    One of the biggest issues of RDBMS and SQL is you can't think in a "procedural" oriented manner. RDBMS and SQL are set oriented. If you can select and manipulate the right sets in the right way, you achieve the best performance. Too many DB developers never learn this "art". Could a tool like this solve this issue?

    Then there are the underlying DBMS "nuisances" - structuring (and maintaining) the database correctly for the data set while using the right database engine features require a deep knowledge of a given product. And a true DBA task is usually this one - fixing people queries is not the main one.

    A few years ago a colleague of mine tried to order a huge machine (lots of GBs of RAM, many cores) because his "critical" queries on a dataset of his were running to slow. It turned out his database structure was dreadful (only varchar fields, missing indexes, lots of duplicated data, etc. etc.). Once properly fixed, everything run in a few tens of seconds on his laptop....

    1. kmac499

      Re: What's really new?

      So maybe there's a market for a simple tool to spot design flaws in Databases.

      Personally I would settle for some consistency in SQL syntax. I just cannot get my head around what is allowable and in what order Why oh why is the syntax for update and insert different?, especially for single row modification..

      Most of the time SQL is more like an adventure game where you have to bribe the dwarf and get the key from under the drawbridge to get the Gold; not to mention the dreaded "error near" message.

  10. John Smith 19 Gold badge
    Unhappy

    TL:DR Guy who does not use spreadsheets "invents" wizard that does nested queries.

    I don't have the time to wade through the full 16 pages that AI reports usually run that seems to be the core of it.

    The big thing seems to be that they demonstrate that you can do everything in the UI that you can do raw SQL in terms of logic and set manipulation.

    IOW it's the MS Access SQL wizard done completely instead a bodge job of the most common tasks. :(

    And note this is all about questioning a DB.

    It has nothing to do with structuring data within one, or paraphrasing the query so it asks what you think it asked.A classic case would be the library query "Give me everything by Smith and Jones," which most people would actually want as author = Smith || Jones ||Smith && Jones.

    Underwhelmed in every way, despite the epic amount of grunt work this probably took to get working.

    1. Doctor Syntax Silver badge

      Re: TL:DR Guy who does not use spreadsheets "invents" wizard that does nested queries.

      "It has nothing to do with structuring data within one"

      Maybe what they should have worked on is a spreadsheet-like interface to allow the user to design the database. Something that would quietly turn those spreadsheet-used-as-database monsters into real databases.

      1. John Smith 19 Gold badge

        Maybe.. a spreadsheet-like interface to allow the user to design the database.

        Excellent idea.

        But that's really hard. :(

        Personally I'd like to start by English language parsing of the description because if the parse fails due to ambiguity it means the user does not understand their problem well enough in the first place.

        Incidentally deterministic parsing of English is possible and has been so for the last 30 odd years. Search "parsifal" for an example, while a PhD at Edingburgh demonstrated that a 2 word lookahead and evaluation stack was adequate to get the job done.

  11. LeoP

    More automatically generated SQL

    Is surely what we need. This will lead to performance problems, that we then need a DBA to diagnose and fix. A real DBA of course. He could even be named Siegfried.

    1. Brewster's Angle Grinder Silver badge

      Re: More automatically generated SQL

      That's why we all still program assembly: computers can't be trusted to generate code efficiently.

      (Where's the can of worms icon?)

  12. BebopWeBop
    Unhappy

    Spreadsheets: doom through hidden complexity

    Complex spreadsheets - with their hidden assumptions (as anyone who has tried to fix a legacy spreadsheet which is behaving in a 'way the user does not expect') are well known to be adequate write once programs, but are extremely difficult to maintain and validate.

    Is this just going to do the same for applications that misuse this type of interface to generate more uncertainty?

  13. Anonymous Coward
    Anonymous Coward

    Throwing complexity at complexity?

    Maybe I'm wrong but at skim level this looks like something that would take so much effort to get your head around, you're halfway there already and might as well just learn SQL in the first place?

  14. Wommit

    I can't see any reason for this reporting tool to upset any DBA at all.I mean even MS Access had a similar tool, and DBAs still worked on real DBs.

    DBAs and DB designers (often the same person) will always be needed to design and optimise a DB. And all of the other administrative tasks that need doing.

    Perhaps this tool will give more DBAs employment fixing the damn stupid stuff that the users built (in the looses possible sense of the word) so that it can perform provably correct and reliably.

    Still what do I care? A few more months and I'll be a lazy arsed retiree.

  15. Scott 53

    "took the covers off there Schema-Independent End-User Front-End "

    If you can't manage English properly you are bound to struggle with SQL.

  16. Sirius Lee

    Limited use cases

    It's hard to tell from the article or video how functional the tool is (maybe the answers to my questions are in articles at the end of the links provided). Most queries I create have to transform some of the values into columns - for example period based data. It's not clear that it's possible to do this using the tool.

    It's easy to assert A is better/faster/cheaper/easier than B if the measurement criteria favour what A does well. Spreadsheets allow users to create presentations that are much more than just query engines.

    How does this tool allow a user to create a 'lag' function since SQL-92 does not include this concept but is essential in any financial reporting. OLAP query engines were created to address these requirements.

    How does this tool compare with PowerBI? Access is nearly 3 decades old and Microsoft has many other query tools that do a better job of allowing a user to query a data source.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like