back to article D-Wave disputes benchmark study showing sluggish quantum computer

Quantum computing device manufacturer D-Wave is disputing a recently published study that claims the Canadian firm's systems aren't reliably faster than more-conventional computing systems. On Thursday an international team of computer scientists published a widely previewed paper in Science detailing a series of benchmark …


This topic is closed for new posts.
  1. ROlsen

    I think I know what's up

    The computer is simultaneously fast and slow

    1. stizzleswick

      Re: I think I know what's up

      I just hope it's not Schrödinger's computer.

      1. Tubs

        Re: I think I know what's up


        Maybe that's the problem: it can't do the calculations while you're looking at it.

    2. Gannon (J.) Dick

      Re: I think I know what's up

      I wanted to simultaneously vote this up and down but obviously El Reg's website is under the influence of superposition (rather early in the day isn't it guys ?)

    3. Anonymous Coward
      Anonymous Coward

      Re: I think I know what's up

      The New Uncertainty Principle, does it work or not, damn it?

      D-Wave should be permitted to define their own test, the 'Catch Me If You Can' problem. But they need to be at least 10x faster than anyone else, otherwise FAIL. That would more or less prove that it's working.

      Unless they're found cheating with a custom FPGA to solve that one problem. Too easy to be caught, so not likely.

      1. ShortLegs

        Re: I think I know what's up

        "The New Uncertainty Principle, does it work or not, damn it?"

        I'm not sure

        Hat, coat, taxi

      2. Anonymous Coward
        Anonymous Coward

        Re: I think I know what's up

        D-Wave should be permitted to define their own test, the 'Catch Me If You Can' problem. But they need to be at least 10x faster than anyone else, otherwise FAIL. That would more or less prove that it's working.

        actually, I thought they did; when they announced the two I recall it was something like 3000x faster than the digital computer in their testing. and this is hardly the first time their claims have been disputed: some swiss researchers found similar results a year ago and they were testing with simulated annealing algorithms (which the d-wave is supposedly optimized for).

  2. Destroy All Monsters Silver badge

    A bicycle with a Ferrari exterior?

    Google also issued its own response to the Troyer paper, noting that in its tests the firm is getting computational speeds of over 35,500 times that of conventional computing systems.

    Yeah, but is that speedup outside of the error ranges? And what problems are they solving? Movie recommendations?

    1. John Tserkezis

      Re: A bicycle with a Ferrari exterior?

      "And what problems are they solving? Movie recommendations?"

      The usual things. How to download porn faster.

      1. illiad

        Re: A bicycle with a Ferrari exterior?

        huh, most large vid & pic sites have limited their access speed, to prevent acces problems..

        only *proper* way to check the d/l speed is to download a linux distro...

      2. Anonymous Coward
        Anonymous Coward

        "And what problems are they solving?"

        >"And what problems are they solving? Movie recommendations?"

        >The usual things. How to download porn faster.

        Actually Google does have a class of hard problems that defy algorithmic solutions:

        If customer A07f5DE7C4 watches latex granny porn, and customer B79F6E3C8 prefers vinyl doggy style, should Google display ads for crunchy or smooth peanut butter?

    2. TheVogon

      Re: A bicycle with a Ferrari exterior?

      "And what problems are they solving?"

      How to spy on more aspects of your life at once and target appropriate adverts presumably. What else would The Borg be doing?.

    3. Michael Wojcik Silver badge

      Re: A bicycle with a Ferrari exterior?

      And what problems are they solving? Movie recommendations?

      In all seriousness, the majority of the problems Google has to solve - from maintaining their huge network of computing resources and workloads, to modeling user preferences - can be transformed into various graph problems. And many of those can be converted into questions of energy distribution in a closed system. As such, annealing approaches, which is what the D-Wave seems to be doing1, should be applicable in many cases, and could offer significant speedups over classical discrete calculation.

      Of course that's true of other analog mechanisms, "quantum" or not, as any number of people have noted.

      1I take no position on whether it's "true" adiabatic quantum computation (AQC), or "true" quantum annealing, etc, or just a hybrid classical analog-digital machine. I haven't looked at it and I'm not qualified to make that determination anyway.

  3. Paul 87

    So basically, bad code causes slowdowns?

    Who'd have thought it!

    1. Destroy All Monsters Silver badge

      So basically, bad code causes slowdowns?

      No, because there is no code.

      It's a "quantum annealing" machine, which is something like a cooling metal, only with configurable strengths between the configuration elements (think analog computer doing neural networks or spin glass annealing), and with more quantum sauce.

      I also notice that there are two Google Guys/Gals in the list of authors:

      > Troels F. Rønnow - Theoretische Physik, ETH Zurich

      > Zhihui Wang - Department of Chemistry, University of Southern California

      > Joshua Job - Center for Quantum Information Science and Technology & Department of Physics, University of Southern California

      > Sergio Boixo - Google, 150 Main Street, Venice Beach, California

      > Sergei V. Isakov - Google, Brandschenkestrasse 110, Zurich

      > David Wecker - Quantum Architectures and Computation Group, Microsoft Research, Redmond

      > John M. Martinis - Department of Physics, University of California Santa Barbara

      > Daniel A. Lidar - Department of Chemistry & Center for Quantum Information Science and Technology & Department of Physics, University of Southern California

      > Matthias Troyer - Theoretische Physik, ETH Zurich

      1. Zane

        Not programmable

        So do I get this right: it's not programmable, i.e. it's not a computer at all, it's just a machine that is optimized for some special problem? Than what is the problem that it solves so fast?


        1. Anonymous Coward
          Anonymous Coward

          Re: Not programmable

          I'd rephrase to a machine optimized to handle a certain class of problems, not any one particular problem. What you do is construct the experimental setup so that when the system "relaxes" you end up with the optimal solution to that particular problem. It's all about getting the preconditions right. There's a certain quantum-mechanical sauce (retrieving the 'solution' without collapsing the wave-function) but that's the idea.

          Trying to compare it to a system setup with a highly specialized, highly optimized GPGPU system really doesn't do it, in my not so humble opinion. My problem here, and as I recall this was pointed out before, is that you have a hell of a time trying to identify when those problem domains actually overlap between the two methods and for which you should detect a measurable (measure, quantum, uhhh...) set of results that positively (p<.01 I believe) demonstrates that the D-Wave is a quantum computer.

          Instead what we end up with is some problem types that demonstrate the D-Wave is ideal for that one particular problem and I mean one, not the whole class of problem, a lot of experiments that show equivalence between classical and quantum, and some for which the D-Wave doesn't come up with a solution. Personally, I am not surprised. First, if this were easy, we'd have done it decades ago. Second, thinking how to properly 'program' the D-Wave would give most anyone migraine's, ala Dr. Feynmann and understanding QM. Lastly, if you don't believe in a personally malignant universe, you will now.

          1. phil dude

            Re: Not programmable

            A nice post! I am still waiting for my slides from D-Wave for the curious presentation given at SC13... I was given the nudge not to discuss "certain uses". Go figure, it is only a hobby for me...

            However, buying a supercomputer to program your quantum computer (that was the presentation's thrust, though it was mentioned it was part of an iterative process), is marketing genius at a computing conference - no really! Buy 2!!

            I have a number of colleagues who really hope this works, but so far the resounding lack of evidence suggests there is much more research to do understanding exactly what the machine they have built *is*. Apparently, if you agree not to use the QC for anything "useful" you can apply online to have a go.....see what I did there?

            In another thread I wrote "it's still a really good fridge". A friend of mine who makes deuterium bullets, thought is was a cryogenics research project on first hearing "microKelvin" mentioned....


            1. Anonymous Coward

              Re: Not programmable

              If you're looking for interesting coincidences (of which I am not certain exist), go read my comment over in Pond Scum:


              Same class of problem, interesting how nature achieves it, and without any reference to micro-Kelvins.

  4. John H Woods

    So are these D-Wave boxes fast ...

    ... until you observe them?

  5. bigtimehustler

    So, seems to me everyone who buys one sees their benefits but a group of scientists who didn't have to justify the business case for buying it don't think its fast? I know who I believe, and frankly that is the people who laid their jobs on the line recommending the purchase of the machines and use them in real world situations, not a fake lab test that no one will ever run on the system ever.

    1. Anonymous Coward
      Anonymous Coward

      Emperor's clothes

      "everyone who buys one sees their benefits but a group of sceptics who didn't have to justify the business case for buying it don't think its fast? "

      By any chance have you worked for Autonomy in the past?

    2. Anonymous Coward
      Anonymous Coward

      " I know who I believe"

      The D-Wave machine may be perfectly genuine, but surely we all know people who've bought some immensely expensive piece of junk and have then managed to spend long enough promoting it internally before the shiny wore off, to polish the CV, and find a job somewhere else? It may not be that common, but it does happen.

      I'd mention two or three cases from my own experience but sadly the details might create enough identification to be libellous; though I can say that two of them involved the purchase of electron microscopes to solve problems that, in fact, could not be tackled in this way.

  6. David Dawson

    We live in a world with quantum computers!

    I'd still quite like my jetpack, but this is really cool.

  7. illiad


    when the PCI-E bus came out, it seemed too complex and slow.... and everyone said all the other new buses were better, PCI-E was more expensive, sometimes even slower....

    then CPU speeds increased, memory got a lot better, and they improved the PCI-E architecture...

    I think you know who the winner is today..... :p

    when they manage to get a 'pc-standard' basic test going on the D-Wave, then we will get something believable...

  8. Tsunamijuan

    The old game of benchmarks

    Every time I discuss benchmarks with people its sorta like discussing politics. You have to take into account whose doing what. What initial bias they have if its intentional or not. Then you have to take into account who ran the benchmarks and how versed they are in what is being tested.

    Most of the time the person running the benchmarks has more experience running one over the others. Specifically if they had an extremely limited amount of time with what they are benchmarking. Then there are in house benchmarks. These will always be skewed somewhat in favor of the companies own software. In part cause they know how to optimize and tweak their software better. Then the other item is left pretty much out of the box or sometimes even unconfigured.

    To put it mildly benchmarking and benchmarks mean nothing unless you have decided on the ruleset before hand that is going to be the standard, and constantly check to make sure that the standard has been adhered too. Even then there is a enough of a margin of error that can throw this stuff of.

    1. Anonymous Coward
      Anonymous Coward

      Re: The old game of benchmarks

      And provided that someone sneaky hasn't programmed that the machine in question detects you running the benchmark. Samsung anyone?

      1. Anonymous Coward
        Anonymous Coward

        Re: The old game of benchmarks

        Not only Samsung. Wasn't there once a benchmark where the compiler identified empty loops and short circuited them on one maker's processors, and iterated them fully on another?

  9. DropBear

    Ok so deciding once and for all exactly how "quantum" the machine is looks like a hard job, and I even get that we're not quite sure exactly what sort of thing it might be good at. But from a purely pragmatical point of view evaluating its usefulness is not hard at all - take a Hard Problem you're interested in, get the best conventional hardware you can, put it side by side with the D-wave kit and see which gives you more answers per second, by how much, and how accurate / optimal each is. That's it! If you really observe 35000 answers from one while a single one from the other, it really is faster, quantum or not - unless it's 35000 time more expensive too, it's a win. If not, we're just playing "The emperor's new clothes" all over again...

    1. Anonymous Coward
      Anonymous Coward

      Apart from all the other factors, it's necessary to take into account problem preparation time - an issue identified as far back as the days of one A M Turing, and the driving force behind early programming languages.

      It's like the interpreted versus compiled languages issue - if I can develop code to solve a particular problem much faster in, say, Python, than in C++, the slower execution time only matters if I need to solve the same problem many times with different parameters. Your argument would be correct if the class of problem we are talking about is like that. But if the problem is a few-off only, getting 35000 answers from one and one from another may not matter at all.

  10. Anonymous Coward
    Anonymous Coward

    A suggestion

    Feed Mr 503 qubit machine a real world problem such as simulating the behavior of a simple superconductor at variable temperatures.

    A regular PC would struggle and even a supercomputer would take ages to do the calculations.

  11. James Loughner

    But does it

    run flight simulator??

    Had to be said

    Mine is the one with the joy stick in the pocket

  12. John Smith 19 Gold badge

    Seems they are playing the old school definition of a "computer"

    IE Something that computes (something).

    The RL10 rocket engine contains a similar "computer" for mixture ratio control.

    Most people today would call it a hardwired analog computer. Operational amplifiers, temperature sensors, no actual ADC/DAC in sight etc.

    So what kind of problems do Google get a 1:35000 speed up on?

    1. Jellied Eel Silver badge

      Re: Seems they are playing the old school definition of a "computer"

      What do you get if you multiply six by nine?

      1. illiad

        Re: Seems they are playing the old school definition of a "computer"

        well however much you like Douglas, it is NOT the right answer.....:P :/

        the real problem is 22 divided by 7.. how long does it take, to get 1 million digits..... :P :D

  13. DerekCurrie

    Judging an Experiment as a Commercial Product is Rarely Helpful.

    The overwhelming sense I have, reading about this situation, is that quantum computing is still in an infantile stage. But here we have it being judged in public as if it's a viable product ready for public consumption, when it obviously is not. The enormous Clunk Factor going on at D-Wave is inevitable at this point. To have them shoved into defensive mode over an obviously Not Ready For Prime Time machine is ridiculous. I hope this wrong attitude toward the technology doesn't kill the baby before it has a chance to grow up.

    And then there's the possibility that D-Wave really does have a dog that won't hunt. But this clearly is not the time for that judgement.

    1. andre 2

      Re: Judging an Experiment as a Commercial Product is Rarely Helpful.

      Considering the QPU is cooled down to millikelvins, in a special room similar to the shielded one used for MRI scanners and shielded from everything including ambient radiation I would say that D-Wave do indeed have a quantum computer.

      calling is a "physics experiment" is like calling Fat Man a "Sunshine simulation device"... Doesen't do it justice and D-Wave deserve serious funding whether their system is fully tested or not.

      We can't afford to be left behind on this, China has its own version in the pipeline and South Korea even has a prototype under construction.

      503 qubits is incredible progress in such a short time even if they are flux ones, ten years ago this would have been science fiction.

      1. Destroy All Monsters Silver badge

        Re: Judging an Experiment as a Commercial Product is Rarely Helpful.

        Given that ... I would say that D-Wave do indeed have a quantum computer

        Although your biggening up is noted, your criteria for QC-ness are frankly lacking in salience.

        You are of course free to send your own money to D-Wave (far be it from me to have a negative opinion on that), but please don't go all "we this" and "we that" here.

    2. John Smith 19 Gold badge

      Re: Judging an Experiment as a Commercial Product is Rarely Helpful.

      "And then there's the possibility that D-Wave really does have a dog that won't hunt. But this clearly is not the time for that judgement."

      I recall a company that put out some of the first Josephson Junction superconductor components. Made in Nb and LHe cooled.

      They were commercial.

      Were they "Ready for prime time?"

      Probably not.

      But they cost their early adoptors a shedload of cash to find that out.

This topic is closed for new posts.

Other stories you might like

  • Hangouts hangs up: Google chat app shuts this year
    How many messaging services does this web giant need? It's gotta be over 9,000

    Google is winding down its messaging app Hangouts before it officially shuts in November, the web giant announced on Monday.

    Users of the mobile app will see a pop-up asking them to move their conversations onto Google Chat, which is yet another one of its online services. It can be accessed via Gmail as well as its own standalone application. Next month, conversations in the web version of Hangouts will be ported over to Chat in Gmail. 

    Continue reading
  • It's a crime to use Google Analytics, watchdog tells Italian website
    Because data flows into the United States, not because of that user interface

    Updated Another kicking has been leveled at American tech giants by EU regulators as Italy's data protection authority ruled against transfers of data to the US using Google Analytics.

    The ruling by the Garante was made yesterday as regulators took a close look at a website operator who was using Google Analytics. The regulators found that the site collected all manner of information.

    So far, so normal. Google Analytics is commonly used by websites to analyze traffic. Others exist, but Google's is very much the big beast. It also performs its analysis in the USA, which is what EU regulators have taken exception to. The place is, after all, "a country without an adequate level of data protection," according to the regulator.

    Continue reading
  • Google has more reasons why it doesn't like antitrust law that affects Google
    It'll ruin Gmail, claims web ads giant

    Google has a fresh list of reasons why it opposes tech antitrust legislation making its way through Congress but, like others who've expressed discontent, the ad giant's complaints leave out mention of portions of the proposed law that address said gripes.

    The law bill in question is S.2992, the Senate version of the American Innovation and Choice Online Act (AICOA), which is closer than ever to getting votes in the House and Senate, which could see it advanced to President Biden's desk.

    AICOA prohibits tech companies above a certain size from favoring their own products and services over their competitors. It applies to businesses considered "critical trading partners," meaning the company controls access to a platform through which business users reach their customers. Google, Apple, Amazon, and Meta in one way or another seemingly fall under the scope of this US legislation. 

    Continue reading
  • Google to pay $90m to settle lawsuit over anti-competitive behavior on the Play Store
    US developers that qualify could receive more than $200,000

    Google is to pay $90 million to settle a class-action lawsuit with US developers over alleged anti-competitive behavior regarding the Google Play Store.

    Eligible for a share in the $90 million fund are US developers who earned two million dollars or less in annual revenue through Google Play between 2016 and 2021. "A vast majority of US developers who earned revenue through Google Play will be eligible to receive money from this fund," said Google.

    Law firm Hagens Berman announced the settlement this morning, having been one of the first to file a class case. The legal firm was one of four that secured a $100 million settlement from Apple in 2021 for US iOS developers.

    Continue reading
  • End of the road for biz living off free G Suite legacy edition
    Firms accustomed to freebies miffed that web giant's largess doesn't last

    After offering free G Suite apps for more than a decade, Google next week plans to discontinue its legacy service – which hasn't been offered to new customers since 2012 – and force business users to transition to a paid subscription for the service's successor, Google Workspace.

    "For businesses, the G Suite legacy free edition will no longer be available after June 27, 2022," Google explains in its support document. "Your account will be automatically transitioned to a paid Google Workspace subscription where we continue to deliver new capabilities to help businesses transform the way they work."

    Small business owners who have relied on the G Suite legacy free edition aren't thrilled that they will have to pay for Workspace or migrate to a rival like Microsoft, which happens to be actively encouraging defectors. As noted by The New York Times on Monday, the approaching deadline has elicited complaints from small firms that bet on Google's cloud productivity apps in the 2006-2012 period and have enjoyed the lack of billing since then.

    Continue reading
  • Google battles bots, puts Workspace admins on alert
    No security alert fatigue here

    Google has added API security tools and Workspace (formerly G-Suite) admin alerts about potentially risky configuration changes such as super admin passwords resets.

    The API capabilities – aptly named "Advanced API Security" – are built on top of Apigee, the API management platform that the web giant bought for $625 million six years ago.

    As API data makes up an increasing amount of internet traffic – Cloudflare says more than 50 percent of all of the traffic it processes is API based, and it's growing twice as fast as traditional web traffic – API security becomes more important to enterprises. Malicious actors can use API calls to bypass network security measures and connect directly to backend systems or launch DDoS attacks.

    Continue reading
  • I was fired for blowing the whistle on cult's status in Google unit, says contractor
    The internet giant, a doomsday religious sect, and a lawsuit in Silicon Valley

    A former Google video producer has sued the internet giant alleging he was unfairly fired for blowing the whistle on a religious sect that had all but taken over his business unit. 

    The lawsuit demands a jury trial and financial restitution for "religious discrimination, wrongful termination, retaliation and related causes of action." It alleges Peter Lubbers, director of the Google Developer Studio (GDS) film group in which 34-year-old plaintiff Kevin Lloyd worked, is not only a member of The Fellowship of Friends, the exec was influential in growing the studio into a team that, in essence, funneled money back to the fellowship.

    In his complaint [PDF], filed in a California Superior Court in Silicon Valley, Lloyd lays down a case that he was fired for expressing concerns over the fellowship's influence at Google, specifically in the GDS. When these concerns were reported to a manager, Lloyd was told to drop the issue or risk losing his job, it is claimed. 

    Continue reading
  • FTC urged to probe Apple, Google for enabling ‘intense system of surveillance’
    Ad tracking poses a privacy and security risk in post-Roe America, lawmakers warn

    Democrat lawmakers want the FTC to investigate Apple and Google's online ad trackers, which they say amount to unfair and deceptive business practices and pose a privacy and security risk to people using the tech giants' mobile devices.

    US Senators Ron Wyden (D-OR), Elizabeth Warren (D-MA), and Cory Booker (D-NJ) and House Representative Sara Jacobs (D-CA) requested on Friday that the watchdog launch a probe into Apple and Google, hours before the US Supreme Court overturned Roe v. Wade, clearing the way for individual states to ban access to abortions. 

    In the days leading up to the court's action, some of these same lawmakers had also introduced data privacy bills, including a proposal that would make it illegal for data brokers to sell sensitive location and health information of individuals' medical treatment.

    Continue reading
  • Google: How we tackled this iPhone, Android spyware
    Watching people's every move and collecting their info – not on our watch, says web ads giant

    Spyware developed by Italian firm RCS Labs was used to target cellphones in Italy and Kazakhstan — in some cases with an assist from the victims' cellular network providers, according to Google's Threat Analysis Group (TAG).

    RCS Labs customers include law-enforcement agencies worldwide, according to the vendor's website. It's one of more than 30 outfits Google researchers are tracking that sell exploits or surveillance capabilities to government-backed groups. And we're told this particular spyware runs on both iOS and Android phones.

    We understand this particular campaign of espionage involving RCS's spyware was documented last week by Lookout, which dubbed the toolkit "Hermit." We're told it is potentially capable of spying on the victims' chat apps, camera and microphone, contacts book and calendars, browser, and clipboard, and beam that info back to base. It's said that Italian authorities have used this tool in tackling corruption cases, and the Kazakh government has had its hands on it, too.

    Continue reading
  • W3C overrules objections by Google, Mozilla to decentralized identifier spec
    Oh no, he DIDn't

    The World Wide Web Consortium (W3C) has rejected Google's and Mozilla's objections to the Decentralized Identifiers (DID) proposal, clearing the way for the DID specification to be published a W3C Recommendation next month.

    The two tech companies worry that the open-ended nature of the spec will promote chaos through a namespace land rush that encourages a proliferation of non-interoperable method specifications. They also have concerns about the ethics of relying on proof-of-work blockchains to handle DIDs.

    The DID specification describes a way to deploy a globally unique identifier without a centralized authority (eg, Apple for Sign in with Apple) as a verifying entity.

    Continue reading

Biting the hand that feeds IT © 1998–2022