back to article Wanna build an AI robot? Don't have an actual robot yet? Try this Holodeck for droids

OpenAI today updated Gym – its system for training intelligent software – so that developers can teach physical robots to hold pens, pick up and move objects, and so on. Gym was launched in 2016, and is a toolkit for teaching programs new tricks, such as playing Atari games and balancing poles, via reinforcement learning (RL …

  1. Anonymous Coward
    Anonymous Coward


    I've been hearing about people in colleagues and unis build robots, and program them for years, all the while the robots bumping into chairs and the "creator" of them scratching heir heads. Robots are 90% a programming problem, 10% hardware.

    A robot needs to adapt to the changes in the environment. You can build an entire robot, put it in the environment, and hope the 100 ton automated tanker does not drive into a house.

    But then we got this thing call a computer, and their power got rather high many years ago. If most games cannot get AI right, what hope is there for a robot? Why waste costs into making many many designs, when you can iterate virtually, and make just a few (see SpaceX)?

    1. jake Silver badge

      Re: Finally!

      I guess you really gotta watch out for them thar heir-head colleagues ...

      1. Anonymous Coward
        Anonymous Coward

        Re: Finally!

        Yep. Good old typos. Still another reason to show trialling something virtually is better than releasing to the public too early. ;)

  2. Chris Gray 1

    Sounds random

    This "sparse rewards" thing may be useful in some situations, but it sounds like you are asking for trouble and more work. If it solves the problem in some unique way, you have no information on how it solved the problem (for problems which don't involve watching the result happen). If you guide it with intermediate rewards, you at least have basic ideas of how it is achieving the result, and so can pick better test cases.

    After all, shaking the wine bottle up and down a few times may well end up with the glass containing the right amount of wine. But, that doesn't sound like a desired solution - you haven't had any way to put reasonable constraints on the solution found.

    1. Dave 126 Silver badge

      Re: Sounds random

      If the virtual robot shakes the champagne bottle up and down until the cork pops and some liquid ends up in the glass... then the virtual robot clearly wants to drive F1 cars when it grows up!

    2. Dave 126 Silver badge

      Re: Sounds random

      > If it solves the problem in some unique way, you have no information on how it solved the problem (for problems which don't involve watching the result happen).

      My joking aside, that's an interesting point; it sounds like our (incomplete/poor) understanding of many biological systems. We understand that it is working but we don't know how.

      Biological systems, from DNA to nervous systems, have arrived at their current state through many iterative process.

  3. Def Silver badge

    Star Trek?

    This is clearly the first step to The Thirteenth Floor.

    1. Fruit and Nutcase Silver badge

      Re: Star Trek?

      It's AI Gym, not as we know it

      1. Swarthy Silver badge
        Thumb Up

        Re: Star Trek?

        @ Fruit and Nutcase:

        Nice! That is some quality stuff. Shame I have but one upvote to give you.

        1. Fruit and Nutcase Silver badge

          Re: Star Trek?



          And Thanks too to The Firm, for Star Trekkin'.

          This is the second time recently that I've had to opportunity to lift a line or two from Star Trekkin'. And in both cases, I'd omitted to mention them. So now I have.

      2. onefang

        Re: Star Trek?

        I'm headed to the replicator, I need a new keyboard.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like