Install OpenAI Universe on Ubuntu 16.04

OpenAI: Write Python program that plays any game. Optionally, use machine learning so that computer learns the game by itself.
OpenAI “hello world” is just ten lines. But installing the environment is difficult.
This article shows you how to install OpenAI Universe on Ubuntu 16.04 and Python 3. We also run a “Hello World” example, where OpenAI plays a simple Flash game.

This installation is complicated, as OpenAI Universe is quite a new piece of software. Following these instructions requires fluency with Linux command line.

Install Required Packages with apt-get

Let’s install the required packages. You could probably do with less packages than what I’m installing here.

$ sudo apt-get update
$ sudo apt-get -y install apt-file build-essential ccze curl docker docker-doc docker-registry facter git golang golang-github-mitchellh-go-vnc-dev htop ipython3 libgl1-mesa-dev libjpeg-turbo8-dev libx11-dev libxcursor-dev libxi-dev libxinerama-dev libxrandr-dev libxxf86vm1 libxxf86vm-dev make mesa-common-dev puppet python3-dev python3-numpy python3-virtualenv python-dev screen tree virtualenv x11*dev xtightvncviewer

Install Docker

Initially, Docker does not work. As OpenAI universe is based on Docker, it will not work without it.

$ docker ps   # does not work yet
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

You must be a member of docker group to use docker.

$ sudo adduser $(whoami) docker
$ newgrp docker
$ groups

Make sure “docker” is mentioned in the ‘groups’ output. Notice that newgrp only applies to one shell – you can log out and back in if you want it to work everywhere.
Docker ps output should print headings and not give any error messages:

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Verify that ‘docker ps’ works before proceeding.

Create VirtualEnv

$ cd $HOME && mkdir openai/ && cd openai/
$ virtualenv --python /usr/bin/python3 env
$ source env/bin/activate

Make sure your prompt starts with “(env)” so that your Python virtual environment is active:

(env) tero@chromium:~/openai$

Compile go-vncdriver from git

$ cd $HOME/openai/

Make sure your prompt starts with “(env)” so that Python modules will install into your virtualenv.

$ git clone
$ pip3 install numpy
$ ./ must compile without errors.

$ pip3 install -e ./

Install OpenAI Universe

$ cd $HOME/openai/
$ git clone
$ cd universe/
$ pip3 install -e .

Hello OpenAI

$ cd $HOME/openai/
$ nano

Write the contents, then save (ctrl-X y enter). This version is adapted from the official example to make sure we run Python version 3.

## Adapted from the official example
## See
import sys
assert sys.version_info[0]==3 # python3 required
import gym
import universe  # register the universe environments
env = gym.make('flashgames.DuskDrive-v0')
env.configure(remotes=1)  # automatically creates a local docker container
observation_n = env.reset()
while True:
 action_n = [[('KeyEvent', 'ArrowUp', True)] for ob in observation_n]  # your agent here
 observation_n, reward_n, done_n, info = env.step(action_n)


$ python3

On the first run, it takes a while to download everything. After some loading, the game opens in a new window – and starts playing automatically! Just like the screenshot below.
Now computer can play your games for you. How are you going to use all the free time? Jogging?


Tested on Ubuntu 16.04.1 LTS, Linux 4.4.0-53-generic. Article has been edited for clarity.

Posted in Uncategorized | Tagged , , , , , , , , , , , , , , , | 5 Comments

5 Responses to Install OpenAI Universe on Ubuntu 16.04

  1. Adamya Tripathi says:

    Hey Tero !
    Thanks for posting such a nice tutorial. I would like to know how to run any python script on the terminal. I am new to linux and openai both.
    I have followed all the steps and it ran perfectly with game rendering on a pop up windows.
    But I don’t know how to run any program casually, I mean I don’t know which commands to run on a terminal to use openai.
    I don’t wanna install any thing again. So, could you provide any command flow as to how simply run any python program on openai.
    Thank you!

  2. Unfortunately, there is very little beginner documentation on OpenAI available. The example code is explained step by step on
    You can run an openAI Python program as described above. To write your own OpenAI program, you could modify my example.
    Good luck with your Python learning.

  3. Ivan says:

    1) also after `docker ps` if you get an error:
    `Cannot connect to the Docker daemon`
    you should manually launch docker:
    sudo /usr/bin/docker daemon
    2) after importing if you have an error:
    `[2017-06-17 15:21:18,314] Remote closed: address=localhost:15901`
    you need to upgrade your OpenSSL by:
    pip install pyOpenSSL –upgrade
    also pyOpenSSL needs dependencies:
    apt-get install -y libffi-dev libssl-dev

  4. Ivan says:

    pip3 for Python 3

  5. HW says:

    Hi, Thank you very much for a very clear and crisp tutorial on installing and running openAI’s gym and universe: It should save a lot of effort for most people– particularly, beginners. Given the succinctness of your exposition, I would only add a trivial point that one has to be in the vncdriver folder to execute Secondly, you have mentioned that the virtual environment’s name must appear, but the instructions for some part do not seem to have it displayed?