OpenAI gym provides a nice modular interface to extend existing environments using environment wrappers. Here we list some wrappers that are used throughout the keras-gym package.


The default preprocessor tries to create a feature vector from any environment state observation on a best-effort basis. For instance, if the observation space is discrete \(s\in\{0, 1, \dots, n-1\}\), it will create a one-hot encoded vector such that the wrapped environment yields state observations \(s\in\mathbb{R}^n\).

import gym
import keras_gym as km

env = gym.make('FrozenLake-v0')
env = km.wrappers.DefaultPreprocessor(env)

s = env.unwrapped.reset()  # s == 0
s = env.reset()            # s == [1, 0, 0, ..., 0]

Other preprocessors that are particularly useful when dealing with video input are ImagePreprocessor and FrameStacker. For instance, for Atari 2600 environments we usually apply preprocessing as follows:

env = gym.make('PongDeterministic-v4')
env = km.wrappers.ImagePreprocessor(env, height=105, width=80, grayscale=True)
env = km.wrappers.FrameStacker(env, num_frames=4)

s = env.unwrapped.reset()  # s.shape == (210, 160, 3)
s = env.reset()            # s.shape == (105,  80, 4)

The first wrapper does down-scaling and grayscaling on each input frame. The second wrapper then stacks consecutive frames together, which allows for the function approximator to learn velocities/accelerations as well as positions for each input pixel.


Another type of environment wrapper is a monitor, which is used to keep track of the progress of the training process. At the moment, keras-gym only provides a generic train monitor called TrainMonitor