Tensor Utilities

keras_gym.utils.box_to_reals_tf	Transform Tensor values from a Box space to the reals.
keras_gym.utils.box_to_unit_interval_tf	Rescale Tensor values from Box space to the unit interval.
keras_gym.utils.check_tensor	This helper function is mostly for internal use.
keras_gym.utils.diff_transform_matrix	A helper function that implements discrete differentiation for stacked state observations.
keras_gym.utils.log_softmax_tf	Compute the log-softmax.
keras_gym.utils.project_onto_actions_tf	Project tensor onto specific actions taken: tensorflow implementation.
keras_gym.utils.project_onto_actions_tf	Project tensor onto specific actions taken: tensorflow implementation.
keras_gym.utils.unit_interval_to_box_tf	Rescale Tensor values from the unit interval to a Box space.

keras_gym.utils.box_to_reals_tf(tensor, space, epsilon=1e-15)

Transform Tensor values from a Box space to the reals. This is done by first mapping the Box values to the unit interval \(x\in[0, 1]\) and then feeding it to the clipped_logit_tf() function.

Parameters:

tensor : nd Tensor

A tensor containing a single instance or a batch of elements of a Box space.

space : gym.spaces.Box

The Box space. This is needed to determine the shape and size of the space.

epsilon : float, optional

The cut-off value used by clipped_logit_tf().

Returns:

out : nd Tensor, same shape as input

A Tensor with the transformed values. The output values are real-valued.

keras_gym.utils.box_to_unit_interval_tf(tensor, space)

Rescale Tensor values from Box space to the unit interval. This is essentially just min-max scaling:

\[x\ \mapsto\ \frac{x-x_\text{low}}{x_\text{high}-x_\text{low}}\]

Parameters:

tensor : nd Tensor

A tensor containing a single instance or a batch of elements of a Box space.

space : gym.spaces.Box

The Box space. This is needed to determine the shape and size of the space.

Returns:

out : nd Tensor, same shape as input

A Tensor with the transformed values. The output values lie on the unit interval \([0,1]\).

keras_gym.utils.check_tensor(tensor, ndim=None, ndim_min=None, dtype=None, same_dtype_as=None, same_shape_as=None, same_as=None, int_shape=None, axis_size=None, axis=None)

This helper function is mostly for internal use. It is used to check a few common properties of a Tensor.

Parameters:

ndim : int or list of ints

Check K.ndim(tensor).

ndim_min : int

Check if K.ndim(tensor) is at least ndim_min.

dtype : Tensor dtype or list of Tensor dtypes

Check tensor.dtype.

same_dtype_as : Tensor

Check if dtypes match.

same_shape_as : Tensor

Check if shapes match.

same_as : Tensor

Check if both dtypes and shapes match.

int_shape : tuple of ints

Check K.int_shape(tensor).

axis_size : int

Check size along axis, where axis is specified by axis=... kwarg.

axis : int

The axis the check for size.

Raises:

TensorCheckError

If one of the checks fails, it raises a TensorCheckError.

keras_gym.utils.diff_transform_matrix(num_frames, dtype='float32')

A helper function that implements discrete differentiation for stacked state observations.

Let’s say we have a feature vector \(X\) consisting of four stacked frames, i.e. the shape would be: [batch_size, height, width, 4].

The corresponding diff-transform matrix with num_frames=4 is a \(4\times 4\) matrix given by:

\[\begin{split}M_\text{diff}^{(4)}\ =\ \begin{pmatrix} -1 & 0 & 0 & 0 \\ 3 & 1 & 0 & 0 \\ -3 & -2 & -1 & 0 \\ 1 & 1 & 1 & 1 \end{pmatrix}\end{split}\]

such that the diff-transformed feature vector is readily computed as:

\[X_\text{diff}\ =\ X\, M_\text{diff}^{(4)}\]

The diff-transformation preserves the shape, but it reorganizes the frames in such a way that they look more like canonical variables. You can think of \(X_\text{diff}\) as the stacked variables \(x\), \(\dot{x}\), \(\ddot{x}\), etc. (in reverse order). These represent the position, velocity, acceleration, etc. of pixels in a single frame.

Parameters:

num_frames : positive int

The number of stacked frames in the original \(X\).

dtype : keras dtype, optional

The output data type.

Returns:

M : 2d-Tensor, shape: [num_frames, num_frames]

A square matrix that is intended to be multiplied from the left, e.g. X_diff = K.dot(X_orig, M), where we assume that the frames are stacked in axis=-1 of X_orig, in chronological order.

keras_gym.utils.log_softmax_tf(Z, axis=-1)

Compute the log-softmax.

Note: This is the tensorflow implementation.

Parameters:

Z : Tensor

The input logits.

axis : int, optional

The axis along which to normalize, default is 0.

Returns:

out : Tensor of same shape as input

The entries may be interpreted as log-probabilities.

keras_gym.utils.project_onto_actions_tf(Y, A)

Project tensor onto specific actions taken: tensorflow implementation.

Note: This only applies to discrete action spaces.

Parameters:

Y : 2d Tensor, shape: [batch_size, num_actions]

The tensor to project down.

A : 1d Tensor, shape: [batch_size]

The batch of actions used to project.

Returns:

Y_projected : 1d Tensor, shape: [batch_size]

The tensor projected onto the actions taken.

keras_gym.utils.project_onto_actions_tf(Y, A)

Project tensor onto specific actions taken: tensorflow implementation.

Note: This only applies to discrete action spaces.

Parameters:

Y : 2d Tensor, shape: [batch_size, num_actions]

The tensor to project down.

A : 1d Tensor, shape: [batch_size]

The batch of actions used to project.

Returns:

Y_projected : 1d Tensor, shape: [batch_size]

The tensor projected onto the actions taken.

keras_gym.utils.unit_interval_to_box_tf(tensor, space)

Rescale Tensor values from the unit interval to a Box space. This is essentially inverted min-max scaling:

\[x\ \mapsto\ x_\text{low} + (x_\text{high} - x_\text{low})\,x\]

Parameters:

tensor : nd Tensor

A numpy array containing a single instance or a batch of elements of a Box space, scaled to the unit interval.

space : gym.spaces.Box

The Box space. This is needed to determine the shape and size of the space.

Returns:

out : nd Tensor, same shape as input

A Tensor with the transformed values. The output values are contained in the provided Box space.