Numpy-Array Utilities

keras_gym.utils.argmax	This is a little hack to ensure that argmax breaks ties randomly, which is something that numpy.argmax() doesn’t do.
keras_gym.utils.argmin	This is a little hack to ensure that argmin breaks ties randomly, which is something that numpy.argmin() doesn’t do.
keras_gym.utils.box_to_reals_np	Transform array values from a Box space to the reals.
keras_gym.utils.box_to_unit_interval_np	Rescale array values from Box space to the unit interval.
keras_gym.utils.check_numpy_array	This helper function is mostly for internal use.
keras_gym.utils.clipped_logit_np	A safe implementation of the logit function \(x\mapsto\log(x/(1-x))\).
keras_gym.utils.feature_vector	Create a feature vector out of a state observation \(s\) or an action \(a\).
keras_gym.utils.idx	Given a numpy array, return its corresponding integer index array.
keras_gym.utils.log_softmax	Compute the log-softmax.
keras_gym.utils.one_hot	Create a dense one-hot encoded vector.
keras_gym.utils.project_onto_actions_np	Project tensor onto specific actions taken: numpy implementation.
keras_gym.utils.reals_to_box_np	Transform array values from the reals to a Box space.
keras_gym.utils.softmax	Compute the softmax (normalized point-wise exponential).
keras_gym.utils.unit_interval_to_box_np	Rescale array values from the unit interval to a Box space.

keras_gym.utils.argmax(arr, axis=-1, random_state=None)

This is a little hack to ensure that argmax breaks ties randomly, which is something that numpy.argmax() doesn’t do.

Note: random tie breaking is only done for 1d arrays; for multidimensional inputs, we fall back to the numpy version.

Parameters:

a : array_like

Input array.

axis : int, optional

By default, the index is into the flattened array, otherwise along the specified axis.

random_state : int or RandomState

This can either be a random seed (int) or an instance of numpy.random.RandomState.

Returns:

index_array : ndarray of ints

Array of indices into the array. It has the same shape as a.shape with the dimension along axis removed.

keras_gym.utils.argmin(arr, axis=None, random_state=None)

This is a little hack to ensure that argmin breaks ties randomly, which is something that numpy.argmin() doesn’t do.

Note: random tie breaking is only done for 1d arrays; for multidimensional inputs, we fall back to the numpy version.

Parameters:

a : array_like

Input array.

axis : int, optional

By default, the index is into the flattened array, otherwise along the specified axis.

random_state : int or RandomState

This can either be a random seed (int) or an instance of numpy.random.RandomState.

Returns:

index_array : ndarray of ints

Array of indices into the array. It has the same shape as a.shape with the dimension along axis removed.

keras_gym.utils.box_to_reals_np(arr, space, epsilon=1e-15)

Transform array values from a Box space to the reals. This is done by first mapping the Box values to the unit interval \(x\in[0, 1]\) and then feeding it to the clipped_logit_np() function.

Parameters:

arr : nd array

A numpy array containing a single instance or a batch of elements of a Box space.

space : gym.spaces.Box

The Box space. This is needed to determine the shape and size of the space.

epsilon : float, optional

The cut-off value used by clipped_logit_np().

Returns:

out : nd array, same shape as input

A numpy array with the transformed values. The output values are real-valued.

keras_gym.utils.box_to_unit_interval_np(arr, space)

Rescale array values from Box space to the unit interval. This is essentially just min-max scaling:

\[x\ \mapsto\ \frac{x-x_\text{low}}{x_\text{high}-x_\text{low}}\]

Parameters:

arr : nd array

A numpy array containing a single instance or a batch of elements of a Box space.

space : gym.spaces.Box

The Box space. This is needed to determine the shape and size of the space.

Returns:

out : nd array, same shape as input

A numpy array with the transformed values. The output values lie on the unit interval \([0, 1]\).

keras_gym.utils.check_numpy_array(arr, ndim=None, ndim_min=None, dtype=None, shape=None, axis_size=None, axis=None)

This helper function is mostly for internal use. It is used to check a few common properties of a numpy array.

Raises:

NumpyArrayCheckError

If one of the checks fails, it raises a NumpyArrayCheckError.

keras_gym.utils.clipped_logit_np(x, epsilon=1e-15)

A safe implementation of the logit function \(x\mapsto\log(x/(1-x))\). It clips the arguments of the log function from below so as to avoid evaluating it at 0:

\[\text{logit}_\epsilon(x)\ =\ \log(\max(\epsilon, x)) - \log(\max(\epsilon, 1 - x))\]

Parameters:

x : nd array

Input numpy array whose entries lie on the unit interval, \(x_i\in [0, 1]\).

epsilon : float, optional

The small number with which to clip the arguments of the logarithm from below.

Returns:

z : nd array, dtype: float, shape: same as input

The output logits whose entries lie on the real line, \(z_i\in\mathbb{R}\).

keras_gym.utils.feature_vector(x, space)

Create a feature vector out of a state observation \(s\) or an action \(a\). This is used in the DefaultPreprocessor.

Parameters:

x : state or action

A state observation \(s\) or an action \(a\).

space : gym space

A gym space, e.g. gym.spaces.Box, gym.spaces.Discrete, etc.

keras_gym.utils.idx(arr, axis=0)

Given a numpy array, return its corresponding integer index array.

Parameters:

arr : array

Input array.

axis : int, optional

The axis along which we’d like to get an index.

Returns:

index : 1d array, shape: arr.shape[axis]

An index array [0, 1, 2, …].

keras_gym.utils.log_softmax(arr, axis=-1)

Compute the log-softmax.

Note: This is the numpy implementation.

Parameters:

arr : numpy array

The input array.

axis : int, optional

The axis along which to normalize, default is 0.

Returns:

out : array of same shape

The entries may be interpreted as log-probabilities.

keras_gym.utils.one_hot(i, n, dtype='float')

Create a dense one-hot encoded vector.

Parameters:

i : int or 1d array of ints

The index of the non-zero entry.

n : int

The dimensionality of the dense vector. Note that n must be greater than i.

dtype : str or datatype

The output data type, default is ‘float’.

Returns:

x : 1d array of length n

The dense one-hot encoded vector.

keras_gym.utils.project_onto_actions_np(Y, A)

Project tensor onto specific actions taken: numpy implementation.

Note: This only applies to discrete action spaces.

Parameters:

Y : 2d array, shape: [batch_size, num_actions]

The tensor to project down.

A : 1d array, shape: [batch_size]

The batch of actions used to project.

Returns:

Y_projected : 1d array, shape: [batch_size]

The tensor projected onto the actions taken.

keras_gym.utils.reals_to_box_np(arr, space)

Transform array values from the reals to a Box space. This is done by first applying the logistic sigmoid to map the reals onto the unit interval and then applying unit_interval_to_box_np() to rescale to the Box space.

Parameters:

arr : nd array

A numpy array containing a single instance or a batch of elements of a Box space, encoded as logits.

space : gym.spaces.Box

The Box space. This is needed to determine the shape and size of the space.

Returns:

out : nd array, same shape as input

A numpy array with the transformed values. The output values are contained in the provided Box space.

keras_gym.utils.softmax(arr, axis=-1)

Compute the softmax (normalized point-wise exponential).

Note: This is the numpy implementation.

Parameters:

arr : numpy array

The input array.

axis : int, optional

The axis along which to normalize, default is 0.

Returns:

out : array of same shape

The entries of the output array are non-negative and normalized, which make them good candidates for modeling probabilities.

keras_gym.utils.unit_interval_to_box_np(arr, space)

Rescale array values from the unit interval to a Box space. This is essentially inverted min-max scaling:

\[x\ \mapsto\ x_\text{low} + (x_\text{high} - x_\text{low})\,x\]

Parameters:

arr : nd array

A numpy array containing a single instance or a batch of elements of a Box space, scaled to the unit interval.

space : gym.spaces.Box

The Box space. This is needed to determine the shape and size of the space.

Returns:

out : nd array, same shape as input

A numpy array with the transformed values. The output values are contained in the provided Box space.