Value Losses

keras_gym.losses.ProjectedSemiGradientLoss Loss function for type-II Q-function.
keras_gym.losses.RootMeanSquaredError Root-mean-squared error (RMSE) loss.
keras_gym.losses.LoglossSign Logloss implemented for predicted logits \(z\in\mathbb{R}\) and ground truth \(y\pm1\).
class keras_gym.losses.ProjectedSemiGradientLoss(G, base_loss=<tensorflow.python.keras.losses.Huber object>)[source]

Loss function for type-II Q-function.

This loss function projects the predictions \(q(s, .)\) onto the actions for which we actually received a feedback signal.

Parameters:
G : 1d Tensor, dtype: float, shape: [batch_size]

The returns that we wish to fit our value function on.

base_loss : keras loss function, optional

Keras loss function. Default: huber_loss.

__call__(self, A, Q_pred, sample_weight=None)[source]

Compute the projected MSE.

Parameters:
A : 2d Tensor, dtype: int, shape: [batch_size, num_actions]

A batch of (one-hot encoded) discrete actions A.

Q_pred : 2d Tensor, shape: [batch_size, num_actions]

The predicted values \(q(s,.)\), a.k.a. y_pred.

sample_weight : Tensor, dtype: float, optional

Tensor whose rank is either 0 or is broadcastable to y_true. sample_weight acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the total loss for each sample of the batch is rescaled by the corresponding element in the sample_weight vector.

Returns:
loss : 0d Tensor (scalar)

The batch loss.

call(self, y_true, y_pred)

Invokes the Loss instance.

Args:
y_true: Ground truth values, with the same shape as ‘y_pred’. y_pred: The predicted values.
class keras_gym.losses.RootMeanSquaredError(delta=1.0, name='root_mean_squared_error')[source]

Root-mean-squared error (RMSE) loss.

Parameters:
name : str, optional

Optional name for the op.

__call__(self, y_true, y_pred, sample_weight=None)[source]

Compute the RMSE loss.

Parameters:
y_true : Tensor, shape: [batch_size, …]

Ground truth values.

y_pred : Tensor, shape: [batch_size, …]

The predicted values.

sample_weight : Tensor, dtype: float, optional

Tensor whose rank is either 0, or the same rank as y_true, or is broadcastable to y_true. sample_weight acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the total loss for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight matches the shape of y_pred, then the loss of each measurable element of y_pred is scaled by the corresponding value of sample_weight.

Returns:
loss : 0d Tensor (scalar)

The batch loss.

call(self, y_true, y_pred)

Invokes the Loss instance.

Args:
y_true: Ground truth values, with the same shape as ‘y_pred’. y_pred: The predicted values.
class keras_gym.losses.LoglossSign[source]

Logloss implemented for predicted logits \(z\in\mathbb{R}\) and ground truth \(y\pm1\).

\[L\ =\ \log\left( 1 + \exp(-y\,z) \right)\]
__call__(self, y_true, z_pred, sample_weight=None)[source]
Parameters:
y_true : Tensor, shape: [batch_size, …]

Ground truth values \(y\pm1\).

z_pred : Tensor, shape: [batch_size, …]

The predicted logits \(z\in\mathbb{R}\).

sample_weight : Tensor, dtype: float, optional

Not yet implemented.

#TODO: implement this

call(self, y_true, y_pred)

Invokes the Loss instance.

Args:
y_true: Ground truth values, with the same shape as ‘y_pred’. y_pred: The predicted values.