自定义损失函数，通过梯度下降在每一步进行更新

问题描述

通过此帖子，我们可以编写自定义损失函数.现在，假设自定义损失函数取决于参数a:

From this post, we can write a custom loss function. Now, assume that the custom loss function depends on parameter a:

def customLoss(yTrue,yPred):
    return (K.log(yTrue) - K.log(yPred))**2+a*yPred

如何在每个步骤中像权重一样以梯度下降的方式更新参数a?:

How can we update parameter a at each step in a gradient descent manner like the weights?:

a_new= a_old - alpha * (derivative of custom loss with respect to a)

P.S.实际的海关损失与上述情况有所不同.请给我一个适用于任何自定义损失函数的通用答案，而不是上述示例的答案.

P.S. the real custom loss is different from the above. Please give me a general answer that works for any arbitrary custom loss function, not an answer to the example above.

推荐答案

创建一个自定义图层来保存可训练的参数.该层不会在调用中返回输入，但是我们将使用输入来遵守如何创建层.

Create a custom layer to hold the trainable parameter. This layer will not return the inputs in its call, but we are going to have the inputs for complying with how you create layers.

class TrainableLossLayer(Layer):

    def __init__(self, a_initializer, **kwargs):
        super(TrainableLossLayer, self).__init__(**kwargs)
        self.a_initializer = keras.initializers.get(a_initializer)

    #method where weights are defined
    def build(self, input_shape):
        self.kernel = self.add_weight(name='kernel_a', 
                                  shape=(1,),
                                  initializer=self.a_initializer,
                                  trainable=True)
        self.built=True

    #method to define the layers operation (only return the weights)
    def call(self, inputs):
        return self.kernel

    #output shape
    def compute_output_shape(self, input_shape):
        return (1,)

使用模型中的图层通过任何输入获取a(这与顺序模型不兼容):

Use the layer in your model to get a with any inputs (this is not compatible with a Sequential model):

a = TrainableLossLayer(a_init, name="somename")(anyInput)

现在，您可以尝试用一种难看的方式来定义损失:

Now, you can try to define your loss in a sort of ugly way:

def customLoss(yTrue,yPred):
    return (K.log(yTrue) - K.log(yPred))**2+a*yPred

如果这行得通，那就准备好了.

If this works, then it's ready.

您还可以尝试使用更复杂的模型(如果您不想在这样的层次上跳过损失时使用a，这可能会导致模型保存/加载方面的问题)

You can also try a more complicated model (if you don't want to use a in the loss jumping over the layers like that, this might cause problems in model saving/loading)

在这种情况下，您将需要y_train用作输入而不是输出:

In this case, you will need that y_train goes in as an input instead of an output:

y_true_inputs = Input(...)

您的损失函数将进入正确包含所有参数的Lambda层:

Your loss function will go into a Lambda layer taking all parameters properly:

def lambdaLoss(x):
    yTrue, yPred, alpha = x
    return (K.log(yTrue) - K.log(yPred))**2+alpha*yPred

loss = Lambda(lambdaLoss)([y_true_inputs, original_model_outputs, a])

您的模型将输出以下损失:

Your model will output this loss:

model = Model([original_model_inputs, y_true_inputs], loss)

您将拥有虚拟损失功能:

You will have a dummy loss function:

def dummyLoss(true, pred):
    return pred

model.compile(loss = dummyLoss, ...)

并训练为:

model.fit([x_train, y_train], anything_maybe_None_or_np_zeros ,....)

这篇关于自定义损失函数，通过梯度下降在每一步进行更新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！