问题描述
有人试过用 TensorFlow 进行多任务深度学习吗?也就是说,共享底层而不共享顶层.一个带有简单插图的例子会有很大帮助.
有一个类似的问题
这个图是根据这个
假设我们正在训练一个分类器来预测图像中的数字,每张图像最多 5 个数字.这里我们定义了6个输出层:digit1
、digit2
、digit3
、digit4
、digit5
>, 长度
.digit
层如果有这样的数字应该输出0~9,或者X
(实践中用实数代替)如果有其位置中没有任何数字.length
也是一样,如果图片包含0~5位就应该输出0~5,如果超过5位就应该输出X
.
现在要训练它,我们只需将每个 softmax 函数的所有交叉熵损失相加即可:
# 定义损失和优化器lossLength = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(length_logits, true_length)), 1e-37, 1e+37))lossDigit1 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit1_logits, true_digit1)), 1e-37, 1e+37))lossDigit2 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit2_logits, true_digit2)), 1e-37, 1e+37))lossDigit3 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit3_logits, true_digit3)), 1e-37, 1e+37))lossDigit4 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit4_logits, true_digit4)), 1e-37, 1e+37))lossDigit5 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit5_logits, true_digit5)), 1e-37, 1e+37))成本 = tf.add(tf.add(tf.add(tf.add(tf.add(cL,lossDigit1),lossDigit2),lossDigit3),lossDigit4),lossDigit5)优化器 = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
Have someone tried doing multitask deep learning with TensorFlow? That is, sharing the bottom layers while not sharing the top layers. An example with simple illustration would help a lot.
There is an similar question here, the answer used keras.
It's similar when just using tensorflow. The idea is this: we can define multiple outputs of a network, and thus multiple loss functions (objectives). We then tell optimizer to minimize a combined loss function, usually using a linear combination.
A concept diagram
This diagram is drawn according to this paper.
Let's say we are training a classifier that predict the digit in the image, with maximum 5 digits per image. Here we defined 6 output layer: digit1
, digit2
, digit3
, digit4
, digit5
, length
. The digit
layer should output 0~9 if there is such a digit, or X
(substitute it with an real number in practice) if there isn't any digit in its position. Same thing for length
, it should output 0~5 if the image contains 0~5 digit, or X
if it contains more than 5 digits.
Now to train it, we just add up all the cross entropy loss of each softmax function:
# Define loss and optimizer
lossLength = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(length_logits, true_length)), 1e-37, 1e+37))
lossDigit1 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit1_logits, true_digit1)), 1e-37, 1e+37))
lossDigit2 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit2_logits, true_digit2)), 1e-37, 1e+37))
lossDigit3 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit3_logits, true_digit3)), 1e-37, 1e+37))
lossDigit4 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit4_logits, true_digit4)), 1e-37, 1e+37))
lossDigit5 = tf.log(tf.clip_by_value(tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(digit5_logits, true_digit5)), 1e-37, 1e+37))
cost = tf.add(
tf.add(
tf.add(
tf.add(
tf.add(cL,lossDigit1),
lossDigit2),
lossDigit3),
lossDigit4),
lossDigit5)
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
这篇关于使用 Tensorflow 进行多任务深度学习的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!