问题描述
我想为自定义数据集制作一个卷积神经网络。分类器只有两个类。我能够正确地读取输入图像，并且还为它们分配了两个相应类的batch_labels。代码执行没有错误，但输出是异常的。
由于某些原因，精度始终为50％。
image = inputs（）
image_batch = tf.train.batch（[image]，batch_size = 150）
label_batch_pos = tf.train.batch（[tf.constant（[0,1]）]，batch_size = 75）＃label_batch for first class
label_batch_neg = tf.train.batch（[ tf.constant（[1,0]）]，batch_size = 75）＃label_batch for second class
label_batch = tf.concat（0，[label_batch_pos，label_batch_neg]）
W_conv1 = weight_variable （[5，5，3，32]）
b_conv1 = bias_variable（[32]）
image_4d = tf.reshape（image，[-1,32,32,3]）
h_conv1 = tf.nn.relu（conv2d（image_4d，W_conv1）+ b_conv1）
h_pool1 = max_pool_2x2（h_conv1）
W_conv2 = weight_variable（[ 5，32,64]）
b_conv2 = bias_variable（[64]）
h_conv2 = tf.nn.relu（conv2d（h_pool1，W_conv2）+ b_conv2）
h_pool2 = max_pool_2x2（h_conv2）
W_fc1 = weight_variable（[8 * 8 * 64,1024]）
b_fc1 = bias_variable（[1024]）
h_pool2_flat = tf。 reshape（h_pool2，[-1,8 * 8 * 64]）
h_fc1 = tf.nn.relu（tf.matmul（h_pool2_flat，W_fc1）+ b_fc1）
h_fc1_drop = tf.nn.dropout h_fc1，0.5）
W_fc2 = weight_variable（[1024，2]）
b_fc2 = bias_variable（[2]）
y_conv = tf.nn.softmax tf.matmul（h_fc1_drop，W_fc2）+ b_fc2）
cross_entropy = -tf.reduce_sum（tf.cast（label_batch，tf.float32）* tf.log（y_conv + 1e-9））
train_step = tf.train.AdamOptimizer（1e-4）.minimize（cross_entropy）
init = tf.initialize_all_variables（）
sess = tf.Session（）
sess.run ）
tf.train.start_queue_runners（sess = sess）
correct_prediction = tf.equal（tf.argmax（y_conv，1），tf.argmax（label_batch，1） ）
accuracy = tf.reduce_mean（tf.cast（correct_prediction，tf.float32））
对于范围（100）中的i：
train_step.run（session = sess）
print（sess.run（accuracy））
print（sess.run（correct_prediction））
当我打印 correct_prediction 张量时，无论如何，我得到以下输出。
[True True True True True True True True True True True True True
True True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True True
True True True True True True True True True True True True True
True True True True True True True True True True True True True
True True True False False False False False False False False
False False False False False False False False False False
False False False False False False False False False假错误
错误False False False False False False False False False False False
False False False False False False False False False False False
False False False False False False False False False False False False
False False False False False]
准确度始终为0.5，权重没有被更新。当我在每个训练步骤之后打印权重时，它们保持不变。我想我有一些编码错误。可能是网络是在同一个图像上一次又一次的训练？但即使如此，权重必须更新。我有150个培训示例，75个属于每个类。
有人能请我指向正确的方向吗？
EDIT ：
这是我初始化权重的方式
def weight_variable（shape，name）：
initial = tf.truncated_normal（shape，stddev = 0.5）
return tf.Variable（initial，name = name）
def bias_variable（shape，name）：
initial = tf.constant（1.0，shape = shape）
return tf.Variable（initial，name = name）
您的网络有一些设计缺陷。由于数学问题，自己计算交叉熵并在输出层上应用softmax不是一个好主意。如果你对数学感兴趣，我可以添加这个，如果不坚持Tensorflow的解释和方法：。
您是否已尝试过许多不同的配置？根据图像的复杂性，更高或更低的内核大小和特征图的数量可能是一个好主意。一般来说，如果你的图像是相对同质的，很多相当相似的信息被加起来，因此如果你有很多特征映射，网络很难收敛。因为你只有两个输出神经元，我假设图像不是很复杂？
接下来是你的辍学。你总是使用0.5的辍学，但通常，为了测试/验证（如你的准确性预测），你不使用dropout。在大多数情况下，您只能使用它进行训练。您可以创建一个占位符，指定您的辍学率，并通过此 sess.run 。
own：
h_fc_drop = tf.nn.dropout（h_fc，keep_prob）
）
accu，top1，top3，top5 = sess.run（[accuracy，te_top1，te_top3，te_top5]，
feed_dict = {
x：teX [i：i + batch_size ]，
y：teY [i：i + batch_size]
keep_prob：1.0
}
）
这使Tensorflow可以计算 accuracy 和 topX 错误率的方程，而我在测试数据输入 teX 和实际标签 teY 中输入保持概率 keep_prob of 1.0 for the dropout。
尽管如此，你的权重初始化在深度神经网络中非常重要。即使你的设计足够你的类型的问题（这也必须调查）你的网络可以拒绝学习，发散或收敛到0，如果你的权重没有正确初始化。您没有向初始化添加详细信息，因此您可能想要查找 Xavier初始化。 是Xavier初始化的一个容易的开始。最后，我可以鼓励你绘制一些权重，特征地图，随着时间的输出等等，以了解你的网络正在做什么。通常这有助于很多。
I am trying to make a convolutional neural network for custom data set. The classifier has only two classes. I am able to read the input images properly and have also assigned them the batch_labels for the two corresponding classes . The code executes without error, but the output is anomalous.For some reason, the accuracy is always 50%.
image=inputs()
image_batch=tf.train.batch([image],batch_size=150)
label_batch_pos=tf.train.batch([tf.constant([0,1])],batch_size=75) # label_batch for first class
label_batch_neg=tf.train.batch([tf.constant([1,0])],batch_size=75) # label_batch for second class
label_batch=tf.concat(0,[label_batch_pos,label_batch_neg])
W_conv1 = weight_variable([5, 5, 3, 32])
b_conv1 = bias_variable([32])
image_4d = tf.reshape(image, [-1,32,32,3])
h_conv1 = tf.nn.relu(conv2d(image_4d, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
W_fc1 = weight_variable([8 * 8 * 64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1, 8*8*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)
h_fc1_drop = tf.nn.dropout(h_fc1, 0.5)
W_fc2 = weight_variable([1024, 2])
b_fc2 = bias_variable([2])
y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
cross_entropy = -tf.reduce_sum(tf.cast(label_batch,tf.float32)*tf.log(y_conv+1e-9))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
tf.train.start_queue_runners(sess=sess)
correct_prediction=tf.equal(tf.argmax(y_conv,1), tf.argmax(label_batch,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
for i in range(100):
train_step.run(session=sess)
print(sess.run(accuracy))
print(sess.run(correct_prediction))
When I print the correct_prediction tensor, I get the following output no matter what.
[ True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True False False False False False False False False False
False False False False False False False False False False False False
False False False False False False False False False False False False
False False False False False False False False False False False False
False False False False False False False False False False False False
False False False False False False False False False False False False
False False False False False False]
The accuracy is always 0.5, as if the weights are not being updated at all. When I print weights after each training step, they remain unchanged. I think I have some coding error. Could it be that the network is training on the same image again and again? But even so, the weights must update. I have 150 training examples, with 75 belonging to each class.Could someone please point me in the right direction ?
EDIT:This is how I initialize weights
def weight_variable(shape,name):
initial = tf.truncated_normal(shape, stddev=0.5)
return tf.Variable(initial,name=name)
def bias_variable(shape,name):
initial = tf.constant(1.0, shape=shape)
return tf.Variable(initial,name=name)
Your network has some design flaws. Because of mathematical issues it is not a good idea to calculate the cross entropy yourself and apply a softmax on the output layer. If you are interested in the mathematics I could add this, if not stick to the Tensorflow explanation and method: tf.nn.softmax_cross_entropy_with_logits.
Have you already tried many different configurations? Depending on the complexity of your images a higher or lower kernel size and amount of feature maps might be a good idea. Generally, if your images are relatively homogeneous, a lot of rather similar information is added up and therefore the network has it harder to converge if you have many feature maps. Since you have only two output neurons I assume are images are not very complex?
The next thing is your dropout. You are always using a dropout of 0.5 but normally, for test/validation (like your accuracy prediction) you do not use dropout. In most cases you only use it for training. You can create a placeholder specifying your dropout rate and feed this sess.run.
Here is some example of my own:
h_fc_drop = tf.nn.dropout(h_fc, keep_prob)
(...)
accu, top1, top3, top5 = sess.run([accuracy, te_top1, te_top3, te_top5],
feed_dict={
x: teX[i: i + batch_size],
y: teY[i: i + batch_size]
keep_prob: 1.0
}
)
This lets Tensorflow calculate my equations for accuracy and topX error rate, while I feed in the test data input teX and the real labels teY for the output with a keep probability keep_prob of 1.0 for the dropout.
Despite this the initialization of your weights is really important in deep neural networks. Even if your design is sufficient for your kind of problem (this also has to be investigated) your network could refuse to learn, diverge or converge to 0 if your weights are not initialized properly. You did not add details to your initialization so you maybe want to look up Xavier initialization. This is an easy beginning for Xavier initialization.
Conclusively I can just encourage you to plot some weights, feature maps, the output over time and so on to get an idea of what your network is doing. Normally this helps a lot.
这篇关于Tensorflow - 具有自定义数据集的卷积神经网络不学习的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！