Google Deep Dream艺术:如何在神经网络中选择一个层并对其进行增强

如果您阅读 1 ，[2]，[3]，[4]从您的链接中，您会看到他们使用了Caffe.该框架已经包含训练有素的网络.您无需手动训练任何东西，只需使用models/文件夹中的.sh脚本下载模型.您想要即插即用过程"，这并不是一件容易的事，因为除了框架之外，我们还需要他们使用的脚本代码，并可能需要对Caffe进行补丁.我试图用他们的描述做点什么. Caffe具有Python和Matlab接口，但其内部结构更多. 以下文本描述了我对如何实现它的想法.我不确定自己的话，因此更像是邀请我一起研究，而不是即插即用过程".但是由于没有人回答，让我把它放在这里.也许有人会修理我.所以据我了解，他们进行了优化 [sum((net.forwardTo(X, n) - enchanced_layer).^2) + lambda * R(X)] -> min 即寻找这样的输入X，以使该网络字的特定层将生成引人入胜"的数据，而不是原始"数据.有一个正则化约束R(X):X应该看起来像自然图像"(没有高频噪声). X是我们的目标图像.初始点X0是原始图像.当我们用X馈入输入时，forwardTo(X, n)是我们的网络在n层中产生的.如果谈到Caffe，则可以进行正向传递(net.forward)并查看您感兴趣的blob(net.blob_vec(n).get_data()). enchanced_layer-我们在其中获取原始图层blob和吸引力"信号.这是什么意思，我不知道.也许他们只是将这些值乘以系数，也许还有其他原因.当输入图像在图层n中产生所需的图像时，sum((forwardTo(X, n) - enchanced_net).^2)将变为零. lambda是正则化参数，而R(X)是X看起来自然的样子.我没有实现它，我的结果看起来非常嘈杂.至于公式，您可以在[2]处查找.我用Matlab和fminlbfgs进行了优化.关键是要找到上述公式的梯度，因为问题的维数太大，无法用数字方法计算梯度.正如我所说，我没有找到R(X)的梯度.至于公式的主要部分，我设法通过以下方式找到它:将图层n上的diff blob设置为forwardTo(X, n) - enchanced_net. (请参阅caffe文档中的set_diff和set_data，set_data用于正向并等待数据，而set_diff用于向后传播并等待数据错误).从图层n-1到输入执行部分反向传播.输入diff blob将包含我们需要的渐变. Python和Matlab接口不包含部分向后传播，但是Caffe C ++内部构件包含它.我在下面添加了一个补丁，使其可以在Matlab中使用.增强第四层的结果: 我对结果不满意，但我认为本文有一些共同之处. 这是在原样"上方产生图片的代码.入口点是"run2.m"，"fit2.m"包含健身功能: https://github.com /galchinsky/caf 这里是Matlab界面的caffe补丁，可提供部分反向传播: https://gist.github.com/匿名/53d7cb44c072ae6320ff I am interested in a recent blog post by Google that describes the use of nn to make art. I am particularly interested in one technique: The post is http://googleresearch.blogspot.co.uk/2015/06/inceptionism-going-deeper-into-neural.html?m=1. My question: the post describes this as a 'simple' case--is there an open-source implementation of a nn that could be used for this purpose in a relatively plug-and-play process? For just the technique described, does the network need to be trained? No doubt for other techniques mentioned in the paper one needs a network already trained on a large number of images, but for the one I've described is there already some kind of open-source network layer visualization package? 解决方案 UPD: Google posted more detail instructions how they implemented it: https://github.com/google/deepdream/blob/master/dream.ipynbThere's also another project: https://317070.github.io/Dream/If you read 1,[2],[3],[4] from your link, you'll see that they used Caffe. This framework already contains the trained networks to play with. You don't need to train anything manually, just download the models using .sh scripts in the models/ folder.You want "plug-and-play process", it's not so easy because besides the framework, we need the code of the scripts they used and, probably, patch Caffe. I tried to make something using their description. Caffe has Python and Matlab interface but there's more in its internals. The text below describes my thoughts on how it could be possibly implemented. I'm not sure about my words so it's more like an invitation to research with me than the "plug-and-play process". But as no one still answered, let me put it here. Maybe someone will fix me.SoAs far as I understand, they run optimization[sum((net.forwardTo(X, n) - enchanced_layer).^2) + lambda * R(X)] -> minI.e. look for such input X so that the particular layer of the netword would produce the "enchanced" data instead of the "original" data.There's a regularization constraint R(X): X should look like "natural image" (without high-frequency noise).X is our target image. The initial point X0 is the original image.forwardTo(X, n) is what our network produces in the layer n when we feed the input with X. If speak about Caffe, you can make full-forward pass (net.forward) and look at the blob you are interested in (net.blob_vec(n).get_data()).enchanced_layer - we take the original layer blob and "enchance" signals in it. What does it mean, I don't know. Maybe they just multiply the values by coefficient, maybe something else.Thus sum((forwardTo(X, n) - enchanced_net).^2) will become zero when your input image produces exactly what you want in the layer n.lambda is the regularization parameter and R(X) is how X looks natural. I didn't implement it and my results look very noisy. As for it's formula, you can look for it at [2].I used Matlab and fminlbfgs to optimize.The key part was to find the gradient of the formula above because the problem has too many dimensions to calculate the gradient numerically.As I said, I didn't manage to find the gradient of R(X). As for the main part of the formula, I managed to find it this way:Set diff blob at the layer n to forwardTo(X, n) - enchanced_net. (see caffe documentation for set_diff and set_data, set_data is used for forward and waits for data and set_diff is used for backward propagation and waits for data errors).Perform partial backpropagation from layer n-1 to the input.Input diff blob would contain the gradient we need.Python and Matlab interfaces do NOT contain partial backward propagation but Caffe C++ internals contain it. I added a patch below to make it available in Matlab.Result of enhancing the 4th layer:I'm not happy with the results but I think there's something in common with the article. Here's the code that produces the picture above "as is". The entry point is "run2.m", "fit2.m" contains the fitness function: https://github.com/galchinsky/cafHere's caffe patch to Matlab interface to make partial backpropagation available: https://gist.github.com/anonymous/53d7cb44c072ae6320ff 这篇关于Google Deep Dream艺术:如何在神经网络中选择一个层并对其进行增强的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！