问题描述
我试图使用新的HttpClient类（在.NET 4.5），以从服务器中检索部分反应，以检查内容。我需要限制检索到的在HTTP请求的内容的前几个字节的数据的大小来限制带宽使用。
I'm trying to use the new HttpClient class (in .NET 4.5) to retrieve partial responses from the server in order to check the content. I need to limit the size of data retrieved to the first few bytes of content in the HTTP requests to limit the bandwidth usage.
我已经无法做到这一点。我已经使用GetAsync（URL，HttpCompletionOption.ResponseHeadersRead），然后使用Content.ReadAsStream（），企图只读标题，然后在一小块读取响应流尝试。我还试图GetStreamAsync（），然后用一小块（1000字节）读取的内容流。
I've been unable to accomplish this. I have tried using GetAsync(url, HttpCompletionOption.ResponseHeadersRead) then use Content.ReadAsStream() in an attempt to only read the headers and then read the response stream in a small chunk. I also tried GetStreamAsync() and then reading the Content stream with a small chunk (1000 bytes).
在这两种情况下，似乎HttpClient的是拉动和缓冲整个HTTP响应，而不是仅仅读出请求的字节数从流
In both cases it appears that HttpClient is pulling and buffering the entire HTTP response rather than just reading the requested byte count from the stream.
起初，我使用招来监测数据，但意识到提琴手实际上可能造成对被代理的全部内容。我转向使用System.Net跟踪（这说明）：
Initially I was using Fiddler to monitor the data, but realized that Fiddler might actually be causing the entire content to be proxied. I switched to using System.Net tracing (which shows):
ConnectStream#6044116::ConnectStream(Buffered 16712 bytes.)
这是全尺寸的，而不是仅仅读了1000字节。我也双冠王Wireshark的检查，以确认确实正在拉过电线的全部内容，它是。对于较大的内容（如110K链接）我的TCP / IP流被截断之前，获取有关20K的数据。
which is the full size rather than just the 1000 bytes read. I've also double checked in Wireshark to verify that indeed the the full content is being pulled over the wire and it is. With larger content (like a 110k link) I get about 20k of data before the TCP/IP stream is truncated.
这两种方法我试过读取数据：
The two ways I've tried to read the data:
response = await client.GetAsync(site.Url, HttpCompletionOption.ResponseHeadersRead);
var stream = await response.Content.ReadAsStreamAsync();
var buffer = new byte[1000];
var count = await stream.ReadAsync(buffer, 0, buffer.Length);
response.Close() // close ASAP
result.LastResponse = Encoding.UTF8.GetString(buffer);
和
var stream = await client.GetStreamAsync(site.Url);
var buffer = new byte[1000];
var count = await stream.ReadAsync(buffer, 0, buffer.Length);
result.LastResponse = Encoding.UTF8.GetString(buffer);
他们都产生几乎相同的.NET跟踪的，其中包括缓冲读。
Both of them produce nearly identical .NET trace's which include the buffered read.
时有可能有HttpClient的实际只读一小块的HTTP Repsonse的，而不是为了不使用全带宽整个响应？ IOW是有办法使用禁用的HTTP连接任何缓冲或者HttpClient的或HttpWebRequest的？
Is it possible to have HttpClient actually read only a small chunk of an Http Repsonse, rather than the entire response in order to not use the full bandwidth? IOW is there a way to disable any buffering on the HTTP connection using either HttpClient or HttpWebRequest?
更新：经过一些更广泛的测试，它看起来既像的HttpClient和HttpWebRequest的缓冲区中的第几个TCP / IP帧 - presumably保证HTTP头被捕获。所以，如果你返回一个足够小的要求，也容易加载得到完全只是因为它是在inital bufferred阅读。但是，加载一个较大的内容的URL时，内容也被截断。对于HttpClient的是20K左右，对于HttpWebRequest的8K左右的地方对我来说。
Update:After some more extensive testing it looks like both HttpClient and HttpWebRequest buffer the first few TCP/IP frames - presumably to ensure the HTTP header is captured. So if you return a small enough request, it tends to get loaded completely just because it's in that inital bufferred read. But when loading a larger content url, the content does get truncated. For HttpClient it's around 20k, for HttpWebRequest somewhere around 8k for me.
使用TcpClient的不具有任何缓冲的问题。当使用它，我得到读在读加一点额外的最接近的缓冲区大小重叠的大小的内容，但也不包括HTTP标头。使用TcpClient的是不是一个真正的选择对我来说，我们必须处理SSL，重定向，验证，分块的内容等。在这一点上我正在寻求实现完全自定义的HTTP客户端只需打开缓冲。
Using TcpClient doesn't have any buffering issues. When using it I get content read at the size of the read plus a bit extra for the nearest buffer size overlap, but that does include the HTTP header. Using TcpClient is not really an option for me as we have to deal with SSL, Redirects, Auth, Chunked content etc. At that point I'd be looking at implementing a full custom HTTP client just to turn of buffering.
推荐答案
才达到你需要做的最好的方法是类似如下：
The best way to achive what you need to do is something like the following:
using System;
using System.Net.Sockets;
namespace tcpclienttest
{
class Program
{
static byte[] GetData(string server, string pageName, int byteCount, out int actualByteCountRecieved)
{
const int port = 80;
TcpClient client = new TcpClient(server, port);
string fullRequest = "GET " + pageName + " HTTP/1.1\nHost: " + server + "\n\n";
byte[] outputData = System.Text.Encoding.ASCII.GetBytes(fullRequest);
NetworkStream stream = client.GetStream();
stream.Write(outputData, 0, outputData.Length);
byte[] inputData = new Byte[byteCount];
actualByteCountRecieved = stream.Read(inputData, 0, byteCount);
// If you want the data as a string, set the function return type to a string
// return 'responseData' rather than 'inputData'
// and uncomment the next 2 lines
//string responseData = String.Empty;
//responseData = System.Text.Encoding.ASCII.GetString(inputData, 0, actualByteCountRecieved);
stream.Close();
client.Close();
return inputData;
}
static void Main(string[] args)
{
int actualCount;
const int requestedCount = 1024;
const string server = "myserver.mydomain.com"; // NOTE: NO Http:// or https:// bit, just domain or IP
const string page = "/folder/page.ext";
byte[] myPartialPage = GetData(server, page, requestedCount, out actualCount);
}
}
}
几点来但是请注意：
Couple of points to note however:
有没有错误处理在那里，所以你可能想将其包装都在一个try / catch或东西，以确保你弄个连接错误，超时未解决的IP解析等。
There's NO error handling in there, so you might want to wrap it all in a try/catch or something to make sure you get hold of any connection errors, timeouts, unsolved IP resolution etc.
怎么一回事，因为你处理的原始数据流，然后HTTP标头也在那里，所以你需要考虑这些。
Beacuse your dealing with the raw stream, then the HTTP headers are also in there, so you'll need to take them into account.
您可以在理论上，把一个回路之前主插口读，在保持抓取数据，直到你得到一个空白\ N于它自己的一条线，这将告诉你在哪里头端，那么你可以抓数据的实际数量，但因为我不知道你的服务器也谈我离开了这一点了： - ）
You could in theory, put a loop in just before the main socket read, in keep grabbing data until you get a blank \n on it's own in a line, that will tell you where the headers end, then you could grab your actual count of data, but since I don't know the server your talking too I left that bit out :-)
如果你复制/粘贴整个code成VS新的控制台项目是可运行的，因为它是，这样你就可以一步吧。
If you copy/Paste the entire code into a new console project in VS it's runnable as it is, so you can single step it.
据我所知HTTP客户端并不能使它的可用原料流给用户，而且即便如此，如果它这样做，因为它分配为流方面，它不太可能，你会有多大的控制权来算，我已经看着它之前放弃了。
As far as I know the HTTP client doesn't make it's raw stream available to the user, and even then if it did because it's allocated as a streaming connection it's not likely you would have much control over it's count, I've looked into it before and given up.
我用的时候，这codeA数字，它很适合我在类似案件中，其实我有坐和使用它，所以我可以看看谁的连接从我的WiFi适配器得到统计的显示器。
I've used this code a number of times and it works well for me in similar cases, in fact I have a monitor that sits and gets stats from my WiFi adapter using it so I can see who's connecting.
有任何疑问，请随时打我就在这里，或者ping我在twitter上我把手@shawty_ds（以防万一你失去了它）
Any questions, feel free to hit me up on here, or ping me on twitter my handle is @shawty_ds (just in case you lost it)
辣妹
这篇关于如何检索与System.Net.HttpClient部分响应的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！