Ice

Ice

关注
发信
关注(28)粉丝(399)

WebRTC onicecandidate:仅使用sdpMid = audio获得ICE候选人，但不用于视频

本文介绍了WebRTC onicecandidate:仅使用sdpMid = audio获得ICE候选人，但不用于视频的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

使用的浏览器是Chrome ...我有调用方和接收方代码来生成SDP和ICE候选者.我得到了调用者代码，以使用sdpMid = video生成正确的SDP和ICE候选者，但对于接收者，我仅获得了为sdpMid = audio生成的ICE候选者.

更新:以下是更改后接收方的localSessionDescription SDP，如建议的那样: 

  v = 0o =-7912682607537349212 2 IN IP4 127.0.0.1s =-t = 0 0a = group:BUNDLE音频视频a = msid语义:WMS 9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFdm =音频9 UDP/TLS/RTP/SAVPF 111103104 9 0 8106105 13126c = IN IP4 0.0.0.0a = rtcp:9输入IP4 0.0.0.0a = ice-ufrag:0D1hLEwxnqReQosQa = ice-pwd:Nsc4EAtefrfgzTetHjJA5lsga =指纹:sha-256 6C:85:D8:33:D8:C6:CB:CE:D4:8E:B4:7A:C2:F5:2F:D0:67:04:25:B2:74:F9:C6:3A:2E:96:E6:56:E7:27:B0:F8a = setup:activea = mid:音频a = extmap:1 urn:ietf:params:rtp-hdrext:ssrc音频级别a = extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-timea = sendrecva = rtcp-muxa = rtpmap:111 opus/48000/2a = fmtp:111 minptime = 10;useinbandfec = 1a = rtpmap:103 ISAC/16000a = rtpmap:104 ISAC/32000a = rtpmap:9 G722/8000a = rtpmap:0 PCMU/8000a = rtpmap:8 PCMA/8000a = rtpmap:106 CN/32000a = rtpmap:105 CN/16000a = rtpmap:13 CN/8000a = rtpmap:126电话事件/8000a = maxptime:60a = ssrc:2958641119 cname:Iu8s16HLxglPDg9ka = ssrc:2958641119 msid:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd bb63739b-cca2-4aa5-90a6-cf4bbaa199afa = ssrc:2958641119 mslabel:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFda = ssrc:2958641119标签:bb63739b-cca2-4aa5-90a6-cf4bbaa199afm =视频9 UDP/TLS/RTP/SAVPF 10010111611796c = IN IP4 0.0.0.0a = rtcp:9输入IP4 0.0.0.0a = ice-ufrag:0D1hLEwxnqReQosQa = ice-pwd:Nsc4EAtefrfgzTetHjJA5lsga =指纹:sha-256 6C:85:D8:33:D8:C6:CB:CE:D4:8E:B4:7A:C2:F5:2F:D0:67:04:25:B2:74:F9:C6:3A:2E:96:E6:56:E7:27:B0:F8a = setup:activea = mid:videoa = extmap:2 urn:ietf:params:rtp-hdrext:toffseta = extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-timea = extmap:4缸:3gpp:视频方向a = sendrecva = rtcp-muxa = rtpmap:100 VP8/90000a = rtcp-fb:100 ccm冷杉a = rtcp-fb:100小a = rtcp-fb:100 nack plia = rtcp-fb:100 goog-remba = rtcp-fb:100 transport-cca = rtpmap:101 VP9/90000a = rtcp-fb:101 ccm冷杉a = rtcp-fb:101无效a = rtcp-fb:101 nack plia = rtcp-fb:101 goog-remba = rtcp-fb:101 transport-cca = rtpmap:116红色/90000a = rtpmap:117 ulpfec/90000a = rtpmap:96 rtx/90000a = fmtp:96 apt = 100a = ssrc-group:FID 3143004909 4248148453a = ssrc:3143004909 cname:Iu8s16HLxglPDg9ka = ssrc:3143004909 msid:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd 778ef702-e7fc-47ea-bb3a-477e0b4262baa = ssrc:3143004909 mslabel:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFda = ssrc:3143004909标签:778ef702-e7fc-47ea-bb3a-477e0b4262baa = ssrc:4248148453 cname:Iu8s16HLxglPDg9ka = ssrc:4248148453 msid:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd 778ef702-e7fc-47ea-bb3a-477e0b4262baa = ssrc:4248148453 mslabel:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFda = ssrc:4248148453标签:778ef702-e7fc-47ea-bb3a-477e0b4262ba

这是为相应的getUserMedia生成的，如下所示:

  navigator.getUserMedia({音频:true，视频:{宽度:1280，高度:720}}，...

ICE候选生成代码为:

  pc.onicecandidate =函数(事件){console.log("Generated Icecandidate:");console.log(event);...};

在console.log上，我看到ICE候选者，如下所示:

  RTCIceCandidate候选:候选人:211156821 1 udp 2122260223 192.168.1.5 41811 typ host generation 0 ufrag kV5Snl0LQhJlYujt"sdpMLineIndex:0sdpMid:音频"

不用说，我无法显示远程视频.我正在本地网络上尝试此操作，因此实际上甚至不需要STUN.

我想知道，为什么我没有sdpMid = video的任何ICE候选人.另外，在生成的四个ICE候选者中，三个ICE候选者的sdpMLineIndex:0，而一个ICE候选者的候选属性为null！

更新1 :我得到了最后一个问题的答案...候选属性为Null.注意:RTCPeerConnection.onicecandidate将被调用一次，其候选属性为空，以表示trick流ICE事件结束."此处解释.

在主叫方，我有10多个ICE候选人，其中一些带有音频，有些带有视频.

我要去哪里错了?

更新2 :这是接收器部分的代码，它不会为视频生成ICE候选者.我已经删除了身份验证和其他部分，仅专注于相关部分.我删除了ICE候选者的缓存并将其发送:

  $(document).ready(function(){var socket = io.connect();var pc = new RTCPeerConnection({"iceServers":[{"url":"stun:stun.l.google.com:19302"}]});pc.onicecandidate =函数(事件){socket.emit('candidateFromReceiver'，event.candidate);console.log(候选生成:");console.log(event.candidate);};pc.onaddstream = function(ev){stream = ev.stream;var video = $('#vid2');video.attr('src'，URL.createObjectURL(stream));video.onloadedmetadata = function(e){video.play();}};socket.on('connect'，function(){console.log(已连接套接字");});socket.on('candidateFromCaller'，function(data){pc.addIceCandidate(new RTCIceCandidate(data));});navigator.getUserMedia = navigator.getUserMedia ||navigator.webkitGetUserMedia ||navigator.mozGetUserMedia;如果(navigator.getUserMedia){navigator.getUserMedia({音频:true，视频:{宽度:1280，高度:720}}，函数(流){var video = $('#vid1');video.attr('src'，URL.createObjectURL(stream));video.onloadedmetadata = function(e){video.play();}pc.addStream(stream);}，错误);socket.on('sdpOffer'，function(data){var sdpOffer = new RTCSessionDescription(data.sdpOffer);pc.setRemoteDescription(sdpOffer，function(){pc.createAnswer(function(sdpAnswer){localSessionDescription =新的RTCSessionDescription(sdpAnswer);pc.setLocalDescription(localSessionDescription，function(){socket.emit('sdpAnswer'，localSessionDescription);}，错误);}， 错误);}，错误);});}函数错误(错误){console.log("ERROR !!!!");console.log(err);}});//document.ready函数结束

如果我在获取用户媒体后立即插入代码以生成要约(就像我在呼叫者代码中一样)，则生成的ICE候选对象也包括视频对象.当然，这只是用于测试其余的代码炸弹，正如预期的那样.

解决方案

(从评论中可以看出，您正在缓存ICE候选人.请不要这样做.我还怀疑时间安排问题可能是某些候选人流失的原因.)

Trickle ICE的全部要点是对候选人进行trick流，也就是说，在候选人出现后立即发送.

使用WebRTC，您的应用负责对等体之间的信令，这是时间敏感的.所以:

  1. 不迟于 setLocalDescription 成功回调中发送 pc.localDescription .
  2. 期望 pc.onicecandidate 在该回调之后立即开始触发.发送给他们.

双方都是正确的(要约和回答).您想在网上看到的是:

 要约，候选，候选，候选

和另一种方式:

 答案，候选，候选，候选

不应该做的事情:

  • 不要缓存ICE候选人.
  • 不要等到您得到答案，那只会浪费时间.
  • 当要约由于任何原因出现在接收端时，请不要延迟致电 setRemoteDescription ，否则它将不准备接收候选人.

更新2:

您的sdp说 a = recvonly 而不是 a = sendrecv ，这意味着接收者被辞职仅接收，不发送任何回报.可能导致以下两种情况之一:

  1. 呼叫者设置createOffer选项，例如 offerToReceiveVideo:false 和/或 offerToReceiveAudio:false .
  2. 接收器没有在 pc.setLocalDescription 之前及时调用 pc.addStream .

如果在 getUserMedia 和收到要约之间存在竞争，第二个可能发生.

更新3:

如果其他所有方法均失败，请与工作代码进行比较.我以前在其他答案中共享过一个交叉表演示，但它仅发送视频，没有收到任何视频.

这是该演示的修改版本，它只能从远程摄像机接收视频.和往常一样，在同一浏览器的两个标签中打开它.

请注意，在Firefox中，单击 Call 后，必须物理聚焦另一个选项卡，然后才能访问相机.

The browser used is Chrome... I have the caller and receiver code to generate SDP and ICE candidates.I get the caller code to generate proper SDP and ICE candidates with sdpMid=video but for the receiver, I am getting ICE candidates generated only for sdpMid=audio.

UPDATE: Here is the localSessionDescription SDP for the receiver after changes, as suggested:

 v=0
 o=- 7912682607537349212 2 IN IP4 127.0.0.1
 s=-
 t=0 0
 a=group:BUNDLE audio video
 a=msid-semantic: WMS 9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd
 m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126
 c=IN IP4 0.0.0.0
 a=rtcp:9 IN IP4 0.0.0.0
 a=ice-ufrag:0D1hLEwxnqReQosQ
 a=ice-pwd:Nsc4EAtefrfgzTetHjJA5lsg
 a=fingerprint:sha-256 6C:85:D8:33:D8:C6:CB:CE:D4:8E:B4:7A:C2:F5:2F:D0:67:04:25:B2:74:F9:C6:3A:2E:96:E6:56:E7:27:B0:F8
 a=setup:active
 a=mid:audio
 a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
 a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
 a=sendrecv
 a=rtcp-mux
 a=rtpmap:111 opus/48000/2
 a=fmtp:111 minptime=10; useinbandfec=1
 a=rtpmap:103 ISAC/16000
 a=rtpmap:104 ISAC/32000
 a=rtpmap:9 G722/8000
 a=rtpmap:0 PCMU/8000
 a=rtpmap:8 PCMA/8000
 a=rtpmap:106 CN/32000
 a=rtpmap:105 CN/16000
 a=rtpmap:13 CN/8000
 a=rtpmap:126 telephone-event/8000
 a=maxptime:60
 a=ssrc:2958641119 cname:Iu8s16HLxglPDg9k
 a=ssrc:2958641119 msid:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd bb63739b-cca2-4aa5-90a6-cf4bbaa199af
 a=ssrc:2958641119 mslabel:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd
 a=ssrc:2958641119 label:bb63739b-cca2-4aa5-90a6-cf4bbaa199af
 m=video 9 UDP/TLS/RTP/SAVPF 100 101 116 117 96
 c=IN IP4 0.0.0.0
 a=rtcp:9 IN IP4 0.0.0.0
 a=ice-ufrag:0D1hLEwxnqReQosQ
 a=ice-pwd:Nsc4EAtefrfgzTetHjJA5lsg
 a=fingerprint:sha-256 6C:85:D8:33:D8:C6:CB:CE:D4:8E:B4:7A:C2:F5:2F:D0:67:04:25:B2:74:F9:C6:3A:2E:96:E6:56:E7:27:B0:F8
 a=setup:active
 a=mid:video
 a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
 a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
 a=extmap:4 urn:3gpp:video-orientation
 a=sendrecv
 a=rtcp-mux
 a=rtpmap:100 VP8/90000
 a=rtcp-fb:100 ccm fir
 a=rtcp-fb:100 nack
 a=rtcp-fb:100 nack pli
 a=rtcp-fb:100 goog-remb
 a=rtcp-fb:100 transport-cc
 a=rtpmap:101 VP9/90000
 a=rtcp-fb:101 ccm fir
 a=rtcp-fb:101 nack
 a=rtcp-fb:101 nack pli
 a=rtcp-fb:101 goog-remb
 a=rtcp-fb:101 transport-cc
 a=rtpmap:116 red/90000
 a=rtpmap:117 ulpfec/90000
 a=rtpmap:96 rtx/90000
 a=fmtp:96 apt=100
 a=ssrc-group:FID 3143004909 4248148453
 a=ssrc:3143004909 cname:Iu8s16HLxglPDg9k
 a=ssrc:3143004909 msid:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd 778ef702-e7fc-47ea-bb3a-477e0b4262ba
 a=ssrc:3143004909 mslabel:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd
 a=ssrc:3143004909 label:778ef702-e7fc-47ea-bb3a-477e0b4262ba
 a=ssrc:4248148453 cname:Iu8s16HLxglPDg9k
 a=ssrc:4248148453 msid:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd 778ef702-e7fc-47ea-bb3a-477e0b4262ba
 a=ssrc:4248148453 mslabel:9f0MAtEwYGWY3pdBDI8ZtTu4dVu92R6IpEFd
 a=ssrc:4248148453 label:778ef702-e7fc-47ea-bb3a-477e0b4262ba

This is generated for the corresponding getUserMedia as in:

 navigator.getUserMedia({ audio: true, video: { width: 1280, height: 720 } },...

The ICE candidate generation code is:

pc.onicecandidate = function (event) {
   console.log("Generated Icecandidate:" );
   console.log(event);
   ...
 };

On console.log, I see ICE candidates, as in:

RTCIceCandidate
candidate: "candidate:211156821 1 udp 2122260223 192.168.1.5 41811 typ host generation 0 ufrag kV5Snl0LQhJlYujt"
sdpMLineIndex:0
sdpMid:"audio"

Needless to say, I am not able to get the remote video displayed. I am trying this on a local network so really even STUN is not required.

I would like to know, why I am not getting any ICE candidates for sdpMid=video. Also, of the four ICE candidates generated,three ICE candidates have a sdpMLineIndex:0 and one ICE candidate has candidate property as null!

UPDATE 1: I got the answer to the last issue... Candidate property as Null. "Note: the RTCPeerConnection.onicecandidate will be called once with an empty candidate property to signal the end of trickle ICE event." This is explained here.

On the caller side, I get more than 10 ICE candidates, some with audio and some with video.

Where am I going wrong?

UPDATE 2:Here is the code for the receiver part which does not generate ICE candidates for video. I have stripped authentication and other sections to focus only on the relevant part. I removed caching of ICE candidate and send it as it comes:

$(document).ready(function () {

  var socket = io.connect();
  var pc = new RTCPeerConnection ({
    "iceServers": [{"url": "stun:stun.l.google.com:19302"}]
  });

  pc.onicecandidate = function (event) {
    socket.emit('candidateFromReceiver',event.candidate);
    console.log("Candidate Generated:");
    console.log(event.candidate);
  }; 

  pc.onaddstream = function(ev) {
    stream = ev.stream;        
    var video = $('#vid2'); 
    video.attr('src', URL.createObjectURL(stream));
    video.onloadedmetadata = function(e) {
      video.play();
    }
  };

  socket.on('connect',function() { console.log("Socket connected"); });
  socket.on('candidateFromCaller', function (data) {
      pc.addIceCandidate(new RTCIceCandidate(data));
  });

  navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia ||
                       navigator.mozGetUserMedia;
  if (navigator.getUserMedia) {
    navigator.getUserMedia({ audio: true, video: { width: 1280, height: 720 } },
      function(stream) {
         var video = $('#vid1'); 
         video.attr('src', URL.createObjectURL(stream));
         video.onloadedmetadata = function(e) {
           video.play();
         }
     pc.addStream(stream);
      },error);

    socket.on('sdpOffer', function(data) {
      var sdpOffer = new RTCSessionDescription(data.sdpOffer);
      pc.setRemoteDescription(sdpOffer, function() {
        pc.createAnswer(function(sdpAnswer) {
          localSessionDescription = new RTCSessionDescription(sdpAnswer);
          pc.setLocalDescription(localSessionDescription, function() {
            socket.emit('sdpAnswer',localSessionDescription);
          },error);
        }, error);
      },error);
    });
  }

  function error(err) {
    console.log("ERROR!!!!");
    console.log(err);
  }

}); // End of document.ready function

If I insert code, just after getting user media, to generate an Offer (as I have in the caller code), the ICE candidates generated include ones for video too. Of course, that was only for testing as the rest of the code bombs after that, as expected.

解决方案

(It sounds from comments that you're caching ICE candidates. Don't do that. I also suspect timing issues may be behind loss of some candidates.)

The whole point of Trickle ICE is to trickle candidates, that is, send candidates as soon as they become available.

With WebRTC, your app is responsible for signaling between peers, which is time-sensitive. So:

  1. Send pc.localDescription no later than in the setLocalDescription success callback.
  2. Expect pc.onicecandidate to start firing immediately after that callback. Send them.

This is true on both sides (for offer and answer). What you want to see on the wire is:

offer, candidate, candidate, candidate

and the other way:

answer, candidate, candidate, candidate

What not to do:

  • Don't cache ICE candidates.
  • Don't wait until you have an answer back, that just wastes time.
  • Don't delay calling setRemoteDescription when an offer comes in on the receiving end for any reason, or it wont be ready to receive candidates.

Update 2:

Your sdp says a=recvonly and not a=sendrecv, which means the receiver is resigned to receive only, without send anything in return. One of two things can cause this:

  1. Caller set createOffer options like offerToReceiveVideo:false and/or offerToReceiveAudio:false.
  2. Receiver did not called pc.addStream in time for (before) pc.setLocalDescription.

The second can happen if there's a race between getUserMedia and receiving an offer.

Update 3:

If all else fails, compare to working code. I've shared a cross-tab demo before in other answers, but it only sent video, didn't receive any.

Here's a modified version of that demo that only receives video from the remote camera instead. As usual, open it in two tabs in the same browser.

Note that in Firefox, after you hit Call, you have to physically focus the other tab before it allows access to the camera.

这篇关于WebRTC onicecandidate:仅使用sdpMid = audio获得ICE候选人，但不用于视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

09-17 03:53
Powered by LMLPHP ©2024 Ice 0.005991