Fixed
Status Update
No update yet.
Comments
br...@chromium.org
Able to reproduce the issue on Windows 7, Ubuntu 14.04 and Mac OS 10.10.4 using chrome Stable M44 - 44.0.2403.130 by following steps mentioned in https://crbug.com/chromium/517240#c0
Observed the video triggers stalled events often. This is a non-regression issue, Hence marking it as untriaged.
Observed the video triggers stalled events often. This is a non-regression issue, Hence marking it as untriaged.
pr...@chromium.org
No repro on Chrome Stable 44.0.2403.130 (linux). The video plays normally and no stalled events seen in the console.
In the attached repro video, the playback appears to be buffering a couple of times.
Could this be a server+network issue ?
tobias.patella@, Does this issue repro 100% ?
In the attached repro video, the playback appears to be buffering a couple of times.
Could this be a server+network issue ?
tobias.patella@, Does this issue repro 100% ?
[Deleted User] <[Deleted User]>
Yes, happening 100% even with "perfect" network conditions when no buffering occurs.
We've originally detected it with our own DASH player where it happens even more often (was reported by one one of our customers to us). With Shaka player it happens as well, but less frequently.
Happy to set up a demo page with our player if that helps.
We've originally detected it with our own DASH player where it happens even more often (was reported by one one of our customers to us). With Shaka player it happens as well, but less frequently.
Happy to set up a demo page with our player if that helps.
yi...@chromium.org
I tried on Mac 10.10 + Chrome 46.0.2486.0 dev build, I am able to repro this bug on both vp9 and mp4 video in shaka player, it repro on both encrypted and un-encrypted video source. as per #3, it repro on DASH player even more often. This bug NOT repro on playing imp4/vp9 video directly.
wolenetz@, can you take a look?
wolenetz@, can you take a look?
wo...@chromium.org
chcunningham@, this seems related (though not same scenario) to https://crbug.com/chromium/302729 and https://crbug.com/chromium/266592 . Please take a look. Feel free to assign back to me if you need.
sw...@gmail.com
Seeing a lot more of these events even though video playback is smooth. Is it that 'stalled' relates to video chunks that are not currently being rendered, but being downloaded/queued for playback?
In Chrome's implementation, does this event have anything to do with user perceived buffering? Any insights will help. Thanks!
In Chrome's implementation, does this event have anything to do with user perceived buffering? Any insights will help. Thanks!
ch...@chromium.org
Hi Swapna, I'm just getting to this bug, so its too early for me to say whether waiting is more reliable. Should know more in the coming days. (wolenetz@ free to chime in)
One foolproof (but not pretty) way to check buffering state would be to use a timer to periodically check the SourceBuffer.buffered ranges and compare to your currentTime. Hope this unblocks you - I'll update the bug as I learn more.
One foolproof (but not pretty) way to check buffering state would be to use a timer to periodically check the SourceBuffer.buffered ranges and compare to your currentTime. Hope this unblocks you - I'll update the bug as I learn more.
sw...@gmail.com
Thank you, chcunningham@. Could you please also comment on what html5 and/or other standard events/properties could one use to detect buffering in progressive streams? Also, SourceBuffer as a part of MSE might not be implemented or might not used for playback? Looking for a possible browser agnostic solution.
I will try your suggestion. Thanks again, appreciate the quick turn around.
I will try your suggestion. Thanks again, appreciate the quick turn around.
ch...@chromium.org
Small update: bug is still real. Confirmed repro using original reporter steps.
Swapna, stalled/progressing events sound like what you want :( - sorry they seem to be buggy. I'll root cause the issue and put out a fix. For just regular audio/video tags, you could check the "seekable" attribute to get the range of the currently buffered media. This range will update as more media is buffered.
Swapna, stalled/progressing events sound like what you want :( - sorry they seem to be buggy. I'll root cause the issue and put out a fix. For just regular audio/video tags, you could check the "seekable" attribute to get the range of the currently buffered media. This range will update as more media is buffered.
ch...@chromium.org
Also, I repro'd without any network issues (no buffering whole time) and I unchecked the 'adaptive' setting (cycle tracks no longer exists) - so its not triggered by adaptation.
sw...@gmail.com
What is a regular 'audio/video' tag? Do you mean to say that 'stalled' is relevant only for MSE based playback and we can ignore the event in other scenarios?
ch...@chromium.org
Hey swapna, by regular audio/video tag I just meant a media element that is not using MSE. What is your scenario? Are you using MSE?
Stalled is definitely relevant for non-MSE playback. For MSE, where the web app takes responsibility for fetching the media, its not clear to me that stalled should ever be emitted. Will know more soon.
Stalled is definitely relevant for non-MSE playback. For MSE, where the web app takes responsibility for fetching the media, its not clear to me that stalled should ever be emitted. Will know more soon.
ch...@chromium.org
+wolenetz, +foolip for proposal below.
Here is whats happening for MSE:
When we begin the loadResource algorithm, we start the progress timer. This timer calls back into WMPI every so often to see if things have progressed [0]. For MSE, we don't have a buffered_data_source_, so progress is entirely based on pipeline_.DidLoadingProgress() [1], which is true whenever the buffered ranges have updated since the last time we asked. So this is where the randomness comes in. If the progress timer fires twice without the app doing a new append, stalled is fired. AFAICT, the progress timer is never stopped in the case of MSE playbacks and the network state is never changed from LOADING (unless we hit an error).
So the question is, what is the right thing to do in the case of MSE. Given that the MSE spec describes "the download" or "bytes received" (in resource fetch algorithm) refer to data passed in via appendBuffer() and appendStream()", the download is really not a download at all. Its an append. And an append cannot stall. The resource fetch algorithm says its up to the UA when to stop the timer (suspend). IMO, for MSE, the timer should never be started. I'm not even sure it makes sense for the networkState to change from IDLE - but thats less clear. Reading between the lines of the spec, I propose for each append call, we:
1. change network state to loading
2. emit a single progress event
3. when append completes, change network state to idle & fire suspend
or, even simpler
1. leave network state idle
2. emit no progress/stalled/suspend events, because these aren't really sensible for MSE and the truth is we're not really doing any network loading at the time of the append, and the updateend event is a better way of knowing the append is complete vs the suspend event.
or perhaps some middle ground.
[0]https://cs.chromium.org/chromium/src/media/blink/webmediaplayer_impl.cc?rcl=1465226162&l=701
[1]https://cs.chromium.org/chromium/src/media/base/pipeline_impl.cc?rcl=1465226162&l=596
Here is whats happening for MSE:
When we begin the loadResource algorithm, we start the progress timer. This timer calls back into WMPI every so often to see if things have progressed [0]. For MSE, we don't have a buffered_data_source_, so progress is entirely based on pipeline_.DidLoadingProgress() [1], which is true whenever the buffered ranges have updated since the last time we asked. So this is where the randomness comes in. If the progress timer fires twice without the app doing a new append, stalled is fired. AFAICT, the progress timer is never stopped in the case of MSE playbacks and the network state is never changed from LOADING (unless we hit an error).
So the question is, what is the right thing to do in the case of MSE. Given that the MSE spec describes "the download" or "bytes received" (in resource fetch algorithm) refer to data passed in via appendBuffer() and appendStream()", the download is really not a download at all. Its an append. And an append cannot stall. The resource fetch algorithm says its up to the UA when to stop the timer (suspend). IMO, for MSE, the timer should never be started. I'm not even sure it makes sense for the networkState to change from IDLE - but thats less clear. Reading between the lines of the spec, I propose for each append call, we:
1. change network state to loading
2. emit a single progress event
3. when append completes, change network state to idle & fire suspend
or, even simpler
1. leave network state idle
2. emit no progress/stalled/suspend events, because these aren't really sensible for MSE and the truth is we're not really doing any network loading at the time of the append, and the updateend event is a better way of knowing the append is complete vs the suspend event.
or perhaps some middle ground.
[0]
[1]
ch...@chromium.org
[Comment Deleted]
ch...@chromium.org
(repasting last comment with typos fixed. typos version is now deleted)
After chatting with wolenetz@, it seems the conditions to emit stalled/progress events are not really clear for MSE. We haven't been able to dream up an implementation for 'stalled' in MSE that provides any added value. Apps are in charge of the fetch and should know clearly (while we cannot know) how their fetch is behaving.
We also considered having 'stalled' for MSE be a signal about playback (rather than fetching), but this is at odds with the HTML5 spec and playback stalls already have their own events and states (e.g. readyState and waiting event).
I'm going to make an MSE spec bug to propose that we do not emit 'stalled' for MSE. We have already dropped the 'suspend' event for MSE with similar rational - seehttps://github.com/w3c/media-source/issues/24 .
I'll leave networkState alone for now. I can see arguments for leaving it as LOADING and it doesn't affect my stance on 'stalled' events for MSE.
After chatting with wolenetz@, it seems the conditions to emit stalled/progress events are not really clear for MSE. We haven't been able to dream up an implementation for 'stalled' in MSE that provides any added value. Apps are in charge of the fetch and should know clearly (while we cannot know) how their fetch is behaving.
We also considered having 'stalled' for MSE be a signal about playback (rather than fetching), but this is at odds with the HTML5 spec and playback stalls already have their own events and states (e.g. readyState and waiting event).
I'm going to make an MSE spec bug to propose that we do not emit 'stalled' for MSE. We have already dropped the 'suspend' event for MSE with similar rational - see
I'll leave networkState alone for now. I can see arguments for leaving it as LOADING and it doesn't affect my stance on 'stalled' events for MSE.
wo...@chromium.org
@#15 I concur. Please update here with a link to that new MSE spec bug (to propose not emitting 'stalled' for MSE) once it's filed. Is your plan to include in the MSE spec bug a similar rationale for suppressing 'progress' for MSE?
ch...@chromium.org
spec bug is here https://github.com/w3c/media-source/issues/88
it recommends not firing both progress and stalled for MSE.
it recommends not firing both progress and stalled for MSE.
se...@chromium.org
[Empty comment from Monorail migration]
ja...@chromium.org
[Empty comment from Monorail migration]
sw...@gmail.com
Use-case: third-party video analytics and optimization solutions
g....@gmail.com
Hi chcunningham,
You suggestion to rely on buffered ranges and current time is not 100% guaranteed.
Here is an example that I have right now an audio stream only.
Ranges: [0.114693, 39.938321]
currentTime: 39.823628
The currentTime is < than end range.
You suggestion to rely on buffered ranges and current time is not 100% guaranteed.
Here is an example that I have right now an audio stream only.
Ranges: [0.114693, 39.938321]
currentTime: 39.823628
The currentTime is < than end range.
ch...@chromium.org
My suggestion to use currentTime and buffered ranges was flawed - I was confusing the "stalled" event for the "waiting" event. "stalled" doesn't actually have any dependency on currentTime - its entirely about how the download (in non-MSE use cases) is progressing.
What are you aiming to do? What are you expecting for currentTime w.r.t. buffered ranges?
What are you aiming to do? What are you expecting for currentTime w.r.t. buffered ranges?
ch...@chromium.org
After looking more closely, I realized the spec does not create a circumstance where MSE could fire the stalled event.
MSE sepc indicates resource fetch algorithm should be run with "mode" as "local".https://www.w3.org/TR/media-source/#mediasource-attach
This means we skip over all firing/scheduling of stalled event, which is only done for "If mode is remote" in the HTML5 Media element spec.https://www.w3.org/TR/html51/semantics-embedded-content.html#resource-fetch-algorithm
This was once unclear, but its actually been settled for some time (since HTML5.1)
Standby for patch to remove "stalled" firing for MSE
MSE sepc indicates resource fetch algorithm should be run with "mode" as "local".
This means we skip over all firing/scheduling of stalled event, which is only done for "If mode is remote" in the HTML5 Media element spec.
This was once unclear, but its actually been settled for some time (since HTML5.1)
Standby for patch to remove "stalled" firing for MSE
ch...@chromium.org
ch...@chromium.org
Also, this means no more periodic 'progress' timers, which are part of the same "remote" mode steps in resource fetch algorithm.
The spec has at least one case in "media data processing steps list" where 'progress' is fired. No change to this (assuming we already adhere...)
The spec has at least one case in "media data processing steps list" where 'progress' is fired. No change to this (assuming we already adhere...)
ch...@chromium.org
Actually, still debating on the 'progress' event. Firing it every 350ms is definitely not what MSE spec would have us do. But perhaps we could still fire it for changes to the buffered range via MSE (SourceBuffer::Append / Remove). This too is not what the spec intends, but I'm betting folks listen to "progress" as a signal to update buffered range UIs in custom media controls. Open to input.
ch...@chromium.org
After polling folks in the video-dev slack channel, many streaming sites do use 'progress' to update their custom buffering UI. Unlike 'stalled', progress does not fire at random - in Chrome its fired in response to chnages to the buffered range. From what I can tell, Safari, Edge, and Mozilla all behave the same. For this reason, no changes planned to 'progress' event.
The same video-dev slack folks indicate that they do *not* use 'stalled' with MSE. Plan is to go ahead and remove that event. Will send an intent to deprecate in the coming days (first trying to gather cross UA support).
The same video-dev slack folks indicate that they do *not* use 'stalled' with MSE. Plan is to go ahead and remove that event. Will send an intent to deprecate in the coming days (first trying to gather cross UA support).
bu...@chromium.org
The following revision refers to this bug:
https://chromium.googlesource.com/chromium/src.git/+/707b821eadc0940b40043f2c5afc9f4df98491f8
commit 707b821eadc0940b40043f2c5afc9f4df98491f8
Author: chcunningham <chcunningham@chromium.org>
Date: Tue May 29 08:42:19 2018
[MSE] Stop firing 'stalled' event.
See Intent to Deprecate
https://groups.google.com/a/chromium.org/d/msg/blink-dev/x54XtrTyOP8/4-5QZlZzDAAJ
MSE's resource is considered 'local' (we don't manage the donwload -
the app does), so the HTML5 spec text around firing 'stalled' does not
apply. See discussion inhttps://crbug.com/517240
No change to 'stalled' event for 'remote' (src="somefile.webm") resources.
This CL also stops delaying the 'load' event for deferred media player
Load() execution. Load() may be deferred for an arbitrary number of
seconds, so we should not hold up the window's 'load' event. Prior to
this change, 'stalled' was serving the unintended roll of unblocking the
the 'load' event for deferred load()s that took > 3 seconds. With stall
now removed for MSE, this weird behavior was uncovered and remedied.
Bug: 517240, 836951
Cq-Include-Trybots: master.tryserver.blink:linux_trusty_blink_rel;master.tryserver.chromium.linux:linux_layout_tests_slimming_paint_v2
Change-Id: I97f2bf972c79582d038e1215f5d630042656bd1c
Reviewed-on:https://chromium-review.googlesource.com/982564
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
Reviewed-by: Mounir Lamouri <mlamouri@chromium.org>
Reviewed-by: Avi Drissman <avi@chromium.org>
Reviewed-by: Luke Halliwell <halliwell@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#562371}
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/chrome/renderer/chrome_content_renderer_client.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/chrome/renderer/chrome_content_renderer_client.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/chromecast/renderer/cast_content_renderer_client.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/chromecast/renderer/cast_content_renderer_client.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/chromecast/renderer/cast_media_load_deferrer.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/chromecast/renderer/cast_media_load_deferrer.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/content/public/renderer/content_renderer_client.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/content/public/renderer/content_renderer_client.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/content/renderer/media/media_factory.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/content/renderer/media/stream/webmediaplayer_ms.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/content/renderer/media/stream/webmediaplayer_ms.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/content/renderer/media_capture_from_element/html_video_element_capturer_source_unittest.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/media/blink/webmediaplayer_impl.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/media/blink/webmediaplayer_impl.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/media/blink/webmediaplayer_params.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-append-buffer-expected.txt
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-append-buffer.html
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/blink/public/platform/web_media_player.h
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/blink/renderer/core/html/media/html_media_element.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/blink/renderer/core/html/media/html_media_element_test.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/blink/renderer/core/paint/video_painter_test.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/blink/renderer/platform/testing/empty_web_media_player.cc
[modify]https://crrev.com/707b821eadc0940b40043f2c5afc9f4df98491f8/third_party/blink/renderer/platform/testing/empty_web_media_player.h
commit 707b821eadc0940b40043f2c5afc9f4df98491f8
Author: chcunningham <chcunningham@chromium.org>
Date: Tue May 29 08:42:19 2018
[MSE] Stop firing 'stalled' event.
See Intent to Deprecate
MSE's resource is considered 'local' (we don't manage the donwload -
the app does), so the HTML5 spec text around firing 'stalled' does not
apply. See discussion in
No change to 'stalled' event for 'remote' (src="somefile.webm") resources.
This CL also stops delaying the 'load' event for deferred media player
Load() execution. Load() may be deferred for an arbitrary number of
seconds, so we should not hold up the window's 'load' event. Prior to
this change, 'stalled' was serving the unintended roll of unblocking the
the 'load' event for deferred load()s that took > 3 seconds. With stall
now removed for MSE, this weird behavior was uncovered and remedied.
Bug: 517240, 836951
Cq-Include-Trybots: master.tryserver.blink:linux_trusty_blink_rel;master.tryserver.chromium.linux:linux_layout_tests_slimming_paint_v2
Change-Id: I97f2bf972c79582d038e1215f5d630042656bd1c
Reviewed-on:
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
Reviewed-by: Mounir Lamouri <mlamouri@chromium.org>
Reviewed-by: Avi Drissman <avi@chromium.org>
Reviewed-by: Luke Halliwell <halliwell@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#562371}
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
is...@google.com
This issue was migrated from crbug.com/chromium/517240?no_tracker_redirect=1
[Monorail components added to Component Tags custom field.]
[Monorail components added to Component Tags custom field.]
Description
Example URL:
Steps to reproduce the problem:
1. Go to
2. Open dev console
3. Add event listener to video element:
document.getElementById('video').onstalled=function(){console.log('stalled')};
4. Load custom manifest:
4. Select 'Cycle Video Tracks'
5. Click 'Load Stream'
6. Watch video and keep eye on console output. Video element will randomly trigger stalled events, on average 4-5 over the period of the complete movie.
What is the expected behavior?
No stalled event should be fired if network conditions are good and all requests are loaded without issues.
What went wrong?
Stalled event is fired for no obvious reason.
Did this work before? N/A
Is it a problem with Flash or HTML5? HTML5
Does this work in other browsers? No IE 11, Edge
Chrome version: 44.0.2403.125 Channel: n/a
OS Version: OS X 10.10.4
Flash Version: Shockwave Flash 18.0 r0