Chromium

Re lecrammulb@

1. Do you see glitches as well? I only can hear the pitch changes.
2. Could you provide a trace capture of your system based on https://web.dev/profiling-web-audio-apps-in-chrome ?

Thanks for looking into this.

1) I think the only misbehavior I hear is pitch changing. No glitches/artifacts/disruption to the audio other than pitch. But I should note that I first noticed this using audio buffers (music samples) and it's definitely a speed wavering/resampling kind of effect. I'll try to set up a test page of my own which demos the issue using audio buffers instead of an oscillator just for a different way to hear the issue and a more vanilla example without jquery.

2) Sure, trace attached. The issue repro'd for me the very first time I tried it today with the trace recording. This trace data reflects the following actions by me: I had http://lesgold.com/rtoy/fiddle.html already loaded in a tab, no other tabs open aside from chrome://tracing; started recording; clicked "Toggle Play" 4 times over about 5-10 seconds, during which I heard the pitch change; stopped recording.

Here's an alternate demo I made using audio buffers:

https://soundplant.org/wavesurfer/srcObject_demo.html

For me within about a minute of clicking "init" and interacting with the different sound buttons and gain slider, I start to hear the speed wavering both up and down. It often occurs just within the first few seconds of starting playback.

Also, I find that if I leave the tab open for a few minutes after initing but playing nothing, if I come back to the page and start sounds after around 5 minutes, the speed wavering is pretty much guaranteed to occur.

One other notable observation: once the pitch misbehavior starts, playback latency becomes notably higher, and highly variable from one sound play to the next.

Hi @Reporter:

As per above comments, can you please provide your feedback to triage this issue further.

Thanks for the help!

Hi @Reporter:

Gentle Ping! As per C#4, can you please provide your feedback to triage further.

Thanks for the help!

Hi @Reporter:

Please provide feedback as per C#4 which helps us triage this issue further.

NOTE: Issue may get closed if we don't see feedback within next 30 days.

Thanks for the help!

[Empty comment from Monorail migration]

I don't know if this bug is still a TBD or if the assumption is that Web Audio 2.0 will negate the need to fix it. I wrote an Electron app (Electron 12/Chromium 89) that uses createMediaStreamDestination() in combination with setSinkId() to enable user control over the audio output device of Web Audio API-generated audio, and this bug still rears its head occasionally. I wanted to post a couple of significant findings from living with this bug longer term with a largish user base:

1) My app has a "refresh rate" setting that sets the global fps of all animations which includes both DOM element animations as well as canvas draws. The default setting is 40fps to avoid pushing the cpu/gpu too much and leave enough headroom for audio processing. A couple of users have reported that if they encounter this pitch glitch, turning down the refresh rate to 30fps or 15fps seems to avoid the glitching entirely. This strongly suggests a link between intensive cpu/gpu use and this bug, though I want to emphasize that I can still trigger the bug in my above posted demo without any cpu stressing.

2) More recently I believe I have stumbled on a workaround to avoid the bug in most cases. It's an old audio dev's trick ;) which is to keep a bit of looping silence playing at all times. This, I guess, helps keep the connection to the output device open (?) and seems to avoid the bug, suggesting maybe it is more likely to occur after the audio output has entered some kind of internal sleep mode due to inactivity. I have created a new version of my demo with this workaround applied here:

https://soundplant.org/wavesurfer/srcObject_demo_silence.html

Note that I can still trigger the bug even in this version if I try really hard and do some weird stuff like clicking the "toggle connect/disconnect..." buttons repeatedly, but in real world use having this looping silence seems to greatly decrease the chances of encountering the bug.

Just wanted to mention this problem seems exacerbated by Windows 11. We were actually not aware of this issue until Win 11 was released and we had some users complain that sound got choppy / robotic when the app window was fully occluded by other windows. Apparently this is new behavior in Win 11 where extra power saving measures are applied to "backgrounded" apps. However, they state this should _not_ happen to apps that have audio playing. My hunch is that however Windows is detecting if audio is playing from the thread is failing, causing additional throttling. That, or it is the same bug and the more aggressive throttling is just making it more apparent. Or some combination of the two.

I just wanted to mention it in case it helps with anyones testing, or anyone else hits upon this. It took me a while of digging to find these issues. We are stuck in the same boat in that we must provide a way for the user to set the sink, so we are stuck with this workaround.

Side note: we tried a similar technique to the silence demo above, just initializing an audio buffer with all zeros. This did not appear to work for us. What did work was using a very small (read: quiet) value like 0.00001 which is inaudible to the user but tricks whatever silence detection is going on here. Maybe that is another hint for whoever is investigating this.

IMO the common factor here is the output is routed from a MediaStreamDestinationNode to a MediaElement, and while the context's AudioDestination outputs silence. https://crbug.com/chromium/1157478#c8 2) kind of confirms my suspicion above.

I'll keep this open and see if our team has bandwidth to investigate further.

I believe I am encountering this issue as well, but reproducible in a different way. For me, if I play an AudioBufferSourceNode to the AudioContext destination, then I wait a bit after the AudioBufferSourceNode has ended, and then I play another AudioBufferSourceNode, the second AudioBufferSourceNode sounds noticeably slowed down. The workaround in https://crbug.com/chromium/1157478#c8 (playing looping silence) works, but of course I'd love to not have to do that.

I can reliably reproduce this issue, but in different circumstances than the original report:

* OS: Linu
* Headphones are connected through bluetooth. I cannot reproduce the issue when using e.g. built-in speakers or headphones connected through the jack output.
* When starting the oscillator, the pitch is off for exactly 10 seconds, while also crackling/stuttering a lot. After that the pitch seems to drop to something slightly below the correct pitch, slowly rises to the correct pitch, and subsequently stays stable.

I have attached an audio recording.

I can confirm observations by https://crbug.com/chromium/1157478#c8 and https://crbug.com/chromium/1157478#c10: directly streaming to the HtmlAudioElement results in clicks and crackles (starting after a few seconds). One simple workaround is to send the same audio to both, the audioCtx.destination and the HtmlAudioElement, but add a gain node to the audioCtx path at 0.00001 (= -100 dB).

I can confirm I can reproduce using the same method as https://crbug.com/chromium/1157478#c12, with the fiddle given in https://crbug.com/chromium/1157478#c2.

WHen using a bluetooth headset connected to an android mobile device, pitch is much higher for 10 seconds with high amounts of crackling, before the crackling dies down but the pitch remains just slightly higher than it should.

On my own application, which is affected, the workaround in https://crbug.com/chromium/1157478#c13 does not seem to work for me.

[Empty comment from Monorail migration]

[Empty comment from Monorail migration]

Sorry for the wall of text, I just thought this information may be useful to someone after I spent a few hours trying to resolve this for our application.

I am back to being able to reproduce this issue, but it's not something that I can really re-create in a simple gist/test. I have been trying to root cause it to do so, but I think it is actually somehow intertwined with the RTC Peer Connection. This is for voice chat and our setup is pretty complex as we need to a) allow per-user volume, b) global volume control, and c) specifying the output device. To do this we create a MediaStream from each PC track, connect it into an individual gain node, connect all of those into a global gain node, and then send it to a MediaStreamDestination node, and finally into an `<audio/>` element where we can set a sink. We do this all in a dedicated audio context with `balanced` latency setting.

What we have noticed is that there is no distortion on the first connection to a call, but distortion occurs for a few seconds when a user has disconnected and re-connected to the call. The distortion is not present when we send everything to the default destination of the audio context and bypass the `<audio/>` element. And the distortion is not present in a "raw" stream which just has all the PC tracks added it to at and output via an `<audio/>` tag. I believe this eliminates both WebAudio or RTC behavior as root causes. It only occurs when combining the RTC audio with a MediaStreamDestination node.

Next I tried performing all our setup/teardown logic (all the things that occur to create this audio pipeline when a user joins a call) without actually disconnecting from the call and again I was unable to recreate the behavior. I ran all the same cleanup/disposal code that runs at disconnect, and then ran all the same code that occurs to re-add the tracks on connect and the issue does not occur. I can only get it to occur when the user actually disconnects and re-connects (establishing a new Peer Connection).

Lastly, I found this does not occur when we use a new `<audio/>` element on each re-connect. Up until this point we had just always left our `<audio/>` element mounted to prevent some re-renders when disconnecting/reconnecting. This seemed fine as ultimately all we cared about was the inputs to the WebAudio graph. Once initialized we left the entire pipeline in place and only modified the incoming streams/gain nodes from each individual participant. Disconnecting was just removing all of those, but leaving everything else (global gain node, destination node, and audio element) in place for the next call. Our solution for now is just to conditionally render the target `<audio/>` element only when the user is connected to a call and remove it when they disconnect. With no other changes this appears to have completely solved our problem.

It's a lot of disparate pieces of information, but hopefully it is of use to someone or maybe gives one of the developers a hint in the right direction of where the problem may lie. My best "hunch" is that maybe there is some sort of stale buffered data or a bad timestamp in the MediaStream and the distortion is basically the MediaStream trying to match the timestamps coming from the AudioContext. This is probably wildly inaccurate as I have only the most rudimentary understanding of how this "bridge" works.

The good news is `setSinkId` for WebAudio appears to be shipping in 110. That means I can finally remove this mess completely from our application and just directly set the sink without having to use `<audio/>` elements at all! Very excited for that and it seems like it may just make this entire issue moot at that point.

[Empty comment from Monorail migration]

[Empty comment from Monorail migration]

We just reproduced this on MacOS 13.3.1, Chrome M112

While we should fix this, there are some potential workarounds based on the application:

1) Use AudioContext.setSinkID instead of MediaStream if the goal is to set the output device

2) Don't disconnect the gainNode, since disconnecting the node causes graph processing to stop

3) Send a small DC offset or very-low-amplitude noise which will also not allow graph processing to stop

This issue was migrated from crbug.com/chromium/1157478?no_tracker_redirect=1

[Multiple monorail components: Blink>Media>Audio, Blink>WebAudio]
[Monorail mergedwith: crbug.com/chromium/1399630, crbug.com/chromium/1430546, crbug.com/chromium/1441941]
[Monorail components added to Component Tags custom field.]

Happy new years later, here are my notes about this bug.

Working with WebAudio + main Audio element + createMediaStreamDestination + player Audio element + createMediaElementSource to preserve mobile background audio playback.

- bluetooth might be the source of this bug, no matter OS or audio device sample rate
- no more pitch glitch on iOs when forcing AudioContext sample rate
- still happening with latest Chromium on Windows 11
- works fine with other Chromium based browsers
- preservesPitch breaks native track seeking feature from lockscreen or control center on iOs
- playing silence forever does not change anything
- linearRampToValueAtTime is not stable on iOs and random Androids
- good luck fading audio in or out in background
- using two audio elements breaks background playback
- also breaks MediaSession on random devices
- this bug is related to an other random audio glitch on mobile devices, after some time sound gets distorted / saturated

It is very hard to reproduce with simple examples. On desktop it happens (sometimes) when browser loses focus, on mobile devices (sometimes) when changing tracks from native lockscreen / control center buttons, or when receiving a phone call, or when browser gets focus, or just over time.

I'm sure there is a solution :) happy to talk about it.

Here is an example. It needs much more testing and improvements.
No more pitch change on tested mobile devices with various audio files (mp3, flac 192/24, wav)


https://github.com/nicopowa/audiotest