Xbox Cloud Gaming supports mouse and keyboard and latency improvements

Microsoft is preparing to add mouse and keyboard support to the Xbox Cloud Gaming (xCloud) service that streams Xbox games to TVs, PCs, mobile devices and more. The software giant teased the add-on earlier this year, and now it’s encouraging game developers to get ready for mouse and keyboard support and some major latency enhancements on Xbox Cloud Gaming soon.

“Xbox has been supporting keyboards and mice for a few years now, and we’re working on adding it to streaming for PC users,” explains Morgan Brown, a software engineer on Microsoft’s Xbox game streaming team. “But you can start adding it to your game right now, and the console keyboard and mouse users will appreciate it. It will light up in streaming when we’re done adding it.”

Microsoft Flight Simulator CEO Jorg Neumann has previously teased that the addition of mouse and keyboard support on Xbox Cloud Gaming may appear this summer. As Microsoft encourages developers to start thinking more about mouse and keyboard support for Xbox games streaming to the PC, it’s likely that we’ll start to see this emerge soon.

It will allow Xbox Cloud Gaming users to stream Xbox games, not PC games, using a mouse and keyboard. We could see games like The Sea of ​​Thieves, Minecraft, Halo InfiniteEven Fortnite all support mice and keyboards through Xbox Cloud Gaming. The list of Xbox games that support mice and keyboards is still relatively small. It will be especially useful when Microsoft expands the Xbox Cloud Gaming library later this year.

Xbox developers will get new APIs to improve the power delay.
Image: Microsoft

In addition to supporting mouse and keyboard, Microsoft also offers developers several ways to improve the power delay in their games. Microsoft has worked on a new Display Details API, which in total can save up to 72 ms waiting time. This is accomplished using Direct Capture, which reproduces hardware features in software to eliminate VSync latency and dual or triple buffering, and even the scaling needed for TVs.

Scaling and artifacts all add extra latency to game streaming, and many games already support Direct Capture to improve the performance of Xbox Cloud Gaming. Latency can drop to as low as 2-12 ms, compared to 8-74 ms through the traditional display pipeline. However, there are some limitations. Direct Capture only supports a maximum resolution of 1440p, and does not yet support dynamic resolution or HDR.

Resolution limitation will not be an issue for most game developers right now, as Xbox Cloud Gaming scales games down to 720p on mobile and 1080p on PC and online. Microsoft expects to eventually support higher resolutions, but there is no timeline for 1440p or 4K support for the new Xbox TV app. “This is something we expect to change over time, based on different devices, network conditions and power stack improvements,” explains Brown. Tools will soon be available for developers to test their games and find out how they support Direct Capture.

Direct Capture improves power delay in Xbox Cloud Gaming.
Image: Microsoft

Latency enhancements are the key to gaming streaming services like Xbox Cloud Gaming, and as Direct Capture shows, it’s not just about reducing network lag. Nvidia launched its RTX 3080 GeForce Now team last year, with impressive latency improvements. Nvidia built its own Adaptive Sync technology, which varies game rendering to match a synchronous display and allows GeForce Now to synchronize streamed games to any 60Hz or 120Hz display.

Nvidia’s Adaptive Sync also reduces some CPU-GPU caching on the server side, and the end result is some impressive latency improvements over what’s available from Google Stadium or Xbox Cloud Gaming. Nvidia even claims to beat an Xbox Series X that runs locally at 60 fps thanks to 120 fps GeForce Now support.