2

I play the video game osu!. Osu! is a game that requires the player to quickly move the mouse pointer to different areas of the screen in a precise manner.

A game such as osu! lays out unique requirements for an input device:

  • It should be absolute (1-to-1 correspondence between position on screen and position of input device), unlike most computer mice. This is to avoid a problem known as "mouse drift", where the mouse drifts away from the central position of the mousepad after some time of gameplay.
  • It should have low latency, ideally less than a millisecond.
  • It should have high refresh rate, ideally around 1000hz.
  • It should have high (spacial) precision.

Such an input device only needs move the mouse cursor. Any mouse buttons or anything of that sort are not used.

Most top players today use a graphics tablet such as the Wacom CTH 480 and XP-PEN G640, normally used for drawing digital art. See this video as an example. Graphics tablets offer absolute input with high precision. However, most graphics tablets suffer from worse latency and refresh rate than conventional computer mice, and need to support things like pressure sensitivity and button presses—functionality that goes unused. I have not seen a graphics tablet that refreshes more than 240 times per second, while most have a refresh rate closer to 120hz. That's 4.16 and 8.33 milliseconds of latency respectively, on top of the "constant" latency.

My question is: What would be a feasible approach to creating a DIY input device that meets all four of the aforementioned requirements? And additionally, are there any reading materials that I should look into? For context, I'm an electronics newbie coming from a computer science background, who is quite comfortable with embedded development.

I have thought about three different possibilities. Perhaps the people here could envision more?

Conventional Graphics tablet

The best explaination I could find regarding how these work come from two Scanlime videos:

Perhaps some of you have better reading material? From what I've gathered these use a set of coils to sense changes in the magnetic field when the pen moves across the tablet surface. I wonder why these can't be made to refresh faster? Perhaps it's just because the pens need time to collect enough energy from the tablet for it to transmit back the button presses and such?

Magnetic tablet (grid of hall effect sensors)

These would use a grid of hall effect sensors to sense the presence of a magnet embedded in the tip of a pen's stylus. Examples projects using hall effect grids include:

Is my understanding correct that the refresh rate of this approach, along with conventional graphics tablet, is limited by the fact that you only have so many bits that can be fed to the microcontroller from the ADCs, resulting in the need to do time multiplexing where only certain subset of sensors are sampled at each time step? If so, couldn't the refresh rate be increased by using a larger microcontroller and multiple ADCs so that more sensors can be sampled at once?

Optical approach using laser pointer

I recently came across this video where a person which used a laser pointer combined with a web-cam to control the mouse. Thus I have the question, does there exist some kind of surface which can detect the position of a laser pointer, similar to a camera, which operates at high refresh rates and low latency? I'm imagining some kind of graphics tablet where the pen is replaced with a laser pointer.

Árni Dagur
  • 21
  • 1
  • 1
    Have you considered what latencies are even possible? USB 1 and 2 have a poll rate of 1ms, only if you go to USB 2 at 480Mbps you can get 0.125ms poll rate, but the amount of MCU you can make a DIY solution drops rapidly, as many don't have 480Mbps mode. Most likely detecting position at 1ms rate is not an issue. Also your display updates at 60Hz to 144Hz rate and it will have some lag and response time so there is no reason to update detected position more than that so every 7-16ms. And most likely it requires custom drivers. And still the OS and USB stack can limit what is possible. – Justme Sep 29 '20 at 07:01
  • Maybe it would also be interesting to find out what windows (for example) does with that signal. If you write a windows driver for your device you pretty much rely on the lower level windows hardware abstraction. I wonder how performant that is. – Stackoverflowsuperbad Sep 29 '20 at 11:19
  • @Justme USB gaming mice have a refresh rate of 1000hz and I know Linux polls my mouse every millisecond. I have a monitor which updates at 240hz, with <1ms response time. Having a refresh rate higher than the computer monitor is still beneficial for two reasons. 1) Osu! is played with muscle memory first and foremost, so you make adjustments based on what you feel with your hand, rather than what you see on the monitor. 2) More samples means that you can introduce smoothing to trade off excess latency for precision. I expect needing to write a custom Linux kernel driver, if not XOrg as well. – Árni Dagur Sep 29 '20 at 17:09
  • @Stackoverflowsuperbad This kind of ties into a question I posted to the Unix Stack Exhange a while back: https://unix.stackexchange.com/questions/531561/linux-how-is-the-input-latency-of-a-wacom-tablet-distributed-among-the-various From what I've gathered, the most important latency improvements you can get in the software side are a) Removing any software smoothing from drivers. For this purpose I use a patched version of libinput. b) Use a preemptable kernel. – Árni Dagur Sep 29 '20 at 17:12
  • @ÁrniDagur Refresh rate and panel response time still does not include processing lag, the time how much it takes from a pixel to come in at the monitor, and when it starts changing on screen. The last time I measured these, they varied significantly between different computer displays, professional studio reference monitors, and TVs. Although only HDMI interface was used for testing. – Justme Sep 29 '20 at 17:19

0 Answers0