-2

I was wondering what it would take to intercept a PC's graphics card output or parts of it.
First of all I'm wondering if you could even intercept the output without increasing the latency between the graphics card and the monitor. Maybe by kind of duplicating the output?

Secondly I'm wondering if you could take an adruino, raspberry pi or similar to take the output and send it back to the computer via USB so that the computer could analyze the data in real time.

Let's say the output is 1080p @ 60Hz, but capturing 10 frames per second would be enough.
The throughput for that should be something like (1920*1080*10)/1000/1000 = 20.7 megabytes per second, I would guess.
That sounds like too much for an arduino... or does it?

Any ideas how something like that could be accomplished without spending hundreds of dollars?

Forivin
  • 127
  • 1
  • 2
  • 12
  • Leaving everything else aside, you only estimated at one byte per pixel. What type of "output" are you asking about? VGA? DVI? HDMI? Also, even if you only want 10fps, the card will still be pumping out at its own desired refresh rate. – Roger Rowland Jan 27 '16 at 11:33
  • Preferably HDMI, but if VGA, DVI or DP would be easier I would go with them. And yes sure they are still sending 6 times the frames that I actually want to capture, but maybe it would be easier for the micro-controller to just buffer 10 frames per second. – Forivin Jan 27 '16 at 11:42
  • Have you [read the HDMI spec](https://en.wikipedia.org/wiki/HDMI)? However, you can't buffer 10 frames per second if the frames are coming at you, say, 6 times faster than you can read - you have to read them at the speed they're arriving. – Roger Rowland Jan 27 '16 at 11:46
  • No, I haven't. I also haven't read the VGA spec, the DVI spec and the Display Port spec. I probably wouldn't understand it anyway. I mean only because I'm using an arduino to emulate a USB keyboard, doesn't mean that I know the whole USB spec. There are libraries to do it. – Forivin Jan 27 '16 at 12:01
  • 1
    Yes of course, but at least it would give you an idea of the speeds and data rates that you need to deal with. I would say that an Arduino wouldn't stand a chance, but maybe a RasPi would - I don't know. I'm trying to help you to understand your question ;-) – Roger Rowland Jan 27 '16 at 12:08
  • There are simple HDMI / DP splitters, to allow you get get a copy of the output. A RasPi or a Arduino have no chance I guess to receive the data as they have no hardware to do so. Maybe you could to it with an FPGA. But this is not easy at all – Botnic Jan 27 '16 at 12:10
  • What exactly are you trying to accomplish, and why can't you do it by reading out the contents of the graphics card's frame buffer directly? Yes, you can do what you describe with off-the-shelf hardware, such as an [FPGA evaluation board](http://zedboard.org/product/zedboard) plus an [HDMI I/O board](http://www.xilinx.com/products/boards-and-kits/1-44p3z5.html), but these will set you back several hundred dollars. – Dave Tweed Jan 27 '16 at 12:19
  • @Roger Rowland Oh, okay. Well, I'm not sure if I should read all that before I even know what I will end up using in the end. – Forivin Jan 27 '16 at 12:21
  • @Botnic That's cool. Is there maybe some kind of circuit that can filter out 80% of the frames? And what could I use to feed the frames back to the computer via USB? theoretically a capture card would probably work, but they are soo expensive. :/ – Forivin Jan 27 '16 at 12:21
  • Have a look at this bord: https://www.crowdsupply.com/numato-lab/opsis# or https://hdmi2usb.tv/home/ – Botnic Jan 27 '16 at 12:23
  • @DaveTweed I just want to get the video output of a graphics card and analyze it using software on the same computer. At first I tried using OS APIs to read that data, but that doesn't work for directx output etc. So I thought the only possible way would be to access the graphics card output at the display connector. I haven't heard about FPGA evaluation boards etc before. If that would be under a hundred dollars I would go with that instead. But which one should I use? – Forivin Jan 27 '16 at 12:30
  • @Botnic But that thing is 350 bucks... that's way too much. – Forivin Jan 27 '16 at 12:32
  • You will have a hard time to find a FPGA kit under 100$ that can handle DP/HDMI. (http://electronics.stackexchange.com/questions/4962/most-affordable-fpga-dev-kit-for-learning-vhdl-and-fpga-theory) – Botnic Jan 27 '16 at 12:35
  • I'd totally be okay with using VGA or DVI. – Forivin Jan 27 '16 at 12:37
  • Now that you've fessed up to what you are really trying to do, there's probably a much better way. Screen shots of DirectX CAN be made. See this discussion on Stackoverflow: http://stackoverflow.com/questions/1962142/take-screenshot-of-directx-full-screen-application – JRE Jan 27 '16 at 12:54
  • With my Stackoverflow hat on, and seeing you mention DirectX - if you're on Win8/10 have you looked at the [Desktop Duplication API](https://msdn.microsoft.com/en-gb/library/windows/desktop/hh404487%28v=vs.85%29.aspx)? – Roger Rowland Jan 27 '16 at 13:00
  • 3
    This is a classic example of an XY question. – JRE Jan 27 '16 at 13:06
  • @JRE Well, I'm talking about directx applications that I do not have any control over. And it's not always just directx... – Forivin Jan 27 '16 at 13:37
  • 1
    Yes, and you can get screenshots of directx programs from other programs. – JRE Jan 27 '16 at 13:39
  • The 10$ version of this if you're willing to accept loss of accuracy is a webcam pointed at a monitor. But the best solution is still trying to capture the display from software. Maybe FRAPS is part of the solution? – pjc50 Jan 27 '16 at 13:57
  • @JRE It doesn't really matter as it's not just directx. I'd like this work on any computer, even Linux ones btw. – Forivin Jan 27 '16 at 14:14
  • @pjc50 Webcam? Erm no. – Forivin Jan 27 '16 at 14:15
  • I think you are still going to have less work to do if you develop a program for the target OS than you would if you try to build a device to capture the video in hardware and then develope a program to communicate with that device. – JRE Jan 27 '16 at 14:26
  • Perhaps you should be more specific about what you mean by "analyze the data". Are you trying to copy or record protected content? Are you trying to do face recognition or text character recognition? – Dave Tweed Jan 27 '16 at 14:33
  • @Forivin if you're happy with a reduced framerate, maybe whatever you're trying to do could be accomplished with a reduced-quality version? *But you've not told us what the ultimate purpose of this is*. Anyway, I've given an answer with some example capture-capable chips. – pjc50 Jan 27 '16 at 14:54
  • Well, it's complicated. I want to create a cross platform tool that can automate processes by sending mouse, keyboard, controller inputs on a hardware level and analyze what's happening by analyzing the pixels on the screen. I know there are tools like AutoHotkey, that can do it on a software level and can do certain jobs much more reliably by accessing APIs, but there is no such thing as AutoHotkey that legitimately runs on any OS. And by any OS I mean Linux, windows, Mac, android, TVs, consoles..... – Forivin Jan 27 '16 at 16:07
  • Is there no way to build a curcuit that uses a certain pattern to put the video data together so that it can simply be streamed to a usb interface of a computer? So that only the computer has to do all the dirty work? – Forivin Jan 27 '16 at 17:22

2 Answers2

1

For completeness, you can get high-speed capture chips from which you might be able to build a suitable device. I have some boards which use the AD9883 to convert VGA to digital to drive an LCD panel. It has a successor which supports HDMI. However, building one of these things is not a beginner hobbyist job as it involves lots of fast signals.

More integrated devices are available: single-chip TV which can run Linux. A visual processor with three very different cores.

pjc50
  • 46,540
  • 4
  • 64
  • 126
0

In answer to your first question: Yes, it is possible to intercept the output of a graphics card without causing any significant latency, or signal degredation. In order to do this, you would simply need to select the adaptor (VGA, HDMI, DVI, DP, etc.) whose protocol you're most comfortable processing, then make a short M-F apator to plug in inline between the card and your monitor. In the adaptor (a short cable would work easiest), you'll want to attach a buffer amplifier to each signal wire you're wanting to copy, then run whatever connection you want from the buffer amplifiers to your signal recording/processing device. Of course, you will need an external power source (usb/wall wart/etc) to power the amplifiers. This approach is very similar in function & design to the "video bugs" that can be attached to security camera feeds, albeit more complex since you're "bugging" a multi-pin connector, rather than a singal signal-wire coax cable.

As for your second question, whatever recording/analyzing/processing device you use will have to be fast enough to read the full output of the graphics card, then separate out whichever frames you need.

Robherc KV5ROB
  • 5,088
  • 1
  • 10
  • 24