Firstly, to drive a TFT screen doesn't take much in the way of processing power.
However, it does take a lot of memory and processing power to drive a TFT screen well.
You can interface pretty much any microcontroller to a TFT screen as long as the TFT screen has its own display controller chip in it. Most cheap "development board" style TFT screens (screens already mounted on a small PCB) have that kind of arrangement. They make interfacing easy, and there is usually one of two interface types - SPI or Parallel. SPI is slow, but only uses a handful of pins. Great for interfacing with small MCUs. Parallel needs around 20 pins to interface with it, but allows much faster access to the screen, and often much more flexible access.
The main thing though is memory - both flash and RAM. Most small TFT screens have their own framebuffer memory built in, so you don't need to worry about that, but if you want to manipulate images in memory before pushing them out (which makes for the smoothest screen refreshes) then you need enough RAM to cope with that. And any images you want to display have to come from somewhere. That could be an SD card (which is usually quite slow) or internal Flash memory (much faster), so the more of that you have the better.
You really don't need a full embedded Linux system just for controlling a TFT screen. Most of the time that just gets in the way.
I do a lot of work with TFT screens on PIC32 chips. The high end of the MX series (such as the PIC32MX795F512L) has both the IO capabilities for fast parallel interfacing (a hardware "Parallel Master Port") and just about enough RAM to allow you some graphical manipulation. The new MZ series though, as it's much faster and has considerably more RAM, is a much better candidate. Of course, they are comparable to the gazillion ARM chips that are out there these days, so you can do similar things with those.
Designing and implementing a full embedded Linux TFT display environment is no small job. Believe me, I have been working on one for the past 6 months now. About 10% is the circuit design. The other 90% is getting Linux built and configured in the right way for your system. And that's not counting trying to squeeze information out of Broadcom. They won't talk to me - they will only talk to my contractees, and relaying information back and forth is a PITA.
So if you really want to create a system with a SoC and Linux, copy the BBB not the Pi. TI are far more open and willing to help.