I haven't developed any production applications for the Linux OS. There are a couple of things that bother me:
What do people do to handle all of these differences? Is there a way to abstract away from these differences? (BTW. I am missing anything?)
Of course you're missing some possibilities - the curse and blessing of open-source software is that anyone can create their own alternative, and many do - but you've listed the most important platforms and distribution formats.
So what do you do to be as compatible as possible? There are two broad options. The first is to distribute source code with compilation instructions. That means that you don't have to deal with any of this, but everyone who wants to use your software has to compile themselves. That is easy in principle, but in practice it amounts to he precondition that your users either have exactly the same set-up as you do, or that they learn how to obtain and compile all the necessary dependencies in the correct versions. That becomes more difficult the more complex your software is, and anyway many, many users, even of Linux, shy away from compiling things themselves because they consider that low-level and difficult.
The other is to become one of the libraries. That means adhering to a few packaging conventions and duplicating some work, both to comply with several packaging formats, and to stay current for each new OS release, but it means that any prospective user need only search for the (hopefully, memorable and informative) name of your software in their package manager and click 'yes'. If you want any kind of penetration of the non-geek market, that is the way to go.
I should add that there are (of course) innumerable variants and mixed strategies you can follow. For instance, you can aim to become an unofficial package. That basically means that you hope someone else will like your software enough to install it and them make an RPM or APT package and share it with others (of course, you can encourage or even help them with this if you want). That means that third-party users only have to register an additional repository, search for your software and then click "yes". The trade-offs are quite as you would expect: you still have to do this multiple times, and the result works only for one particular target, but the bulk of the work is done by someone else.
We do that at work. We do not want to distribute the source code, so binary-only distributions are the only choice. Our solution to this shared library version problem is to link most libraries statically. (Exception: libc is linked dynamically.) You can specify an individual library as /full/path/to/library.a. Please note that the linking order of statical libraries is more important than the linking order of dynamic libraries. Also note that the /full/path/to library may differ between various distributions. The full path in our case is a configurable option in the makefiles (we don't use GNU autotools for many good reasons). Also, note that CentOS, RHEL and Fedora do not offer statically linkable libraries, due to the opinion of Ulrich Drepper. So, the compilation machine cannot use an operating system in this list. Of course, the compiled binaries may be run on CentOS, RHEL and Fedora.
The .deb vs .rpm problem is solved by using tar.gz as the distribution package. You simply unpack it to a directory where you want the binaries and then start the program. All libraries in our case are statically linked, so there are no .so files in our distribution package. Thus, LD_LIBRARY_PATH environment variable is unnecessary.
Our graphical user interface is actually built neither on GTK nor on QT. It uses Java. This is an approach that can be considered for other applications, too. The benefit of Java is that the GUI can be run on Windows machines as well.
If you want to build a GUI application and there is no particular reason to use C or C++ as the language, I would advise you to heavily consider some JVM-based language. This means your application is useful on Windows as well. In our case, we need to process raw network packets at high performance using raw sockets in the main program, so it has to be implemented using C (or C++). But the GUI is a separate Java application.
Also, some people these days implement GUIs using languages such as Python. This is an option worth considering, too. Neither Python nor Java suffers from memory corruption due to invalid pointer usage, so they are in that sense clearly superior to C and C++. However, do note that using Python means you have to ship the source code.
different distros have different set of libraries, + Debian vs RPM packages
You know your "surface dependencies". So for each distro you need to find the packages that provide those dependencies (and document that), and to test that it does also bring all transitive dependencies (not just in terms of underlying libraries, but also in terms of "built with the right options" or whatever runtime-constraints might exist). Finding the set of packages is a one-time task that's not too complicated, but you'll have to test at least for every release, so a good test setup is necessary (good automation, and/or goot QA team).
But if you have a complex set of dependencies, it becomes likelier that you'll run into complex constraints about which version of what is needed, and sometimes you have to abandon some versions of some distros... or start addressing this by trying more difficult things that bring another set of complexity: compile yourself and ship your own glibc and libraries, or build statically-linked binaries, but you'd be opening a new can of worms... This will heavily depend on what you use to build your application. If you use C++ for example, you have a lot of other details to consider (the system compiler's ABI, kernel ABI...). From experience, the most awful thing is when you use several commercial libraries, that each have their own set of supported "platforms", and if you look closer you discover they incur weird constraints on what version of the compiler you can use, and you have to call them and negociate custom builds... (that experience was 15 years ago, maybe things are better now).
Docker can help simplify the infrastructure you need for tests. It can even allow you to skip all this, if you prefer to stick with 1 distro, and ship your app with a docker image that builds on that distro (which will work the same under all distros). Some server-side systems do that, but I've not seen that second option in the wild for desktop applications, so I'm sure there are other difficulties (guessing: displaying stuff must not be too complicated if you can pass $DISPLAY and the network settings allow it, but would copy-paste work? Drag-and-drop?...).
Of course if what you build is open source, better embrace each distro's system and become part of the ecosystem, as described in Kilian's answer.
Different UI - Gnome vs KDE
Side-note: each desktop environment (DE) comes with its own set of packages, and some of your dependencies might depend on packages that are only available with one DE. But usually whatever package you need is available to install, so let's say this is addressed by the "distros packages" section)
For the look and feel, if you want your app to "look native", again it depends on what you use. Your UI toolkit might be GTK-based, or Qt (there are Qt look-and-feels that can look like GTK apps -- this also depends on the distro and the packages, but for some common distros it might be worth documenting "install package so and so, go there in your settings, choose this...").
Or you might be using a "neutral" toolkit, and you don't care about that. But you'd still have "desktop integration" problems like "How do I open that web page with user's default browser?" or "Open that pdf file with the default PDF viewer". There are gnome-specific and KDE-specific commands for that, but there are also generic tools, I'd recommend xdg-open. You might run into funny edge cases, so you might want to offer the ability for the user to configure system commands for desktop integration, but with a robust base config that should work on 99% setups (See XDG standard and xdg-open)
Two approaches to the libraries & the UI issues:
For the packaging and distribution side of things you just need to bundle your software in multiple ways and arrange to distribute them in the appropriate manner(s) - use scripting or Jenkins in a master/slave arrangement for this.
You also need to address testing on multiple linux platforms - tools like Docker and Travis are your friends in this respect.
