The general answer is that adopting Linux kernel drivers (inside your other kernel) requires following Linux conventions, kernel ABI & API, architecture, and pre- and post- conditions. While the system call interface (see syscalls(2)) is stable and at least upward compatible (a user-land executable working on Linux 2.2 kernel should be runnable on Linux 4.18 kernel, at least when that executable is self-contained so statically linked), the Linux kernel interface is not that stable (so a kernel driver for Linux 2.2 won't even compile on Linux 4.18). Of course, there are also licensing considerations (not everyone wants to work with GPLv2 code).
This adoption of Linux drivers is not always possible, and might not even always be desirable. Look into the microkernel approach with drivers in user space (e.g. GNU Hurd). Look also into the unikernel way of thinking (e.g. MirageOS).
The (old, unmaintained) OSkit research project aimed to wrap Linux drivers for re-use by other kernels.
If you want to make your experimental operating system, you should look into OSDEV wiki. You might even want to make something with a design very different of the Unix (or Windows) way. Actually most OSes today (Linux, Unix, Windows, MacOSX, Android, VMS) have borrowed the ideas of Multics. You could dare inventing something different. For example, you might not even have files but some other ways of persistence (like the experimental GrassHopper OS did). Be sure to read Operating Systems: Three Easy Pieces to understand the terminology of OSes.
You could also adopt the following approach: use (some almost unmodified) Linux kernel, and build everything else (in user space) in your own way (this is still extremely ambitious and will take you many years of work). FWIW, the Pliant system is going that route (and J.PItrat's CAIA system, which still requires a C compiler, etc...). You might then not even use a Linux libc but something else that you have written. Look for example into Bones, a Scheme compiler which runs just on the Linux kernel (without the libc).
You could in principle (with many years of work) make even your own display server (and design its protocol) similar in spirit to Xorg or Wayland and redesign from scratch your GUI above that.
If you are interested more in user interface and less in OS internals, you could work on your own desktop environment (with your own windows manager) on Linux. Then, you'll get a very different look and feel.
But don't forget a hard fact: a Linux distribution is today a very complex system (of more than ten billions lines of code). Your lifetime is not enough to reinvent that alone. And today's desktop or laptop hardware (and essential firmware, like UEFI) is much more complex than what it was when Linus started coding Linux (e.g. even your keyboard wants an USB subsystem).
My hope (but I am pessimistic) is that several academic research teams will today be allowed and funded to work on brand new operating systems ideas (and free software prototypes). This would require funding for more than three years. This sadly does not happen, at least not in Europe (e.g. H2020 research grants only last 3 years) and probably not in North America. Remember Brooks' law and his Mythical Man Month book (first written in 1975, still very actual)... So sadly, a new Linus (making from scratch a new OS) is improbable today, since systems software research is irrelevant (e.g. because a small academic team needs more than 5 years of funding to make an OS with new ideas: remember Brook's observation: "if one woman can make a baby in 9 months, 9 women cannot make a baby in 1 month").
