Before thinking about the tools, I would recommend to have a look at the process.
With multiple projects managed by multiple teams, if you don't want the teams and projects tied to each other, from an organizational point of view it is usually best only to only reuse released versions of the libs of other team's projects, ideally of a certain quality. That minimizes the risk of getting unwanted "breaking changes" into your own project when the other team does further development for the library, driven by the requirements of their own project, but not yours.
In a .Net/Visual Studio environment, this means the released versions can be either published as source code and/or in precompiled form (as a debug-mode DLL, a PDB for debugging, and a release-mode DLL). The precompiled form brings you the advantage of reduced build times for everyone. It may be necessary if the compilation of the lib requires additional tools which are not used by every other team.
Assumed your teams/projects work in different repositories, you have basically two options to distribute and deploy those libs among the teams, when you want to make sure you can reliably version the source code with all its dependencies, and a fully automated build process:
Copy source code and precompiled DLLs/PDBs of a released library version into your own repo. This may not be resource efficient, as you noted in your question.
Have a central server with the released versions of the libs, and an automatism which pulls during a build the required version automatically from there.
#2 is where tools like Nuget are for (a private Nuget server, for example). Instead of polluting your own repo with the source and DLLs of tons of other projects/libraries, you simply add Nuget package references to your csproj
files, describing which lib and version you need. The Nuget server also knows about the meta data like version of the libs, compatible .Net framework versions, variants etc and lets you manage these things explicitly.
Of course, those internally developed libraries will have bugs and missing features, and often these issues will pop up the first time during reusage by a "foreign" project. Hence you also need a process for getting bug fixes and feature extensions back into the library development. How you do this precisely depends on your organization (for example, "do teams have write access to other team's repos?", "How do teams communicate at all?", or "do teams fix the reused libs first in their own fork of the lib"?) and the precise versioning and release strategy for each library (for example, "do teams fix bugs only in the latest version, or do teams provide patch releases to formerly released versions?").
However, bug fixing for foreign libs is usually more effort than fixing "own" libraries. That's why a certain quality of those libs is important. Automated tests can help to keep that quality above a certain level, and they can be also an instrument to communicate the bugs found by other teams, and avoid regression issues.