Separating a "wad of stuff" utility project into individual components with "optional" dependencies

Question

Over the years of using C#/.NET for a bunch of in-house projects, we've had one library grow organically into one huge wad of stuff. It's called "Util", and I'm sure many of you have seen one of these beasts in your careers.

Many parts of this library are very much standalone, and could be split up into separate projects (which we'd like to open-source). But there is one major problem that needs to be solved before these can be released as separate libraries. Basically, there are lots and lots of cases of what I might call "optional dependencies" between these libraries.

To explain this better, consider some of the modules that are good candidates to become stand-alone libraries. CommandLineParser is for parsing command lines. XmlClassify is for serializing classes to XML. PostBuildCheck performs checks on the compiled assembly and reports a compilation error if they fail. ConsoleColoredString is a library for colored string literals. Lingo is for translating user interfaces.

Each of those libraries can be used completely stand-alone, but if they are used together then there are useful extra features to be had. For example, both CommandLineParser and XmlClassify expose post-build checking functionality, which requires PostBuildCheck. Similarly, the CommandLineParser allows option documentation to be provided using the colored string literals, requiring ConsoleColoredString, and it supports translatable documentation via Lingo.

So the key distinction is that these are optional features. One can use a command line parser with plain, uncolored strings, without translating the documentation or performing any post-build checks. Or one could make the documentation translatable but still uncolored. Or both colored and translatable. Etc.

Looking through this "Util" library, I see that almost all potentially separable libraries have such optional features that tie them to other libraries. If I were to actually require those libraries as dependencies then this wad of stuff isn't really untangled at all: you'd still basically require all the libraries if you want to use just one.

Are there any established approaches to managing such optional dependencies in .NET?

Answer 1

Refactor Slowly.

Expect this to take some time to complete, and may occur over several iterations before you can completely remove your Utils assembly.

Overall Approach:

1. First take some time and think of how you want these utility assemblies to look when you're done. Don't worry too much about your existing code, think of the end goal. For example, you may wish to have:

   • MyCompany.Utilities.Core (Containing algorithms, logging, etc.)
   • MyCompany.Utilities.UI (Drawing code, etc.)
   • MyCompany.Utilities.UI.WinForms (System.Windows.Forms-related code, custom controls, etc.)
   • MyCompany.Utilities.UI.WPF (WPF-related code, MVVM base classes).
   • MyCompany.Utilities.Serialization (Serialization code).

2. Create empty projects for each of these projects, and create appropriate project references (UI references Core, UI.WinForms references UI), etc.

3. Move any of the low-hanging fruit (classes or methods that don't suffer from the dependency issues) from your Utils assembly to the new target assemblies.

4. Get a copy of NDepend and Martin Fowler's Refactoring to start analyzing your Utils assembly to start work on the tougher ones. Two techniques that will be helpful:

   • In many cases you will need to Invert Control through the means of interfaces, delegates, or events.
   • In the case of your "optional" dependencies, see below.

Handling Optional Interfaces

Either an assembly references another assembly, or it doesn't. The only other way to use functionality in a non-linked assembly is through an interface loaded through reflection from a common class. The downside to this is that your core assembly will need to contain interfaces for all of the shared features, but the upside is that you can deploy your utilities as needed without the "wad" of DLL files depending on each deployment scenario. Here's how I would handle this case, using the colored string as an example:

1. First, define the common interfaces in your core assembly:

   

   For example, the IStringColorer interface would look like:

    namespace MyCompany.Utilities.Core.OptionalInterfaces
    {
        public interface IStringColorer
        {
            string Decorate(string s);
        }
    }
   

2. Then, implement the interface in the assembly with the feature. For example, the StringColorer class would look like:

   using MyCompany.Utilities.Core.OptionalInterfaces;
   namespace MyCompany.Utilities.Console
   {
       class StringColorer : IStringColorer
       {
           #region IStringColorer Members
   
           public string Decorate(string s)
           {
               return "*" + s + "*";   //TODO: implement coloring
           }
   
           #endregion
       }
   }
   

3. Create a PluginFinder (or maybe InterfaceFinder is a better name in this case) class that can find interfaces from DLL files in the current folder. Here is a simplistic example. Per @EdWoodcock's advice (and I agree), when your projects grows I would suggest using one of the available Dependency Injection frameworks (Common Serivce Locator with Unity and Spring.NET come to mind) for a more robust implementation with more advanced "find me that feature" capabilities, otherwise known as the Service Locator Pattern. You can modify it to suit your needs.

   using System;
   using System.Linq;
   using System.IO;
   using System.Reflection;
   
   namespace UtilitiesCore
   {
       public static class PluginFinder
       {
           private static bool _loadedAssemblies;
   
           public static T FindInterface<T>() where T : class
           {
               if (!_loadedAssemblies)
                   LoadAssemblies();
   
               //TODO: improve the performance vastly by caching RuntimeTypeHandles
   
               foreach (Assembly assembly in AppDomain.CurrentDomain.GetAssemblies())
               {
                   foreach (Type type in assembly.GetTypes())
                   {
                       if (type.IsClass && typeof(T).IsAssignableFrom(type))
                           return Activator.CreateInstance(type) as T;
                   }
               }
   
               return null;
           }
   
           private static void LoadAssemblies()
           {
               foreach (FileInfo file in new DirectoryInfo(Directory.GetCurrentDirectory()).GetFiles())
               {
                   if (file.Extension != ".DLL")
                       continue;
   
                   if (!AppDomain.CurrentDomain.GetAssemblies().Any(a => a.Location == file.FullName))
                   {
                       try
                       {
                           //TODO: perhaps filter by certain known names
                           Assembly.LoadFrom(file.FullName);
                       }
                       catch { }
                   }
               }
           }
       }
   }
   

4. Lastly, use these interfaces in your other assemblies by calling the FindInterface method. Here is an example of CommandLineParser:

   static class CommandLineParser
   {
       public static string ParseCommandLine(string commandLine)
       {
           string parsedCommandLine = ParseInternal(commandLine);
   
           IStringColorer colorer = PluginFinder.FindInterface<IStringColorer>();
   
           if(colorer != null)
               parsedCommandLine = colorer.Decorate(parsedCommandLine);
   
           return parsedCommandLine;
       }
   
       private static string ParseInternal(string commandLine)
       {
           //TODO: implement parsing as desired
           return commandLine;
       }
   

   }

MOST IMPORTANTLY: Test, test, test between each change.

Answer 2

You could make use of interfaces declared in an additional library.

Try to resolve a contract (class via interface) using a dependency injection (MEF, Unity etc). If not found, set it to return a null instance.
Then check if the instance is null, in which case you don't do the extra functionalities.

This is especially easy to do with MEF, since it's the textbook use for it.

It would allow you to compile the libraries, at the cost of splitting them into n + 1 dlls.

HTH.

Answer 3

I thought I'd post the most viable option we've come up with so far, to see what the thoughts are.

Basically, we'd separate each component into a library with zero references; all the code that requires a reference will be placed into an #if/#endif block with the appropriate name. For example, the code in CommandLineParser that handles ConsoleColoredStrings would be placed into #if HAS_CONSOLE_COLORED_STRING.

Any solution that wishes to include just the CommandLineParser can easily do so, since there are no further dependencies. However, if the solution also includes the ConsoleColoredString project, the programmer now has the option to:

• add a reference in CommandLineParser to ConsoleColoredString
• add the HAS_CONSOLE_COLORED_STRING define to the CommandLineParser project file.

This would make the relevant functionality available.

There are several issues with this:

• This is a source-only solution; every consumer of the library must include it as a source code; they can't just include a binary (but this isn't an absolute requirement for us).
• The library project file of the library gets a couple of solution-specific edits, and it's not exactly obvious how this change is committed to SCM.

Rather un-pretty, but still, this is the closest we've come up with.

One further idea we considered was using project configurations rather than requiring the user to edit the library project file. But this is absolutely unworkable in VS2010 because it adds all project configurations to the solution unwantedly.

Answer 4

I'm going to recommend the book Brownfield Application Development in .Net. Two directly relevant chapters are 8 & 9. Chapter 8 talks about relayering your application, while chapter 9 talks about taming dependencies, inversion of control, and the impact this has on testing.

Answer 5

Full disclosure, I'm a Java guy. So I understand you're probably not looking for the technologies I'll mention here. But the problems are the same, so perhaps it'll point you in the right direction.

In Java, there are a number of build systems which support the idea of a centralized artifact repository that houses built "artifacts" - to my knowledge this is somewhat analogous to the GAC in .NET (please execuse my ignorance if it's a strained anaology) but more than that because it is used to produce independent repeatable builds at any point in time.

Anyway, another feature that is supported (in Maven, for instance) is the idea of an OPTIONAL dependency, then depending on specific versions or ranges and potentially excluding transitive dependencies. This sounds to me like what you're looking for, but I could be wrong. Take a look at this intro page on dependency management from Maven with a friend that knows Java and see if the problems sound familiar. This will allow you to build your application and construct it with or without having these dependencies available.

There are also constructs if you need truly dynamic, pluggable architecture; one technology that tries to address this form of runtime dependency resolution is OSGI. This is the engine behind Eclipse's plugin system. You'll see it can support optional dependencies and a minimum/maximum version range. This level of runtime modularity imposes a good number of constraints on you and how you develop. Most people can get by with the degree of modularity Maven provides.

One other possible idea that you could look into that might be orders of magnitude simpler to implement for you is to use a Pipes and Filters style of architecture. This is largely what has made UNIX such a long standing, successful ecosystem that has survived and evolved for half a century. Take a look at this article on Pipes and Filters in .NET for some ideas about how to implement this kind of pattern in your framework.

Answer 6

Perhaps the book "Large-scale C++ software design" by John Lakos is useful (of course C# and C++ or not the same, but you can distil useful techniques from the book).

Basically, re-factor and move the functionality that uses two or more libraries into a separate component that depends on these libraries. If needed, use techniques like opaque types etc.