How do I manage a very large set of rules and magic numbers in my program?

Question

I'm somewhat new to programming (I'm a mechanical engineer by trade), and I'm developing a small program during my downtime that generates a (solidworks) part based on input from various people from around the plant.

Based on only a few inputs (6 to be exact), I need to make hundreds of API calls that can take up to a dozen parameters each; all generated by a set of rules I've gathered after interviewing everyone that handles the part. The rules and parameters section of my code is 250 lines and growing.

So, what is the best way to keep my code readable and manageable? How do I compartmentalize all my magic numbers, all the rules, algorithms, and procedural parts of the code? How do I deal with a very verbose and granular API?

My main goal is to be able to hand someone my source and have them understand what I was doing, without my input.

Answer 1

Based on what you describe, you're probably going to want to explore the wonderful world of databases. It sounds like many of the magic numbers you describe - particularly if they are part dependent - are really data, not code. You'll have much better luck, and find it far easier to extend the application in the long run, if you can categorize how the data relates to the parts and define a database structure for it.

Keep in mind, 'databases' don't necessarily mean MySQL or MS-SQL. How you store the data is going to depend a lot on how the program is used, how you are writing it, etc. It may mean an SQL type database, or it may simply mean a formatted text file.

Answer 2

Unless you anticipate extending this to multiple parts I'd be reluctant to add a database just yet. Having a database means a big pile of stuff to learn for you, and more stuff to install to get it to work for other people. Adding an embedded database keeps the final executable portable, but someone with your source code now has one more thing to get working.

I think a list of clearly named constants and rule-implementing functions will help a lot. If you give everything natural names and focus on literate programming techniques you should be able to make a readable program.

Ideally you'll end up with code that says:

LeftBearingHoleDepth = BearingWidth + HoleDepthTolerance;
if (not CheckPartWidth(LeftBearingHoleDepth, {other parameters})
    {whatever you need to adjust}

Depending on how local the constants are I'd be tempted to declare them in the functions they're used in where possible. It's quite useful to turn:

SomeAPICall(10,324.5, 1, 0.02, 6857);

into

const NumberOfOilDrainHoles = 10
const OilDrainHoleSpacing = 324.5
{etc}
SomeAPICall(NumberOfOilDrainHoles, OilDrainHoleSpacing, {etc}

That gives you largely self-documenting code and also encourages anyone who modifies the code to give similarly meaningful names to what they add. Starting local also makes it easier to deal with the total number of constants you'll accumulate. It gets a bit annoying if you have to keep scrolling through a long list of constants to make sure the value is the one you want.

One tip for names: put the most important word on the left. It may not read quite as well, but it makes finding things easier. Most of the time you're looking at a sump and wondering about the bolt, not looking at a bolt and wondering where it does, so call it SumpBoltThreadPitch not BoltThreadPitchSump. Then sort the list of constants. Later, to extract all the thread pitches you can get the list in a text editor and either use the find function, or use a tool like grep to return only the lines that contain "ThreadPitch".

Answer 3

I think your question reduces to: how do I structure a computation? Please notice you want to manage "a set of rules", which are code, and "a set of magic numbers", which are data. (You can see them as "data embedded in your code", but they are data nonetheless).

Furthermore, making your code "understandable to others" is in fact the general goal of all programming paradigms (see e.g. "Implementation Patterns" by Kent Beck, or "Clean Code" by Robert C. Martin for authors on software who state the same goal as you, for any program).

All hints in these books would apply to your question. Let me extract some hints specifically for "magic numbers" and "sets of rules":

1. Use named constants and Enumerations to replace magic numbers

   Example of constants:

   if (partWidth > 0.625) {
       // doSomeApiCall ...
   }
   return (partWidth - 0.625)
   

   should be replaced with a named constant so that no later changes can introduce a typo and break your code, e.g. by changing the first 0.625 but not the second.

   const double MAX_PART_WIDTH = 0.625;
   
   if (partWidth > MAX_PART_WIDTH) {
       // doSomeApiCall ...
   }
   return (partWidth - MAX_PART_WIDTH)
   

   Example of Enumerations:

   Enumerations can help you put together data that belong together. If you are using Java, remember that Enums are objects; their elements can hold data, and you can define methods that return all elements, or check some property. Here an Enum is used in constructing another Enum:

   public enum EnginePart {
       CYLINDER (100, Materials.STEEL),
       FLYWHEEL (120, Materials.STEEL),
       CRANKSHAFT (200, Materials.CARBON);
   
       private final double maxTemperature;
       private final Materials composition;
       private EnginePart(double maxTemperature, Materials composition) {
           this.maxTemperature = maxTemperature;
           this.composition = composition;
       }
   }
   
   public enum Materials {
       STEEL,
       CARBON
   }
   

   The advantage being: now nobody can wrongly define an EnginePart that is not made out of steel or carbon, and nobody can introduce an EnginePart called "asdfasdf", as would be the case if it was a string that would be checked on content.

2. The Strategy pattern and the Factory method pattern describe how to encapsulate "rules" and pass them to another object that makes use of them (in the case of the Factory pattern, the usage is building something; in the case of the Strategy pattern, the usage is whatever you want).

   Example of Factory method pattern:

   Imagine you have two types of Engines: one where each part has to be connected to the Compressor, and one where each part can be freely connected to whatever other parts. Adapted from Wikipedia

   public class EngineAssemblyLine {
       public EngineAssemblyLine() {
           EnginePart enginePart1 = makeEnginePart();
           EnginePart enginePart2 = makeEnginePart();
           enginePart1.connect(enginePart2);
           this.addEngine(engine1);
           this.addEngine(engine2);
       }
   
       protected Room makeEngine() {
           return new NormalEngine();
       }
   }
   

   And then in another class:

   public class CompressedEngineAssemblyLine extends EngineAssemblyLine {
       @Override
       protected Room makeRoom() {
           return new CompressedEngine();
       }
   }
   

   The interesting part is: now your AssemblyLine constructor is separated of what type of Engine it is handling. Maybe the addEngine methods are calling a remote API...

   Example of Strategy pattern:

   The Strategy pattern describes how to introduce a function into an object in order to change its behavior. Let us imagine you sometimes want to polish a part, sometimes you want to paint it, and by default you want to review its quality. This is a Python example, adapted from Stack Overflow

   class PartWithStrategy:
   
       def __init__(self, func=None) :
           if func:
               self.execute = func
   
       def execute(self):
           # ... call API of quality review ...
           print "Part will be reviewed"
   
   
   def polish():
       # ... call API of polishing department ...
       print "Part will be polished"
   
   
   def paint():
       # ... call API of painting department ...
       print "Part will be painted"
   
   if __name__ == "__main__" :
       strat0 = PartWithStrategy()
       strat1 = PartWithStrategy(polish)
       strat2 = PartWithStrategy(paint)
   
       strat0.execute()  # output is "Part will be reviewed"
       strat1.execute()  # output is "Part will be polished"
       strat2.execute()  # output is "Part will be painted"
   

   You may expand this to holding a list of Actions you want performed, and then calling them in turn from the execute method. Maybe this generalization could be better described as a Builder pattern, but hey, we don't want to get picky, do we? :)

Answer 4

You may want to use a rules engine. A rules engine gives you a DSL (Domain Specific Language) that is designed to model the criteria needed for a certain outcome in an understandable way, as explained in this question.

Depending on the implementation of the rules engine, the rules may even be changed without recompiling the code. And because the rules are written in their own, simple language, they can be changed by the users as well.

If you're lucky there is a ready-to-use rules engine for the programming language you are using.

The downside is that you have to get acquainted with a rules engine which may be hard if you're a programming beginner.

Answer 5

My solution to this problem is quite different: layers, settings, and LOP.

First wrap the API in a layer. Find sequences of API calls that are used together and combine them into your own API calls. Eventually there should be no direct calls to the underlying API, just calls to your wrappers. The wrapper calls should start to look like a mini language.

Second, implement a 'settings manager'. This is a way to associate names with values dynamically. Something like this. Another mini language.

Baseplate.name="Base plate"
Baseplate.length=1032.5
Baseplate.width=587.3

Finally, implement your own mini language in which to express designs (this is Language Oriented Programming). This language should be understandable to the engineers and designers who contribute the rules and settings. The first example of such a product that comes to mind is Gnuplot, but there are many others. You could use Python, although personally I wouldn't.

I understand that this is a complex approach, and may be overkill for your problem, or requiring skills you have yet to acquire. It's just how I would do it.

Answer 6

I'm not sure I got the question correctly, but it sounds like you should group things in some structures. Say if you are using C++, you can define things like:

struct SomeParametersClass
{
    int   p1;  // this is for that
    float p2;  // this is a different parameter
    ...
    SomeParametersClass() // constructor, assigns default values
    {
        p1 = 42; // the best value that some guy told me
        p2 = 3.14; // looks like a know value, but isn't
    {
};

struct SomeOtherParametersClass
{
    int   v1;  // this is for ...
    float v2;  // this is for ...
    ...
    SomeOtherParametersClass() // constructor, assigns default values
    {
        v1 = 24; // the best value 
        v2 = 1.23; // also the best value
    }
};

You can instanciate these at beginning of program:

int main()
{
    SomeParametersClass params1;
    SomeOtherParametersClass params2;
    ...

Then your API calls will look like (assuming you can't change the signature):

 SomeAPICall( params1.p1, params1.p2 );

If you can change the signature of API, then you can pass the whole structure:

 SomeAPICall( params1 );

You can also group all the parameters into a larger wrapper:

struct AllTheParameters
{
    SomeParametersClass      SPC;
    SomeOtherParametersClass SOPC;
};

Answer 7

I'm surprised that no one else has mentioned this...

You said:

My main goal is to be able to hand someone my source and have them understand what I was doing, without my input.

So let me say this, most of the other answers are on the right track. I definitely think databases could help you out. But another thing that will help you out is commenting, good variable names, and proper organization / separation of concerns.

All the other answers are heavily technical based, but they're ignoring the fundamentals that most programmers learn. Since you're a mech engie by trade, my guess is you're not used to this style of documentation.

Commenting and choosing good, succinct variable names helps immensely with readability. Which is easier to understand?

var x = y + z;

Or:

//Where bandwidth, which was previously defined is (1000 * Info Rate) / FEC Rate / Modulation * carrier spacing / 1000000
float endFrequency = centerFrequency + (1/2 bandwidth);

This is pretty language independent. No matter what platform, IDE, language, etc. you're working with, proper documentation is the cleanest, easiest way to make sure someone can understand your code.

Next it's on to managing those magic numbers and tons of concerns, but I think GrandmasterB's comment handled that pretty well.