How to monitor a long-running program programmatically

Question

What I've currently got can be summed up with this pseudo code:

public static void Main(string[] args)
{
    var listOfObjects = Database.GetObjectsToUploadToOnlineService();
    Parallel.ForEach(Upload)
}

private static void Upload(MyUploadObject obj)
{
    //Build object (takes a few milliseconds)
    //Format to JSON (takes a few more milliseconds)
    //Upload (can take up to a max of 10 minutes)
    //Wait for a response (can take up to a max of 10 minutes)
    //Save response to our database (takes a few milliseconds)
}

This program is just set up on our server as a scheduled task. We own the program and can do what we want with it. My question stems from a blog about making automated monitoring checks (I don't have a link handy).

So that got me to think: How in the world can I modify my program so that I can fire up another "monitoring" program? Or should this change from being a console program to to say a WPF program that is hidden?

Overall, I would prefer to be able to just run a program on my computer that checks on the progress of the program through the network, so I don't have to RDP into the server to check on its status (but it wouldn't be the end of the world).

Overall, I suppose I would like to see a window that says something like: Process x of y so far, the following items are being processed. List the items in, like, a table and have them say like "Uploading" or "Waiting for response". Maybe if I get crazy I could also how a queue of failed items (but that would just be extra).

My mind keeps leaning towards an event, but I can't figure out how I would run a program that can subscribe and unsubscribe to a running program. Is this even possible?

Do you want to monitor the progress inside the (10 minute lasting) upload/response operations, or do you want to know how many uploads have taken place inside the "Parallel.ForEach" loop? — Doc Brown, Apr 24 '15 at 13:35
Sounds to me like what you want is maybe a simple UDP receiver program. You can drop status packets on the network, and if there's nothing there to receive them, no harm done. — Robert Harvey, Apr 24 '15 at 14:04

Doc Brown · Answer 1 · 2015-04-24T14:41:23.723

From your comments I see you have a client/server database available, and you uploader already has a connection and write access? Then it would probably be easiest to add a "monitoring" or "status" table to the database and let your uploader report its progress there (log each "interesting" step there, probably the 5 steps you listed above).

To view the status, create a second program with GUI, which connects to the database and displays the status (for example, by polling in intervals of 1 minute). You can run that program on your local desktop, assumed it is possible to connect to the database from there.

Of course, when writing to a status table interfers with your database transactions, you might use a separate connection. You can also utilize a message queue mechanism like MSQM for it, but that might be an oversized solution for your simple case.

+1 This is the best approach to keeping it simple and not over-complicating something that shouldn't be too complex. — Thomas Stringer, Apr 24 '15 at 15:12

score 8 · Accepted Answer · answered Apr 24 '15 at 14:13

What you need is a way to communicate between your programs that does not tie them to each other. There are a number of ways to do this, you could have the task write to a file and have to monitor read from the file, make the task available via WCF so the monitor can 'poll' for changes, use the network...

To avoid reinventing the wheel, have a look at the log4net logging targets If I were to implement something like this I would probably use log4net and log to UDP or Telnet and have the monitor on the other end connect to it. log4net takes care of everything for you, including not throwing exceptions when there is no monitor active.

I think i am giong to go with this answer because we already use log4net for well.. logging :) So this would fit nicely with what he have. I had no clue log4net could do this! Thank you — Robert Snyder, Apr 25 '15 at 01:45

score 3 · Answer 3 · answered Apr 24 '15 at 14:27

Just to build on @JDT's answer, a common way to do this is writing messages to a message queue. Whenever something important happens in your application it would write a message and send it to the message queue. Typically the format of the message is XML or similar. It's the queue writer.

The monitoring application reads (queue reader) the queue and picks the messages and processes them by saving them and analyzing the message versus prior messages or conditions. If a condition is met, the monitoring application will raise an alert signifying a potential problem.

This decouples your application from monitoring itself as it is just firing away messages. It's up to the monitoring application to determine when based on the current messages that something good or bad has happened. Queue's are used because they offer a good means to store the messages without loss.

You want to centralized messaging if there are multiple instances of your application running. If there is only 1 instance, then using other stores like the windows event log or a file can be sufficient.

How to monitor a long-running program programmatically

3 Answers3