log levels and stdout vs stderr

Question

I'm writing an app which uses a logger with different logging levels (info, debug, warning, error, etc.); but - I'm used to the custom of writing program output to stdout and error information to stderr.

My question: Suppose my logger uses all of these levels, and that I use the logger to report all errors. How do I combine the stdout/stderr idiom with logger levels? That is, should I have the logger...

• only write to the standard error stream?
• only write to stdout?
• choose between stdout and stderr depending on the levels (e.g. info and weaker to stdout, warning and more severe to stderr)?
• do something else?

Note: It's a C++ app in case you believe that matters.

Answer 1

For a background process that could be considered a service or a daemon, syslog is appropriate.

For a user program, especially one designed for user interaction, syslog is probably not appropriate, so stderr/stdout is more appropriate.

Traditionally, unix programs are designed with pipes in mind, where you might be taking input from stdin and processing it and sending the results to stdout. The point of stderr is so that messages designated for the user have somewhere to go other than contaminating the output stream.

From that viewpoint, it would be wrong to send any user designated messages to stdout.

However, if your program is more interactive, where it would not make sense to save the output or pipe it to another program, it is probably moot if the messages go to stdout or stderr, and it may make more sense to send everything to stdout. Also, stdout and stderr may be buffered differently, so if you are sending data to both, the errors may end up misplaced within the mixed data stream.

The key question to ask is if it makes sense to separate the program output from the error stream in any situation, or if embedding those log messages in the output stream to annotate it is more appropriate.

Answer 2

choose between stdout and stderr depending on the levels (e.g. info and weaker to stdout, warning and more severe to stderr)?

This. Unless there's a good reason not to I like stderr to get errors. This is customary expected behavior.

When making this call simply pretend that the level logging system doesn't exist. Send the log message where you would have sent it before. Nothing in a level logging system justifies changing the meaning or use of stderr. If you use it, use it right.

A case can be made that warnings don't belong in stderr. A warning is a potential problem but doesn't indicate that anything the system was asked to do hasn't been done. Understand that some sys admins monitor stderr for any text at all and configure automated responses to it.

---

The stdout and stderr custom is to directly hard code those destinations / concepts into your app and configure where they go on the command line when the app is run (be that, console, file, etc).

The logger custom is to directly hard code logging levels into your app and configure where they go, and if they go, in a logging configuration file that is aware not only of the log level but what logger was used to send them. This configuration also controls where they go (stdout, stderr, file, etc).

They can be configured to work together just fine. The difference between them is that configuring stdout and stderr output at the command line is a 30 second google search. Configuring most logging system with logging levels involves planning. Even if all you want to do is make it work like the old school stdout and stderr.

The reason planning is needed is the flexibility. Log level tools let you make noisy things shut up when you don't need them and bring them back when you do. It also lets you chose which things to do this to. All without touching code.

With stdout and stderr all you get is two buckets and a choice of where to dump them.

So you combine them by configuring the logging tool to wisely target the most appropriate bucket (stdout or stderr) for the log message. Not that you must use either std bucket. But if you use them then people who know how to use the std buckets, and don't feel the need to mess with logging levels, will appreciate not having to touch that confusing config file.

Answer 3

In unix systems with syslogd, the logging levels (info, debug, warning, error) are handled by the logging daemon. Your code would just log everything (using syslog calls instead of stderr), and tag each message with one of the log levels, and syslog itself would decide what to do with it.

This would allow for complex filtering of logs separate from your program, and log levels could be changed on the fly without interacting with your program. Generally, however, few people bother to configure syslogd and just leave the defaults (which would be to throw away everything but error). But if someone was trying to debug a problem, this might still be used.

These days, with systemd, it's trivial to just dump to stderr and let systemd take care of it, but this lacks some of the flexibility of syslogd which is still in use.