This is a part of the rsyslog.conf documentation.
Back to rsyslog.conf manualRsyslog has a modular design. This enables functionality to be dynamically loaded from modules, which may also be written by any third party. Rsyslog itself offers all non-core functionality as modules. Consequently, there is a growing number of modules. Here is the entry point to their documentation and what they do (list is currently not complete)
Please note that each module provides configuration directives, which are NOT necessarily being listed below. Also remember, that a modules configuration directive (and functionality) is only available if it has been loaded (using $ModLoad).
It is relatively easy to write a rsyslog module. If none of the provided modules solve your need, you may consider writing one or have one written for you by Adiscon's professional services for rsyslog (this often is a very cost-effective and efficient way of getting what you need).
There exist different classes of loadable modules:
Input modules are used to gather messages from various sources. They interface to message generators.
Output modules process messages. With them, message formats can be transformed and messages be transmitted to various different targets.
Parser modules are used to parse message content, once the message has been received. They can be used to process custom message formats or invalidly formatted messages. For details, please see the rsyslog message parser documentation.
The current modules are currently provided as part of rsyslog:
Message modification modules are used to change the content of messages being processed. They can be implemented using either the output module or the parser module interface. From the rsyslog core's point of view, they actually are output or parser modules, it is their implementation that makes them special.
Currently, there do not exist any such modules, but could be written with the methods the engine provides. They could be used, for example, to:
Message modification modules are usually written for one specific task and thus usually are not generic enough to be reused. However, existing module's code is probably an excellent starting base for writing a new module. Currently, the following modules existin inside the source tree
String generator modules are used, as the name implies, to generate strings based on the message content. They are currently tightly coupled with the template system. Their primary use is to speed up template processing by providing a native C interface to template generation. These modules exist since 5.5.6. To get an idea of the potential speedup, the default file format, when generated by a string generator, provides a roughly 5% speedup. For more complex strings, especially those that include multiple regular expressions, the speedup may be considerably higher.
String generator modules are written to a quite simple interface. However, a word of caution is due: they access the rsyslog message object via a low-level interface. That interface is not guaranteed yet to stay stable. So it may be necessary to modify string generator modules if the interface changes. Obviously, we will not do that without good reason, but it may happen.
Rsyslog comes with a set of core, build-in string generators, which are used to provide those default templates that we consider to be time-critical:
Note that when you replace these defaults be some custom strings, you will loose some performance (around 5%). For typical systems, this is not really relevant. But for a high-performance systems, it may be very relevant. To solve that issue, create a new string generator module for your custom format, starting out from one of the default generators provided. If you can not do this yourself, you may want to contact Adiscon as we offer custom development of string generators at a very low price.
Note that string generator modules can be dynamically loaded. However, the default ones provided are so important that they are build right into the executable. But this does not need to be done that way (and it is straightforward to do it dynamic).
Library modules provide dynamically loadable functionality for parts of rsyslog, most often for other loadable modules. They can not be user-configured and are loaded automatically by some components. They are just mentioned so that error messages that point to library moduls can be understood. No module list is provided.
Depending on their module type, modules may access and/or modify messages at various stages during rsyslog's processing. Note that only the "core type" (e.g. input, output) but not any type derived from it (message modification module) specifies when a module is called.
The simplified workflow is as follows:
As can be seen, messages are received by input modules, then passed to one or many parser modules, which generate the in-memory representation of the message and may also modify the message itself. The, the internal representation is passed to output modules, which may output a message and (with the interfaces newly introduced in v5) may also modify messageo object content.
String generator modules are not included inside this picture, because they are not a required part of the workflow. If used, they operate "in front of" the output modules, because they are called during template generation.
Note that the actual flow is much more complex and depends a lot on queue and filter settings. This graphic above is a high-level message flow diagram.
[manual index] [rsyslog.conf] [rsyslog site]
This documentation is part of the
rsyslog project.
Copyright © 2008-2010 by Rainer Gerhards and
Adiscon. Released under the GNU GPL
version 3 or higher.