Articles  
 

IHE and the syslog message size

Written 2005-08-04 by Rainer Gerhards.

This is work in progress, expect updates to this page!

This is not yet a real article, but more a lab notebook. I had an interesting discussion on IHE and syslog. The issue with that is that IHE defines log records of up to 32K, while syslog only allows records of up to 1k - at least in current standards. Thankfully, many syslog implementations do not take this limit as fixed and ignore the standard in that regard. Also, the upcoming new standard allows for larger messages, so this issue somewhat disappears.

Anyhow... This time, I was curious enough to do a lab session on what needs to be done to make the receivers "IHE-aware". I am noting down the results on this page and will update it whenever I have some additional results. I will also update it when I have some more ideas on how to make IHE really work with current syslog.

I used two simple perl scripts as the syslog clients/senders. Each of them tries to send a ca. 32K test message (with nonsense data) to a syslog server. One does it via udp, the other via tcp.

Results

Perl run on Target Proto Result
Windows MonitorWare Agent/
WinSyslog
UDP 4k sharp, 2 additional 4k messages received, rest truncated
Windows MonitorWare Agent/
WinSyslog
TCP 32k (and more) - everything received
Windows rsyslogd[33k] (Red Hat 8) UDP 4k sharp, additional text received in 7 follow-up messages, 1 segment missing
Windows rsyslogd[33k] (Red Hat 8) TCP 32k - everything received
Linux (Red Hat 8) MonitorWare Agent/
WinSyslog
UDP 4k sharp, additional text received in follow-up messages, all data
Linux (Red Hat 8) MonitorWare Agent/
WinSyslog
TCP 32k - everything received
Linux (Red Hat 8) rsyslogd[33k] (Red Hat 8) UDP 4k sharp, additional text received in follow-up messages, all data
Linux (Red Hat 8) rsyslogd [33k] (Red Hat 8) TCP 32k  - everything received
Windows rsyslogd[1k]  (Red Hat 8) UDP 1k sharp, additional text received in 7 follow-up messages, rest missing
Windows rsyslogd[1k]  (Red Hat 8) TCP 1k sharp, additional text received in 31 follow-up messages

Notes:

  • WinSyslog and MonitorWare Agent have no inherent message size limitation and accept whatever the IP stack forwards to them. So test were conducted without any special setup.
  • Rsyslogd has a hardcoded size limitation, which is easily modifyable. In the table, rsyslogd[33k] means rsyslogd compiled with support for 33k message size, rsyslogd[1k] means compiled with 1k message size (the default).
  • The Windows workstation used in the table above was Windows XP SP2, Build 2600
  • most test were conducted using local area networking, some with WLAN (as an additional checkpoint)
  • all test were performed at least three time, more often if there were some inconsistencies between the test results. Additional tests were also done in the case where only 7 out of 8 message segements were received. All results are clearly reproducable.

(Preliminary) Conclusion

UDP-based syslog is unable to support more than 4k of message size, even if the receiver application imposes no limit. RFC 3164 based solutions might be extended to glue packets together, but this mode is not well-defined. Different operating systems seem to come to different results in the amount of data that can be transmitted. An approach might be to send large messages in multiple chunks and have the receiver glue them together. However, that would require specification.

TCP-based syslog provides full support for 32k message sizes (and beyond), but is not currently standardized.

RFC 3195 syslog would support larger message sizes in theory. As was found out, one can argue that no strict upper message limit size is specified (though I think this is more an omission in the RFC, but we might use that to our advantage). However, there is currently no known implementation supporting messages of larger than 1k. Liblogging seems to be modifiable to support it with moderate effort. However, this needs to be proven, especially if a "clean" solution is desired (that would lead to full implementation of BEEP framing and segmentation).

Revision History

Copyright

Copyright (c) 2005 Rainer Gerhards and Adiscon.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be viewed at http://www.gnu.org/copyleft/fdl.html.

MonitorWare
 Home
 The Products
MonitorWare Products
Product Comparison
Which one to Purchase?
Order and Pricing
Upgrade Insurance Info
News Releases
Version History
MonitorWare Tools
 Event Repository
 Download
 Reference library
General Information
Step-by-step guides
 - All
 - Installation and Configuration
 - Services related
 - Actions related
 - Central Monitoring
Common Uses
Syslog configuration
Syslog Log Samples
Security Reference
 Help
Support
Manual
FAQ
 - All
 - General questions
 - Configurations related
 - Monitorware Agent
 - Monitorware Console
Articles
Seminars Online
 - All
 - General
 - MonitorWare Console
 - MonitorWare Agent
 - WinSyslog related
 - EventReporter
 Order & pricing
Order now
Product Comparison
Pricing Information
Upgrade Insurance Info
Local Reseller
 Contact Us
 Search
 
 



Printer Version Send this page to a friend

Copyright © 1988-2005 Adiscon GmbH All rights reserved.
Contact us via Secure Web Response | Privacy Policy
Topic Links: syslog | Free Weblinks Directory