Logging
The Logback logging framework is used to manage the logs. The default logging configuration for the Azure IoT Hub Connector writes logs with a log level of INFO or higher to standard out. For more control over the logging configuration, you can create a new logging configuration XML file (logback.xml) and instruct the Azure IoT Hub Connector to use it by
setting the environment variable.
As of v.3.0.0 of the Connector, you can control the amount of disk space log files use and also the length of time that the files are retained. This configuration for logging is called a rolling policy. Refer to
Configuring a Rolling Policy for details.
For information on how to configure the appenders and encoders for the logging configuration, refer to the manual page on the Logback website:
http://logback.qos.ch/manual/ and the examples below.
Types of Logs
There are two types of logs:
• Operational — Contains messages that describe the state of the application or changes in the state of the application (the Azure IoT Hub Connector). It is intended for system administrators or any person interested in the general health of the system. For example:
◦ Azure IoT Protocol Adapter has started. This message indicates successful startup.
◦ Unable to connect to Azure IoT Hub endpoint. This message indicates failure to start up.
◦ Unexpected error occurred while consuming a message: partitionId=ID* + stack trace. This message indicates that the Connector received a message from Azure about unexpected error handling.
◦ When the egress handler starts up, operational logging messages indicate the success or failure of the startup.
◦ When an error occurs when writing a property to the edge (egress), an error message indicates that an error occurred.
• Functional — Contains messages that describe the lower-level details about what the individual application components are doing to accomplish their purpose (or function), including the context. In general, functional logs are more useful for debugging an application when problems arise. For example:
◦ Sending an HTTP request to service endpoint (a write property service call, for example): endpoint=http://localhost:8720/some-service.
◦ Stack traces
To help differentiate the operational and functional logs, all operational logs are tagged with the "operational" marker. This tagging allows you to do something special for operational logs. The example in the next section shows how.
 |
Log files include the date and the timestamp of each message.
|
Separating Operational and Functional Logging in the logback.xml File
By default, all logging information is combined into one file. To separate the operational logs from the functional logs, add the following appender information to the logback.xml file:
<appender name="OPSLOG" class="ch.qos.logback.core.FileAppender">
<file>azure-iot-ops.log</file>
<append>true</append>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
<marker>operational</marker>
</evaluator>
<OnMismatch>DENY</OnMismatch>
</filter>
<encoder>
<pattern>$d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
In addition, to send operational logs exclusively to a separate file, you need to exclude them from the functional log by adding the following clause inside the LOGS appender:
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
<marker>operational</marker>
</evaluator>
<OnMatch>DENY</OnMatch>
</filter>
In addition, add an appender-ref for the operational log (OPSLOG):
<root level="info">
<appender-ref ref="LOG" />
<appender-ref ref="OPSLOG" />
</root>
The first appender-ref specifies the functional log. The second one is for the operational log.
Resolving a Reference in the Configuration File to the Metrics Logger
The reference to the metrics logger in the configuration file for the Connector is resolved in the logback.xml file by using the following appender:
<appender name="METRICS" class="ch.qos.logback.core.FileAppender">
<file>metrics.log</file>
<append>false</append>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<logger name="metrics" additivity="false">
<appender-ref ref="METRICS" />
</logger>
Configuring a Rolling Policy
The default logging configuration for the Connector creates two logs, metrics.log and cxserver.log. The default configuration sends nothing to the metrics log. With v.2.0 of the Connector, the cxserver.log could grow until it consumed all disk space. When this happened, the VM upon which the Connector was installed was brought down. To prevent this situation from occurring, the default configuration for v.3.0.0 and later of the Connector uses a RollingFileAppender with a SizeAndTimeBasedRollingPolicy instead of the previous FileAppender. By default, the logs roll daily. The logback.xml and logback-long-sample.xml both roll at 10MB and keep for 7 days or until the size reaches 100MB, whichever happens first. In addition, the files are kept in a logs subdirectory instead of in the parent \bin directory. Log messages have the date and timestamp.
The following example of a logback.xml file illustrates the default policy of the RollingFileAppender. By default, the logs roll daily, and the maximum size of the cxserver.log and metrics.log files is 10MB, with a maximum of seven days' worth of logs but a maximum total of 100MB across all files.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOG_HOME" value="./logs" />
<appender name="LOGS"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_HOME}/azure-iot.log</file>
<append>false</append>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_HOME}/azure-iot.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<!--Each file should be 10 MB, with a max of 7 days of logs, but a max total of 100MB-->
<maxFileSize>10MB</maxFileSize>
<maxHistory>7</maxHistory>
<totalSizeCap>100MB</totalSizeCap></rollingPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<appender name="METRICS"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_HOME}/metrics.log</file>
<append>false</append>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_HOME}/metrics.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<!--Each file should be 10 MB, with a max of 7 days of logs, but a max total of 100MB-->
<maxFileSize>10MB</maxFileSize>
<maxHistory>7</maxHistory>
<totalSizeCap>100MB</totalSizeCap></rollingPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<logger name="metrics" additivity="false">
<appender-ref ref="METRICS" />
</logger>
<root level="warn">
<appender-ref ref="LOGS" />
</root>
</configuration>