Logging¶
The B-Human framework offers sophisticated logging functionality that can be used to log the values of selected representations while the robot is playing. There are two different ways of logging data: online logging and remote logging.
Online Logging¶
The online logging feature can be used to log data directly on the robot during regular games. It is implemented as part of the robot control program and is designed to log representations in real-time. Log files are written into the directory /home/nao/logs or – if available – into /media/usb/logs, i.e. a USB flash drive on which the directory called logs will be created automatically. To use a USB flash drive for logging, it must already have been plugged in when the B-Human software is started.
Online logging starts as soon as the robot enters the ready state and stops upon entering the finished state. The name of the log file consists of the names of the head and body of the robot as well as its player number, the scenario, and the location. If connected to the GameController, the name of the opponent team and the half are also added to the log file name. Otherwise, "Testing" is used instead. If a log file with the given name already exists, a number is added that is incremented for each duplicate.
To retain the real-time properties of the threads, the writing of the file is done in a separate thread without real-time scheduling. This thread uses the remaining processor time that is not used by one of the real-time parts of the system. Communication with this thread is realized using a very large ring buffer (usually around 2.4 GB). Each element of the ring buffer represents one frame of data. If a buffer is full, the current frame cannot be logged.
Due to the limited buffer size, the logging of very large representations such as the CameraImage is not possible. It would cause a buffer overflow within less than two minutes rendering the resulting log file unusable. However, without images a log file is nearly useless, therefore loggable JPEG images are generated and used instead.
In addition to the logged representations, online log files also contain timing data, which can be seen using the timing view, and annotations, which can be seen using the annotation view.
Configuring the Online Logger¶
The online logger can be configured by changing the following values in the file Config/Scenarios/<scenario>/logger.cfg:
enabled: The logger is enabled if this is true. Note that it is not possible to enable the logger inside the simulator.path: Log files will be stored in this path on the NAO.numOfBuffers: The number of buffers allocated.sizeOfBuffer: The size of each buffer in bytes. Note that numOfBuffers × sizeOfBuffer is always allocated if the logger is enabled.writePriority: Priority of the logging thread. Priorities greater than zero use the real-time scheduler, zero uses the normal scheduler, and negative values (-1 and -2) use idle priorities.minFreeDriveSpace: The minimum amount of disk space that should always be left on the target device in MB. If the free space falls below this value the logger will cease operation.representationsPerThread: List of all representations that should be logged per thread.
Remote Logging¶
Online logging provides the maximum possible amount of debugging information that can be collected without breaking the real-time capability. However in some situations one is interested in high precision data (i.e. full resolution images) and does not care about real-time. The remote logging feature provides this kind of log files. It utilizes the debugging connection to a remote computer and logs all requested representations on that computer. This way the heavy lifting is outsourced to a computer with much more processing power and a bigger hard drive. However sending large representations over the network severely impacts the NAO's performance and can result in a loss of the real-time capability.
To reduce the network load, it is usually a good idea to limit the number of representations to the ones that are really needed for the task at hand. The follow listing shows the commands that need to be entered into SimRobot to record a minimal vision log. Alternatively, call Includes/VisionLog can be executed.
dr off
dr representation:BodyContour
dr representation:CameraImage
dr representation:CameraInfo
dr representation:CameraMatrix
dr representation:ImageCoordinateSystem
dr representation:OdometryData
log start
log stop
log save <filename>
Log File Format¶
In general, log files consist of serialized message queues. Each log file consists of up to five chunks. Each chunk is prefixed by a single byte defining its type. The enum LogFileFormat in Src/Libs/Framework/LoggingTools.h defines these chunk identifiers in the namespace LoggingTools that have to appear in the given sequence in the log file if they appear at all:
logFileSettings: This chunk contains the parts of theSettings(robot name, player number, location and scenario) that should be restored when replaying the log file. They are part of the standard log file name as well, but keeping them in the file allows to rename the file.logFileMessageIDs: This chunk contains a string representation of theMessageIDsstored in this log file. It is used to convert theMessageIDsfrom the log file to the ones defined in the version of SimRobot that is replaying the log file. Thereby, log files still remain usable after the enumerationMessageIDwas changed.logFileTypeInfo: This chunk contains the specification of all datatypes used in the log file. It is used to convert the logged data to the specifications that are defined in the version of SimRobot that is replaying the log file. If the specification changed, messages will appear in SimRobot's console about the representations that are converted. Please note that a conversion is only possible for representations the specification of which is fully registered. This is not the case for representations that usereadandwritemethods to serialize their data, e.g.CameraImageandJPEGImage. Therefore, such representations cannot be converted and will likely crash SimRobot when trying to replay log files containing them after they were changed.logFileUncompressed: Uncompressed log data is stored in a singleMessageQueue. Its serialized form starts with an eight byte header containing two values. These are the size used by the log followed by the number of messages in the queue. Those values can also be set to -1 indicating that the size and number of messages is unknown. In this case the amount will be counted when reading the log file. The header is followed by several frames. A frame consists of multiple log messages enclosed by anidFrameBeginandidFrameFinishedmessage. Every log message consists of its id, its size and its payload. This kind of chunk is created by online logging and remote logging. This chunk is opened as memory mapped file, i.e. the data is only read from disk when it is accessed, which speeds up opening log files considerably.logFileCompressed: This chunk contains a sequence of compressedMessageQueues. Each queue contains a single frame worth of log data and is compressed using Google's Snappy1 compression. This chunk can still be read, but it is not created anymore, because the log files mainly consist of JPEG images that cannot be compressed any further.logFileIndices: This chunk contains indices to look up frames, the number of message types per thread, and annotations per thread. It is created when a log is first opened in the simulator and if the log is also writable. The indices speed up subsequent loading of the log file significantly.
The first three chunks and the last one are optional. Of the remaining two chunks (logFileUncompressed and logFileCompressed), exactly one must be present in a log file.
This figure, depicting the format of a chunk of the type logFileUncompressed, is based on the one by Trocha, 20102.
Replaying Log Files¶
Log files are replayed using the simulator. A special module, the LogDataProvider, automatically provides all representations that are present in the log file. All other representations are provided by their usual providers, which are simulated. This way log file data can be used to test and evolve existing modules without access to an actual robot. If the name of a log file follows the naming convention used by the online logger or the logFileSettings chunk is present, the head name, body name, player number, scenario, and location will be set accordingly before the log file is replayed. Thereby, modules will read the matching versions of their configuration files. The SimRobot scene ReplayRobot.ros2 can be used to load the log file. In addition to loading the log file this scene also provides several keyboard shortcuts for navigation inside the log file. However, the most convenient way to control log file playback is the log player view. If you want to replay several log files at the same time, simply create a file Config/Scenes/Includes/replay.con and add several sl statements to it.