In my previous blog post, I showed how to manually troubleshoot Windows Containers by opening an interactive session to a container, running a PowerShell session, and using the Get-WinEvent cmdlet to view the application logs on the container instance. While knowing how that process works is extremely important to understand how to troubleshoot an application on Windows containers, it is not very convenient and doesn’t scale very well. Also, as I mentioned in that blog post, you might not even have access to the container host in some cases. We’ll get to those in a future blog post, but for now, I wanted to show a different tool that can help you in troubleshooting an application on Windows Containers.
Why do we even need the Log Monitor tool?
Windows and Linux have different ways to store and present logs and events to its users. Traditionally, Windows has been using the EVT format that can be viewed on a structured way in Event Viewer. On the other hand, Linux has provided a streamlined approach with Standard Output (stdout) that other tools, such as Docker, rely upon.
Docker has always had a mechanism to extract logs from a Linux containers via the “docker logs” command. That command has a default stdout configuration that brings the application logs out of the container without the need to open the container interactively – like we shown in the previous blog post. However, that did not work on Windows until the launch of the Log Monitor tool.
In a nutshell, the Log Monitor tool presents the Windows logs in the stdout format so other tools, such as Docker, can gather the information necessary to display it. However, the benefits are not limited to that.
Here are a few of the benefits of the Log Monitor tool:
- You can filter which types of events/logs you want the Log Monitor tool to expose on stdout. For example, you can filter the application log for “error” and “warning” messages only – in case you’re not interested on “information” events.
- You can choose from Event Logs, Custom Log Files, or Event Tracing for Windows (ETW). This is particularly helpful if your application is writing on a different log source. An example of this is the IIS logs located on the “C:\inetpub” folder.
- Other tools that look for stdout and interact with the container runtime can leverage the fact that Windows Containers are behaving much like a Linux Container. For example, if you move from Docker to ContainerD as the container runtime, the logs should still be visible from the container host via (for example) “crictl logs”.
How to use the Log Monitor tool
In order to collect the logs from the Windows container instance and then pass that to stdout, the Log Monitor tool needs to be instructed on what is its source and what filters to apply. By default, the tool expects a “LogMonitorConfig.json” file alongside its executable. Here’s the step by step to get started with some samples:
Create a JSON file with the Log Monitor configuration
In our example, we are going to monitor an IIS instance. Here is a sample file that can be used for that purpose:
{
"LogConfig": {
"sources": [
{
"type": "EventLog",
"startAtOldestRecord": true,
"eventFormatMultiLine": false,
"channels": [
{
"name": "application",
"level": "Warning"
},
{
"name": "application",
"level": "Error"
}
]
},
{
"type": "File",
"directory": "c:\\inetpub\\logs",
"filter": "*.log",
"includeSubdirectories": true
},
{
"type": "ETW",
"eventFormatMultiLine": false,
"providers": [
{
"providerName": "IIS: WWW Server",
"providerGuid": "3A2A4E84-4C21-4981-AE10-3FDA0D9B0F83",
"level": "Information"
},
{
"providerName": "Microsoft-Windows-IIS-Logging",
"providerGuid": "7E8AD27F-B271-4EA2-A783-A47BDE29143B",
"level": "Information"
}
]
}
]
}
}
The example above uses the three options the Log Monitor tool can have as source. The first option is the Event Log just like you see on Event Viewer. However, we have filtered for Warnings and Errors only.
The next one is a file type, and in this case we’ll use the IIS logs file.
Finally, we’ll use the ETW option to get real-time events from the IIS providers. (If you don’t know which ETW provider you’re looking for, you can use the Get-ETWTraceProvider cmdlet on PowerShell. It displayes the information you’ll need from the available ETW providers on that machine.)
You can save this file as “LogMonitorConfig.json” on the same folder you’ll build your container image.
Build a new container image with Log Monitor and its config
Next, you’ll need to build the image with the Log Monitor tool. Here’s a sample docker file to build an image based on the IIS base container image:
FROM mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2022
WORKDIR /LogMonitor
COPY LogMonitorConfig.json .
RUN powershell.exe -command wget -uri https://github.com/microsoft/windows-container-tools/releases/download/v1.1/LogMonitor.exe -outfile LogMonitor.exe
# Change the startup type of the IIS service from Automatic to Manual
RUN sc config w3svc start=demand
# Enable ETW logging for Default Web Site on IIS
RUN c:\windows\system32\inetsrv\appcmd.exe set config -section:system.applicationHost/sites /"[name='Default Web Site'].logFile.logTargetW3C:"File,ETW"" /commit:apphost
EXPOSE 80
# Start "C:\LogMonitor\LogMonitor.exe C:\ServiceMonitor.exe w3svc"
ENTRYPOINT ["C:\\LogMonitor\\LogMonitor.exe", "C:\\ServiceMonitor.exe", "w3svc"]
The docker file above uses the LTSC 2022 base container image with IIS installed, copy the json file we just built, downloads the Log Monitor tool from GitHub (you can download it separately if you prefer), then configures IIS so the Log Monitor tool would be able to get the ETW providers its preconfigured to, and finally it configures the Log Monitor tool as the entry point for the container.
With the docker file above also on the same location of your json file and the app you want to try out (in case you have one), you can run the below command to build an image:
docker build -t testapp:v1 .
Next, we can run the container to try out the web page:
docker run -d -p 8080:80 --name testcontainer testapp:v1
The command above will start a container that will have its port 80 mapped to the port 8080 on the container host. Now, open a web browser and try to access http://localhost:8080. If you did not deploy a different application, this will open the IIS splash screen from inside the container.
Use Docker Logs to retrieve the logs from the container
Now we can use the docker logs command to see what’s happening inside the container:
docker logs testcontainer
The output of the command above might be short or very long depending on how many events you filtered or chose to show or not. Here’s what I got from my test environment:
On the above, I’m highlighting that the logs shown in the output provide the details on the visitor (myself) to the website we’re monitoring. The same way, the logs would have shown other errors related to my application.
Conclusion
The Log Monitor tool is a great tool to get Windows Event, log files, and ETW into stdout, which is the standard format for containers. There are many ways to filter the source of your logs and you can make it work with your own application.
On the next blogs in the series, we’ll start exploring ways to make this process less manual and more efficient. Let me know what you think of this series so far in the comments section.
Posted at https://sl.advdat.com/3sPUOmRhttps://sl.advdat.com/3sPUOmR
