I’m currently working on a microservices-based Spring Boot application, where I’m using Logback as my logging framework. The application is now running live on a remote server. Since I don’t have access to the server, I don’t get to see the logs when I want to see them or when there is some error. I have to ask someone from the sysadmin team to send me the log file every time and wait for their response. This kind of human dependency eats up a decent amount of time in any software development lifecycle. I try to avoid such dependencies and seek to circumvent them. So, I have written a simple snippet of code that can read the log file from the server filesystem and send it back to me in an API response.
Let me first take you through how I have configured the logging mechanism in my Spring Boot app and then we will see how to fetch app logs from a remote server.
Configuring Logback #
Here is my logback.xml configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern> %d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M %L - %msg%n </pattern>
</encoder>
</appender>
<property name="APP_LOG" value="logs/customer-dashboard.log"/>
<appender name="FILE-ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${APP_LOG}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>logs/archived/customer-dashboard.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<!-- max size for an archived file -->
<maxFileSize>10MB</maxFileSize>
<!-- max ten archived files are kept,
old files are deleted -->
<maxHistory>10</maxHistory>
<!-- total size of all archived files,
delete old archived files if size exceeds -->
<totalSizeCap>1GB</totalSizeCap>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern> %d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M %L - %msg%n </Pattern>
</encoder>
</appender>
<logger name="com.test.customerdashboard" level="debug" additivity="false">
<appender-ref ref="FILE-ROLLING"/>
<appender-ref ref="STDOUT"/>
</logger>
<root level="info">
<appender-ref ref="FILE-ROLLING"/>
<appender-ref ref="STDOUT" />
</root>
</configuration>
I have set up two appenders here, one for logging in to the console and another one to log to a file. Loggers are responsible for capturing events and passing them to the appender, and appenders are responsible for documenting the log events to a destination. The encoder pattern prints the entire timestamp, name of the thread in which the log message occurred, log level, the class name with the package name, method name, line number, and the message, respectively.
Expand this to understand the encoder pattern in more detail
Encoder pattern: %d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M %L - %msg%n
%d – Returns the time when the log message occurred.
%thread – Returns the name of the thread in which the log message occurred.
%-5level – Returns the logging level of the log message (ERROR, WARN, INFO, DEBUG, and TRACE).
%logger{64} – Returns the package with the package and class name where the log message occurred. The number 64 inside the brackets represents the maximum length of the package and class name combined. You can change this number as per your need.
%M – Returns the name of the method where the log message has occurred.
%L - Line number
%msg – Returns the actual log message.
%n – Line break.
The rolling policy defines how and when to split the log file and make an archived log file out of old log entries. The current log file is stored inside the /logs folder and the archived logs inside the /logs/archived folder.
Get the log file #
I have used the file reading and handling classes in Java to read the log file from the server filesystem.
import org.springframework.core.io.ByteArrayResource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
import java.util.stream.Collectors;
@RestController
public class FetchLogFileController {
@GetMapping("/log")
public ResponseEntity<?> fetchCurrentLogFile(@RequestParam(required = false) String fileName) {
String logFolderPath = "logs/";
String currentLogFilePath = logFolderPath + fileName;
if(isEmptyOrNull(fileName)){
return fetchFolderContents(logFolderPath);
}else{
return fetchFile(currentLogFilePath);
}
}
@GetMapping("/log/archives")
public ResponseEntity<?> getArchivedLogs(@RequestParam(required = false) String fileName) {
String archivedFolderPath = "logs/archived/";
String archivedLogFilePath = archivedFolderPath + fileName;
if(isEmptyOrNull(fileName)) {
return fetchFolderContents(archivedFolderPath);
}else{
return fetchFile(archivedLogFilePath);
}
}
private ResponseEntity<?> fetchFile(String filePath) {
ByteArrayResource resource;
try {
Path path = Paths.get(filePath);
resource = new ByteArrayResource(Files.readAllBytes(path));
} catch (Exception e) {
return ResponseEntity.ok().body(e.getMessage());
}
return ResponseEntity.ok().contentType(MediaType.APPLICATION_OCTET_STREAM).body(resource);
}
private ResponseEntity<?> fetchFolderContents(String folderPath) {
List<String> folderContents;
try {
Path path = Paths.get(folderPath);
File folder = path.toFile();
folderContents = Arrays.stream(Objects.requireNonNull(folder.list()))
.filter(contents -> contents.endsWith(".log"))
.collect(Collectors.toList());
} catch (Exception e) {
return ResponseEntity.ok().body(e.getMessage());
}
return ResponseEntity.ok().body(folderContents);
}
private boolean isEmptyOrNull(String str) {
return str == null || "".equals(str.trim());
}
}
I have set the content type of the response to application/octet-stream to avoid any processing or encoding by browsers or any other clients since doing so could have taken a long time to fetch the file.
The /log API endpoint displays a list of all .log files inside the /log folder, and on passing the file name of the log that you wish to view to the fileName parameter, you get to see the content of that log file. Similarly, the /log/archives API endpoint displays a list of all .log files inside the /log/archived folder. Pass the file name in the parameter and it fetches that log file for you.
