Getting the Most From Your SAS Logging Part 1
Configuring a Uniform Logging Architecture
Author(s): Maciej Baranowski
Introduction
If not correctly managed, logging can impact the stability of your SAS environment. The first part of this three-part blog series discusses the storage problem, and how to configure a uniform architecture to solve it.
If you’re a system or application administrator focused on problem diagnostics, you probably rely heavily on logging.
Logs are records of events that happen in a systems environment, either by a person or by a running process. They help administrators track what happened and troubleshoot problems. Logging pinpoints the what, when and how a failure occurred, and is helpful at guiding you in the right direction to fix these problems.
Another problem arises, however, when the growth of your generated logs aren’t managed, affecting the stability of your environment. In order to get the most out of logging, this blog series will explore how you can use it to assist in problem resolution, with minimal impact on your storage. In short;
- This first blog will talk you through configuring a uniform logging architecture using log4j, as well as a scripted approach to automatically archive and remove logs.
- Part 2 delves into how the Elasticsearch, Logstash, Kibana (ELK) stack can be used for swift log traversal and log visualisation.
- Part 3 focuses on the use of Loki and Promtail to build a centralised log solution in order to have all relevant logs at your fingertips.
The Problem with Logging
Configuration and management of logging is a core task undertaken when administering a SAS environment of any size. It forms one of the five pillars of environment management.
As we mentioned, necessary measures should be taken to contain the growth and size of generated logs or administrators may find themselves in a position where the storage affects the stability of their SAS platform.
What Logs are Available Out of the Box
At a glance, the SAS platform is divided into three tiers; Metadata Tier, Compute Tier and Web Tier. Depending on the current setup of your SAS platform, you can expect the following logs to be there:
- Metadata Tier
- Metadata Server Logs
- Compute Tier
- Workspace Server Logs
- Batch Server Logs
- Stored Process Server Logs
- Object Spawner Logs
- Web Tier
- Web Server Logs
- Web Application Server Logs
Logging Detail Levels
To configure the services above, you can utilise the below list to determine the logging level required.
| Logging Level | Description |
| TRACE | Provides detailed information. |
| DEBUG | Provides detailed information about the flow through the system. |
| INFO | Provides information about run-time events of interest. |
| WARN | Shows warning messages (for example, use of deprecated APIs, poor use of an API, or other run-time situations that are undesirable). |
| ERROR | Shows all error messages. |
| FATAL | Shows message for fatal errors. |
| OFF | Specifies that logging is turned off. |
This list is ordered by the amount of detail contained in these logs, TRACE being the most and FATAL being the least.
Using the Log4j Configuration
Each service features a log4j configuration file which handles the way the logs are formed. These are initially typically configured to append to an existing log indefinitely without taking consideration of size.
User created logs, for example Workspace Server logs which are unique for each session, are an exception to this rule.
Avoid Platform Failure
The SAS software suite, depending on your requirements, can be jam-packed with useful applications which help your user base achieve their data analysis goals. This also means that each component of SAS Software produces logs, to a varying degree.
However, they have one thing in common; there is no automatic clean up utility which means they will keep growing until you clear them down.
Each component of your SAS software installation requires the ability to read and write to the disk to function correctly. Failure to carry out regular maintenance and truncation of logs carries the risk of occluding the disks responsible for housing your SAS configuration.
Without regular maintenance, your platform may come to a standstill. This poses the threat of potential data loss.
Environment Management can Help
Demarq’s environment management solution assists with the proper care and maintenance of your SAS configuration directory. It delivers efficiencies in three ways:
- The processes are fully automated, leaving no room for human error
- There’s a one-time setup delivrtu ease of installation
- The periodic rotation of logs using log4j configuration adds structure to your logs
To get the required daily rolling configuration shown above, follow these steps:
Uncomment the already existing DailyRollingFileAppender in the SASWIPServices-log4j.xml file.
In order to prevent conflicting file appenders, you will need to comment out the FileAppender at the top of the SASWIPServices-log4j.xml file
This will have the following effect on your log structure:
Tailored archival and deletion periods allow you to keep only what’s necessary, based on your retention requirements for auditing procedures.
The breakdown of the script is as follows:
An example of the script can be seen below:
How Does this Impact Your System Storage?
To show the impact this solution can have on your SAS estate, Demarq has been monitoring the archive on a SAS deployment on which this solution is in place.
The below space savings of 18 gigabytes from the Workspace Server Logs in just two months can significantly stabilise your SAS environment.
The next part of the blog series delves into how the Elasticsearch, Logstash, and Kibana (ELK) stack can be used for swift log traversal and log visualisation.