SailPoint proper way to construct a rolling log file

There is some confusion of how to set up the log4j2.properties file in order to log to a rolling log file.  Here is what the OOTB log4j2.properties file shows:

# Below is an example of how to create a logger that writes to a file.

# Uncomment the following five lines, then uncomment the 

# rootLogger.appenderRef.file.ref definition below

#appender.file.type=File

#appender.file.name=file

#appender.file.fileName=C:/Windows/Temp/sailpoint.log

#appender.file.layout.type=PatternLayout

#appender.file.layout.pattern=%d{ISO8601} %5p %t %c{4}:%L - %m%n

This setup does not roll.  The file also contains the following:

#appender.meter.type=RollingFile

#appender.meter.name=meter

#appender.meter.fileName=C:/Windows/Temp/meter.log

#appender.meter.filePattern=C:/Windows/Temp/meter-%d{yyyy-MM-dd}-%i.log.gz"

#appender.meter.layout.type=PatternLayout

#appender.meter.layout.pattern=%m%n

#appender.meter.policies.type=Policies

#appender.meter.policies.size.type=SizeBasedTriggeringPolicy

#appender.meter.policies.size.size=10MB

#appender.meter.strategy.type=DefaultRolloverStrategy

#appender.meter.strategy.max=5

The main issue with this is the use of the date and the gzip options.  It also doesn't have the proper pattern layout.

The best practice is something like this:

appender.rolling.type=RollingFile

appender.rolling.name=rolling

appender.rolling.fileName=D:/iiq83/logs/sailpoint.log

appender.rolling.filePattern=D:/iiq83/logs/sailpoint-%i.log"

appender.rolling.layout.type=PatternLayout

appender.rolling.layout.pattern=%d{ISO8601} %5p %t %c{4}:%L - %m%n

appender.rolling.policies.type=Policies

appender.rolling.policies.size.type=SizeBasedTriggeringPolicy

appender.rolling.policies.size.size=20MB

appender.rolling.strategy.type=DefaultRolloverStrategy

appender.rolling.strategy.max=5

The word "rolling" can be substituted by any other name.  Be sure to leave out the date and the gzip references in the filePattern part.

Some advice: Never use the time based policy.  Never use the startup policy.  With this method you can archive files by their age.

Usage for this includes the following to make sure the files are correctly being written to:

rootLogger.level=warn

rootLogger.appenderRef.stdout.ref=stdout

rootLogger.appenderRef.rolling.ref=rolling

And all logging should be structured like this:

logger.objexp.name=com.sailpoint.objectexporter.task

logger.objexp.level=trace

logger.objexp.appenderRef.rolling.ref=rolling

logger.objexp.additivity=false

The second portion of this block, must be unique.

The file name you chose, should be provided in the third line in two places as shown.

SailPoint logging best practice

SailPoint recommends using the log4j calls for logging.  This is in contrast to the occasional use of commons logging in some of their codes.  Just for reference:

import org.apache.log4j.Logger;

Logger alog=Logger.getLogger("com.sailpoint.class.module");

The above creates the Logger class and uses log4j (BEST)

import org.apache.commons.logging.Log;

import org.apache.commons.logging.LogFactory;

Log alog=LogFactory.getLog("com.sailpoint.class.module");

The second example uses the commons logging and is not best, but works.  Commons logging actually uses log4j so it's a needless extra layer.

Providing proper construction, level, and content for log statements is important.  Why do we use logging instead of a debugger? Because for most web applications, you can't pause the program to inspect its variables.  Logging is the best option. 

Some people, generally those who don't understand log4j, will start with a statement like:

log.error("I am here");

and then later something like:

log.error("Now I am here");

Let's examine the errors in this method of troubleshooting:

1) Use of an OOTB log object.

2) Using error call for simple debugging.

3) General lack of information about "where" you are in the program.

4) Lack of any meaningful information in the log output.

In more detail:

1) Never use an OOTB log object, always create a new one.  I always call mine alog but you could call your blog or clog, go nuts.  I have seen plogger and others, but I don't like to type so alog fits my style.  You could overwrite log also but I don't recommend that.

2) Learn the log levels and their meaning.  TRACE is the most verbose and should be confined to inner loops in iterative code, that you would expect to print if there is a very insidious logic issue you are trying to track down.  DEBUG is best for debugging code and should be used for most log statements.  INFO is rarely used but could be used for method entry statements. WARN is the first level of statements that the OOTB log setup will always print, and should be a warning. ERROR should be used in catch blocks and anywhere a severe error is found.  Their respective methods, trace(), debug(), info(), warn(), and error() send this data to the logger.  You have to learn how to set the log levels in the log4j2.properties file (or log4j.properties for older versions).

3) In the text of your log statement, provide a coded prefix or "tag" that indicates the initials of the client, a 3 or 4 character module code, and a unique number.  For example, for Acme's WorkDay customization rule, the prefix on the first log statement will be something like "ACM-WDC-001" and the final logging would be something like:

String fileNumber=(String)object.getAttribute("FILENUMBER");

alog.debug("ACM-WDC-001 Entered WorkDay Customization rule for user "+fileNumber);

Each different module has a unique 3 or 4 character module code and each log statement has a unique number, which does not have to be strictly sequential.

4) Notice above that the log statement contains information about what is happening, rather than just a dead line "I am here".  Always try to include data and don't forget to null check things before printing.

Use of these techniques also makes troubleshooting with your SIEM more effective.

Formatting dates in Linux for sensible log file names

I like to make my log filenames have date stamps and so in my shell scripts I typically will create a log file name at the start of the script:

# Create name for log file
timenow=`date +%Y%m%d-%H%M%S`
logfilename="utils-${timenow}.log"

and the time will now be included in the log filename.

You can remove the %S to take off the seconds if you want.

Sharing log files with users who are not in the oinstall group

Sometimes a client wants to be able for a user to view log files for an Oracle application.  There are many ways to do this:

1. Give the user sudo rights to the oracle user.
2. Put the user in the oinstall group (assuming that was the default group used in the installation for the oracle user)
3. Open up the umask to 0022 so that any user can read the files.
4. Do the following:

First, you need to give read access to all of the folders in the chain.  Let's say you have a middleware home of:

/u01/oracle/products/middleware

and in there you have a domain home of

$MW_HOME/user_projects/domains/oim_domain

and in there you have a server

$DOMAIN_HOME/servers/oim_server1

In this case every folder between /u01 and oim_server1 would have to be granted 755 privileges.  It is easy enough to just go through and chmod each folder in order and then check from a user who has not been granted any of 1-3.

Next, the umask in the .bash_profile does have to be 0027 or better for people to read the files if they are in the correct group.

To make this work here is what needs to happen:

As root, execute the command:
# groupadd oshare
(I made up that group name oshare but you can call it whatever you want).
# usermod -a -G oshare oracle
# usermod -a -G oshare username
(username is the user you want to share files with)
# cd <that oim_server1 folder>
# chown -R oracle:oshare logs
# chmod -R 2755 logs

That should do it.  I have not tested this.

To reverse this go back and perform:
# chown -R oracle:oinstall logs

If you want the user to be able to delete files and not just read them, change the 2755 above to 2775.
You will have to do this in any log folder you want to share.  I would not advise sharing any other folder.  This does include the ADR folders.

Getting java.util.logging to work with JUnit

In my development I normally create the JUnit test cases, but had trouble getting the java.util.logging statement to activate in the code I was testing.  I discovered a couple articles on the web and pieced them together for a solution.

As a reference, here is how I normally implement logging, with my OIM Flat File Connector XMLParser as the class:

import java.util.logging.*;

public class FlatFileXMLParser implements FlatFileParser {
    private static final Logger logger = 
        Logger.getLogger(FlatFileXMLParser.class.getName());

public void parse(File flatFile, FlatFileRecordHandler recordHandler,
                  ParserConfig config) throws Exception {
  String methodName="parse";
  logger.logp(Level.FINE, getClass().getName(), methodName,
    "FF-XMLP-001 entering");

Some explanation:
1) I use java.util.logging and never use log4j.
2) I use logp - never anything else.  One statement=commonality
3) I define String methodName to provide the method name in every module.
4) I add tags so that I can grep on the tags.  Each statement gets a tag.
5) Increment the numbers in the method and skip to next 100 on next method.
6) Debug statements as Level.FINE, use good judgement.

When I attempt to test these code modules I found that the logging was not being generated.  I found a great writeup and took most of this from it.  What I did was put the following into the JUnit class - not in the functional class:

public class AppTest extends TestCase {
  static {
    Logger rootLogger = Logger.getLogger("");
    System.setProperty("java.util.logging.SimpleFormatter.format",
           "[%1$tF %1$tr] [%2$s] %4$s: %5$s %n");
    for(Handler handler : rootLogger.getHandlers()) {
      handler.setLevel(Level.FINEST);
      handler.setFormatter(new SimpleFormatter());
    }
    rootLogger.setLevel(Level.FINEST);
  }

After this is the constructor and the test methods.  The logger will record the details to the output screen and you can track your code.

Good luck testing !!