Sunday, November 15, 2009

Netbeans Platform custom Log Viewer

If you have complex applications building in some diagnostic components are almost as important as the functionality itself. It's inevitable that an application will give some problems at one time or another, and diagnostics, if done well, will save you a lot of time in avoiding this, foreseeing and preventing this, or after the fact resolving the issue.

The most useful type of diagnostic is probably log messages. One of the most used components I've ever developed was my log4j wrapper. Whenever I write an application it's the first library I add onto my dependency list. I also made it very easy to get an instance to the logger. Assume I create a class called "DataFile", all I do to get a logger is something like this (always the first line in the class):
public class DataFile
{
private static final Log log = Log.getLog(DataFile.class);
}

This will create a log4j logger with the name "package.DataFile". This way I can also configure my logs in a very flexible manner based no package. If you don't know what I mean, log4j allows you to configure log appenders in a hierarchical manner. If you have 2 packages "com.blogspot" and "com.blogspot.qbeukes", you can configure all com.blogspot to goto one location, and com.blogspot.qbeukes to goto another location. This way "com.blogspot.anotherpackage" will goto the same location as com.blogspot, and com.blogspot.qbeukes.subq to goto the same location as com.blogspot.qbeukes. Since it follows the same name patterns as java packages, having my logs created based on a class gives me the flexibility of classifying my logs in a way that groups the same as my code is grouped.

There is one problem with log files though. When a technician is in the field and fails to fix a problem, I often need the log file to analyze exactly what is happening. For this he need to locate and copy the log file. They aren't always comfortable with Linux, so this time around I figured I'd create log file access as a feature into the application. So any administrator user can access the log files and save them onto a removable media right from the application.

What inspired this idea was Netbeans' existing IDE log viewer component. It allows you to view the log output from Netbeans' loggers, and tails the output as it changes. I had a look at how it uses this and tried to implement it myself, only to be disappointed to find out that some of the classes are protected. Luckily the actual mechanism is nothing more than opening a file and piping it's output into Netbeans' existing output windows.

So I copied it's source into a class called LogViewer, create the actions in my layer.xml and got my very own LogViewer. Here's the code if you want to use it. Remember that I copied Netbeans code, so this code is licensed under the Netbeans' licenses. For more info see http://www.netbeans.org/.

First, create a class called LogViewer.java:
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.LinkedList;
import com.blogspot.qbeukes.logging.Log;
import org.openide.util.RequestProcessor;
import org.openide.windows.IOProvider;
import org.openide.windows.InputOutput;

public class LogViewer implements Runnable
{
  private static final Log log = Log.getLog(LogViewer.class);

  boolean shouldStop = false;

  FileInputStream filestream = null;

  BufferedReader ins;

  InputOutput io;

  File fileName;

  String ioName;

  int lines;

  Ring ring;

  private final RequestProcessor.Task task = RequestProcessor.getDefault().create(this);

  /** Connects a given process to the output window. Returns immediately, but threads are started that
   * copy streams of the process to/from the output window.
   * @param process process whose streams to connect to the output window
   * @param ioName name of the output window tab to use
   */
  public LogViewer(final File fileName, final String ioName)
  {
    this.fileName = fileName;
    this.ioName = ioName;
  }

  private void init()
  {
    final int LINES = 2000;
    final int OLD_LINES = 2000;
    ring = new Ring(OLD_LINES);
    String line;

    // Read the log file without
    // displaying everything
    try
    {
      while ((line = ins.readLine()) != null)
      {
        ring.add(line);
      } // end of while ((line = ins.readLine()) != null)
    }
    catch (IOException e)
    {
      log.error("Failed reading log file.", e);
    } // end of try-catch

    // Now show the last OLD_LINES
    lines = ring.output();
    ring.setMaxCount(LINES);
  }

  @Override
  public void run()
  {
    final int MAX_LINES = 10000;
    String line;

    shouldStop = io.isClosed();

    if (!shouldStop)
    {
      try
      {
        if (lines >= MAX_LINES)
        {
          io.getOut().reset();
          lines = ring.output();
        } // end of if (lines >= MAX_LINES)

        while ((line = ins.readLine()) != null)
        {
          if ((line = ring.add(line)) != null)
          {
            io.getOut().println(line);
            lines++;
          } // end of if ((line = ring.add(line)) != null)
        }

      }
      catch (IOException e)
      {
        log.error("Failed reading log file and printing to output.", e);
      }
      task.schedule(10000);
    }
    else
    {
      ///System.out.println("end of infinite loop for log viewer\n\n\n\n");
      stopUpdatingLogViewer();
    }
  }
  
  /* display the log viewer dialog
   *
   **/
  public void showLogViewer() throws IOException
  {
    shouldStop = false;
    io = IOProvider.getDefault().getIO(ioName, false);
    io.getOut().reset();
    io.select();
    filestream = new FileInputStream(fileName);
    ins = new BufferedReader(new InputStreamReader(filestream));

    init();
    task.schedule(0);
  }

  /* stop to update  the log viewer dialog
   *
   **/
  public void stopUpdatingLogViewer()
  {
    try
    {
      ins.close();
      filestream.close();
      io.closeInputOutput();
      io.setOutputVisible(false);
    }
    catch (IOException e)
    {
      log.error("Failed to close log file streams.", e);
    }
  }

  private class Ring
  {
    private int maxCount;

    private int count;

    private LinkedList<String> anchor;

    public Ring(int max)
    {
      maxCount = max;
      count = 0;
      anchor = new LinkedList<String>();
    }

    public String add(String line)
    {
      if (line == null || line.equals(""))
      { // NOI18N
        return null;
      } // end of if (line == null || line.equals(""))

      while (count >= maxCount)
      {
        anchor.removeFirst();
        count--;
      } // end of while (count >= maxCount)

      anchor.addLast(line);
      count++;

      return line;
    }

    public void setMaxCount(int newMax)
    {
      maxCount = newMax;
    }

    public int output()
    {
      int i = 0;
      for (String s : anchor)
      {
        io.getOut().println(s);
        i++;
      }

      return i;
    }

    public void reset()
    {
      anchor = new LinkedList<String>();
    }
  }
}

You also need an action that gathers the basic logic to open a log viewer, called LogViewerAction.java:
package com.blogspot.qbeukes.diagnostics.logviewer;

import java.io.File;
import java.util.Map;
import java.util.Map.Entry;
import com.blogspot.qbeukes.logging.Log;
import net.kunye.platform.client.maintenance.logs.LogViewer;
import org.openide.util.HelpCtx;
import org.openide.util.actions.CallableSystemAction;

public abstract class LogViewerAction extends CallableSystemAction
{
  protected static final Log log = Log.getLog(LogViewerAction.class);

  /**
   * @param map
   * @return A new prepared log viewer action
   */
  public static LogViewerAction getLogViewerAction(Map map)
  {
    Object o = map.get("viewerAction");
    if (!(o instanceof LogViewerAction))
    {
      log.error("Received an invalid viewerAction type: " + o.getClass().getName());
      throw new IllegalArgumentException("Invalid viewerAction attribute. Has to be a 'newvalue' of type LogViewerAction.");
    }

    LogViewerAction logViewer = (LogViewerAction)o;
    log.debug("Creating new action of type: " + logViewer.getClass().getName());
   
    for (Object entry : map.entrySet())
    {
      Object key = ((Entry)entry).getKey();
     
      if ("instanceCreate".equals(key)
          || "viewerAction".equals(key))
      {
        continue;
      }

      // we have a valid property, add it
      Object value = ((Entry)entry).getValue();
      logViewer.putValue((String)key, value);

      // log it
      if (value == null)
      {
        value = "null";
      }
      log.trace("Adding LogViewerAction attribute '" + (String)key + "'='" +
          value.toString() + "' (" + logViewer.getClass().getName() + ").");
    }

    return logViewer;
  }

  public void viewLog(File f)
  {
    LogViewer p = new LogViewer(f, getName());
    try
    {
      p.showLogViewer();
    }
    catch (java.io.IOException e)
    {
      log.error("Showing log action failed.", e);
    }
  }

  @Override
  public String getName()
  {
    return (String)getValue("displayName");
  }

  @Override
  public HelpCtx getHelpCtx()
  {
    return HelpCtx.DEFAULT_HELP;
  }

  @Override public String iconResource()
  {
    return "org/netbeans/core/resources/log-file.gif"; // NOI18N
  }
}

Then extend the LogViewerAction class for a specific log. For this one I'll just open the client's log again:
package com.blogspot.qbeukes.diagnostics.logviewer;

import java.io.File;

public class ClientLogAction extends LogViewerAction
{
  public ClientLogAction()
  {
    putValue("userDirLogFile", "/var/log/messages.log");
  }
 
  @Override
  public void performAction()
  {
    String logFilename = (String)getValue("logFile");

    if (logFilename == null)
    {
      // FIXME This may not be used this way anymore.
      String userDir = System.getProperty("netbeans.user");
      if (userDir == null)
      {
        return;
      }
      // FIXME the same as above
      logFilename = userDir + getValue("userDirLogFile");
    }

    log.debug("Viewing client log file: " + logFilename);
    File f = new File(logFilename);
    viewLog(f);
  }
}

Finally, add the action to your layer.xml:
...
  <folder name="Menu">
    <folder name="View">
      <file name="org-netbeans-core-actions-LogAction.shadow_hidden"/>
      <file name="com-blogspot-qbeukes-diagnostics-logviewer-ClientLogAction.instance">
        <attr name="instanceCreate" methodvalue="com.blogspot.qbeukes.diagnostics.logviewer.LogViewerAction.getLogViewerAction"/>
        <attr name="viewerAction" newvalue="com.blogspot.qbeukes.diagnostics.logviewer.ClientLogAction"/>
        <attr name="userDirLogFile" stringvalue="/var/log/messages.log"/>
        <attr name="displayName" stringvalue="Client Log"/>
      </file>
    </folder>
  </folder>
...

To explain the layer a bit. Firstly, the LogViewerAction has a factory method for creating and executing a specific action. With a more complex LogViewerAction you should be able to creating a separate class for each log. Though as you see opening the ClientLog requires accessing properties and some validation and defaults. For other logs it is even more complex. So it would have to be quite the complex class to provide opening of all kinds of log files.

So for your own class you can leave everything as it and just change the 'viewerAction' attribute to point to your own class. You can also remove or change the 'userDirLogFile' attribute, which is used inside the ClientLogAction class to reference a specific log under Netbeans' "userdir", which is a directory created for each user to store state/cache/configuration/logs.

Finally it's the displayName property, which will be used as the name of the Action and LogViewer window, in other words displayed on the menu and as the title of the log viewer.

Beyond this all logs aren't located on the same machine as the client application. The most important logs are located on the server (in our case at least). So I also made a network log viewer for viewing log files on the server. I developed this in company time, though, so I can't publish it (since they own the code). This one is very useful as it supports viewing rotated logs by specifying date ranges and exception highlighting. I will make one at some point and share it.

1 comment:

daftmock said...

Wow, cool post! I've been able to take your sample code and tweak it enough into a simple plugin. I'd love to add exception highlighting, any thoughts on where I should start looking?