com.ccg.io
Class FileLineIndex

java.lang.Object
  com.ccg.io.FileLineIndex

public class FileLineIndex
extends Object

Maintains a index file for an associated ASCII log file allowing one to quickly jump to any line in the file.

Originally I needed to construct a GUI component in Java that would allow one to browse arbitrarily long ASCII files. Unfortunately, this does not seem to come standard with the stock set of Swing components. In order to provide the functionality I wanted, I needed some help with being able to "jump" to any line in a ASCII log file.

This class provides that functionality. It creates a separate index file (with a ".lidx" extension) that is used/maintained to provide a means to quickly jump to the start of any line in the file.

Before using this class, one should be aware of the following conditions:

• It is intended for ASCII log files (it doesn't make sense to use this class with binary data).
• The ASCII log file associated must either remain the same in size or only have additional information appended to the end of the file. You can not use this class if the file contents might be modified (other than new information written to the end of the file).
• This class does not support multiple Java processes sharing the same index file. You will not be happy with the results if you try to have two separate Java programs update the same source/index file (at the moment, I'm not aware of a method for java to do file locking).

If you can live with the above restrictions, you should find that this class works pretty well for you.

To make use of the class, you should think about the file which you will be processing (this will affect how you open the file and what methods you use). Here are some things to consider:

• Do you need to specify where the index file should be written/updated at? Typically you don't care, and can just use the open method which only expects you to specify the source file you want indexed.
• Do you want the index file to persist between sessions? You can set this object such that the index file is deleted when the file is closed - this keeps your disk cleaner, but requires the index file to be regenerated (recreated) if you open the same file at a later point in time.
• Do you expect your file to grow in size as your program runs? If so, you may want to register yourself to be notified when this occurs and start the background update checker which will periodically check for changes in the file size.
• Will you be writing additional information to the end of the file yourself? If so, you may want to use the methods in this class to do so (though it shouldn't be necessary). All writes done in this manner are sure to be thread safe (synchronization is taken care of for you).

Refer to the open javadoc for some more information, or take a quick look at the example program FileLineTest.java.

Since:

1.0

Version:

$Revision: 1.5 $

Author:

$Author: pkb $

See Also:

FileLineTest.java, TailViewer, FileLineIndexListener

Constructor Summary

FileLineIndex()
Bare construction of the object - won't be usable until you open a file.

Method Summary

void	addFileLineIndexListener(FileLineIndexListener flil) Monitor for changes in the source file.
void	appendLinesUpdate(byte[] b, int ofs, int len) Write out one or more lines of text to the end of the log file and then update the index (notifying others).
void	buildIndex() Rebuild the entire index file.
void	clear() Clear the contents of the file and reset the index.
void	close() Close the files used by the object.
long	getFileLength() Get the size of the source file (in bytes).
File	getIndexFile() Get a copy of the index file which was opened and we use as our index into the source file.
String	getLine(long line) Retrieve a particular line from the indexed file.
long	getLineCount() Get the number of lines in the file which is indexed.
File	getSourceFile() Get a copy of the source file which was opened and we read the lines of text from.
int	getUpdateMillis() Get the frequency (how often in milliseconds) we check to see if the file was modified.
boolean	isAppendAllowed() Is one allowed to append information to the end of the source file set?
boolean	isIndexDeletedOnClose() Is the index file to be removed (erased) on close?
boolean	isOk() Is is the object properly initialized and ready to work with?
long	lineToOffset(long line) Get the offset within the source file of a particular line.
void	loadIndex() Try to use an existing index file or rebuild if necessary.
long	offsetToLine(long ofs) Determine the line index of a particular offset in the source file.
void	open(File f) Open and associate a file with the object.
void	open(File src, File idx, boolean allowAppend) Open a source and index file used for processing.
void	removeFileLineIndexListener(FileLineIndexListener flil) Stop monitoring for changes in the source file.
void	setIndexDeletedOnClose(boolean val) Set whether the index file should be deleted when the object is closed.
void	setUpdateMillis(int val) Set the frequency (how often in milliseconds) we check to see if the file was modified.
boolean	tailUpdate() Check to see if new information has been appended to the file and update the object/index accordingly.
String	toString() A string representation of the object for debug output purposes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FileLineIndex

public FileLineIndex()

Bare construction of the object - won't be usable until you open a file.

Since:

1.0

See Also:

open(java.io.File)

Method Detail

toString

public String toString()

A string representation of the object for debug output purposes.

Overrides:

toString in class Object

Returns:

A string representation of the object.

Since:

1.0

isOk

public boolean isOk()

Is is the object properly initialized and ready to work with?

Returns:

This won't return true until you open and then either build or load your associated index file successfully.

isAppendAllowed

public boolean isAppendAllowed()

Is one allowed to append information to the end of the source file set?

When one initially opens the file, they can indicate whether they want to permit one to append lines to the end of the existing source file using the appendLinesUpdate method. This method indicates whether this option is supported or not.

Returns:

true if you are allowed to use the appendLinesUpdate(byte[], int, int) method to add information to the end of the associated file.

See Also:

open(File,File,boolean), appendLinesUpdate(byte[], int, int)

open

public final void open(File f)
                throws IOException

Open and associate a file with the object.

This is a convenience method, it is the same as invoking: open(f,null,true) (it assumes the default index file and that you will want to be able to append new lines to the end of the file).

Parameters:

f - Source file to be opened (must not be null).

Throws:

IOException - If a I/O error is encountered while trying to open your source file or index file.

Since:

1.0

See Also:

open(File,File,boolean)

open

public void open(File src,
                 File idx,
                 boolean allowAppend)
          throws IOException

Open a source and index file used for processing.

This method is used for the following:

• It first closes any open files (if required).
• It associates a source file (to read lines from) with the object.
• It associates a index file to be used to jump to specific lines in the source file.
• It then sets a flag as to whether the source file is to be opened with write access (for the purpose of appending lines to the end).

It is important to note that once a source file is opened, it does not mean that the object will become OK. Before the object is fully usable the index file must also be verified or created. You have several choices for this (most people will want to use either the tailUpdate or setUpdateMillis option:

tailUpdate()

This method is designed to be invoked repeatedly. It is similar to the loadIndex method except that it checks for new information having been appended to the end of the source file. When it discovers new information - it updates the end of the index file (this is a much more efficient than rebuilding the entire index file).

buildIndex()

This method will reconstruct the entire index file (which may take a while) even if it already exists.

loadIndex

This method will check to see if a valid index file already appears to exist, if it appears so, the object will then become usable - if not it will attempt to rebuild the index file.

setUpdateMillis(int millis)

This method of opening the index is intended when you expect external processes to update the source file. It does the following:

• Starts a background thread to do the initial tailUpdate() (in case the index needs to be recreated).
• Adds a schedule entry to the system scheduler to periodically check run the tailUpdate() method to keep the object up to date with the file on disk.

Parameters:

sfile - The source file you will be working with. It must not be null. In addition, the file MUST exist unless you set the allowAppend option to true.

ifile - The index file to use (typically you pass null in which case the index file is determined automatically from the source file name in a consistent manner). NOTE: You may need to specify this if your source file resides in a read only directory (such as a CDROM).

allowAppend - Set to true if you want to be able to write information to the end of the source file (if your application needs to be able to append lines of text). Set to false if you want to open the source file as read only (in which case the source file MUST exist).

Throws:

IOException - If we have problems opening any of the required files.

Since:

1.0

close

public void close()
           throws IOException

Close the files used by the object.

Once closed, the object won't be usable until opened again (and a index is loaded).

Throws:

IOException - If we have a unexpected problem closing one of the open files.

Since:

1.0

getSourceFile

public File getSourceFile()

Get a copy of the source file which was opened and we read the lines of text from.

Returns:

A copy of the source file where the lines of text are read from.

Throws:

IllegalStateException - If you invoke this prior to successfully opening the source file.

Since:

1.0

See Also:

getIndexFile()

getIndexFile

public File getIndexFile()

Get a copy of the index file which was opened and we use as our index into the source file.

Returns:

A copy of the index file where the offsets of each line of the source file are stored.

Throws:

IllegalStateException - If you invoke this prior to successfully opening the source file.

Since:

1.0

See Also:

getSourceFile()

setIndexDeletedOnClose

public void setIndexDeletedOnClose(boolean val)

Set whether the index file should be deleted when the object is closed.

Typically one will want to leave index files around so that they can be re-used between sessions (especially since they take a long time to create for large files or other applications are simultaneously using them). However, if you are using this class with small text files, you may not care that you have to rebuild your index file each time the application starts instead you may want the index files cleaned up for you (in this case you want to invoke this method and pass true).

Parameters:

val - Pass true if you want your index file deleted on close.

See Also:

isIndexDeletedOnClose()

isIndexDeletedOnClose

public boolean isIndexDeletedOnClose()

Is the index file to be removed (erased) on close?

Returns:

If true is returned, it means that the index file will be removed when the object is "closed".

See Also:

setIndexDeletedOnClose(boolean)

getLineCount

public long getLineCount()

Get the number of lines in the file which is indexed.

This method returns the number of lines in the file that is currently indexed.

Returns:

Number of indexed lines which can be retrieved from the file.

Since:

1.0

See Also:

getLine(long)

getFileLength

public long getFileLength()

Get the size of the source file (in bytes).

Returns:

Length of source file in bytes.

Since:

1.0

getLine

public String getLine(long line)
               throws IOException

Retrieve a particular line from the indexed file.

This method is used to retrieve a particular line from the indexed file. The first line of the file has a index of 0 and the last line of the file will have a index of (getLineCount()-1).

Parameters:

idx - Index of line you want to retrieve from the file (uses Java array scheme - index 0 refers to first line of file). You must pass a value in the range [0,(getLineCount()-1)].

Returns:

Line retrieved from the file.

Throws:

IOException - If an unexpected I/O error is encountered while retrieving the line.

Since:

1.0

lineToOffset

public long lineToOffset(long line)
                  throws IOException

Get the offset within the source file of a particular line.

This method is used to determine the starting position of a particular line from the source file. The first line of the file has a index of 0 and the last line of the file will have a index of (getLineCount()-1).

Parameters:

idx - Index of line you want to retrieve the offset for (uses Java array scheme - index 0 refers to first line of file). You must pass a value in the range [0,(getLineCount()-1)].

Returns:

Offset in the source file for the start of the specified line.

Throws:

IOException - If an unexpected I/O error is encountered while retrieving the line.

Since:

1.0

offsetToLine

public long offsetToLine(long ofs)
                  throws IOException

Determine the line index of a particular offset in the source file.

This is the inverse of the lineToOffset method. It takes a arbitrary offset position in the source file and determines what line of the file corresponds to this position.

Parameters:

ofs - The offset in the source file you want to find the line number of (must be in the range [0,(getFileLength()-1]).

Returns:

The line number in the file which contains the specified offset.

Throws:

IOException - If an unexpected I/O error is encountered while retrieving the information.

Since:

1.0

See Also:

lineToOffset(long)

buildIndex

public void buildIndex()
                throws IOException

Rebuild the entire index file.

This method will rebuild the entire index file for the source file. This will involve reading every line from the source file (so it may take a long time).

Throws:

IOException - If an unexpected I/O error is encountered while reading information or writing to the index file.

Since:

1.0

See Also:

open(File,File,boolean)

clear

public void clear()
           throws IOException

Clear the contents of the file and reset the index.

This erases the entire file and resets the corresponding index file.

Throws:

IOException - If an unexpected I/O error occurs while we reset file sizes and rebuild the index file.

Since:

1.0

loadIndex

public void loadIndex()
               throws IOException

Try to use an existing index file or rebuild if necessary.

This method tries to re-use an existing index file (if it is found and appears to be valid), otherwise it will rebuild the index file (which may take a while).

Throws:

IOException - If an unexpected I/O error is encountered while reading information or writing to the index file.

Since:

1.0

See Also:

open(File,File,boolean)

tailUpdate

public boolean tailUpdate()
                   throws IOException

Check to see if new information has been appended to the file and update the object/index accordingly.

You can use this object to monitor a log file which you expect to grow in size - even if the log file is being updated by an external process. Does the following:

• The first time it is invoked, it behaves like loadIndex method (it checks to see if the index already exists, or creates it if not). This can take a long time if the index file needs to be created for a large text file.
• On subsequent invocations, it checks to see if the source file has grown in size. If so, it assumes that the file was only appended to and it updates its internal state and the corresponding index file accordingly. This never takes long as long as you do it fairly often.

Throws:

IOException - If an unexpected I/O error is encountered while reading information or writing to the index file.

Since:

1.0

See Also:

open(File,File,boolean)

setUpdateMillis

public void setUpdateMillis(int val)

Set the frequency (how often in milliseconds) we check to see if the file was modified.

NOTE: If the previous value was zero and you set a new positive value, the following things will occur:

• A background thread will be started for the first loading of the file information (in case it takes a long time due to the need to create the entire index).
• After the information has been loaded, a Schedulable object will be added to the default system scheduler to periodically check and see if the file has been updated (you specify how often).

Parameters:

val - New int value to assign (values less than or equal to 0 will disable the background updating, values greater than 0 will enable it).

See Also:

getUpdateMillis()

getUpdateMillis

public int getUpdateMillis()

Get the frequency (how often in milliseconds) we check to see if the file was modified.

Returns:

Current int value assigned (will be 0 if we are not doing background updates).

See Also:

setUpdateMillis(int)

appendLinesUpdate

public void appendLinesUpdate(byte[] b,
                              int ofs,
                              int len)
                       throws IOException

Write out one or more lines of text to the end of the log file and then update the index (notifying others).

This method safely allows one to:

• Write one or more lines of simple text (standard ASCII - not UNICODE) output to the end of a file (you may include multiple lines by separating each line by a '\n', or '\r\n' or '\r' in the byte array).
• Automatically translates the new line separators to the values used by the operating system (for example '\n' becomes '\r\n' when written to a Windows files).
• Update the internal index.
• Notify others that new information is available.

NOTE: You do NOT need to include a "\n" at the end of your byte stream (unless you want a extra line).

Parameters:

b - Array of bytes to start writing information from (must not be null).

ofs - Starting offset in array to start writing from (must be valid index into array).

len - Number of bytes to write out (this plus the offset must not equal or exceed the array length).

Throws:

IOException - If an I/O issue is encountered while writing information to the system.

Since:

1.0

See Also:

tailUpdate()

addFileLineIndexListener

public void addFileLineIndexListener(FileLineIndexListener flil)

Monitor for changes in the source file.

Some applications may need to be notified when changes occur with the indexed source file. This method can be used to monitor updates. Anytime a update is detected (when the number of index lines changes), all registered listeners are notified about the object which the change occurred on, the prior number of lines available and the new number of lines available.

Parameters:

flil - The listener to register (null is ignored).

Since:

1.0

See Also:

removeFileLineIndexListener(com.ccg.io.FileLineIndexListener)

removeFileLineIndexListener

public void removeFileLineIndexListener(FileLineIndexListener flil)

Stop monitoring for changes in the source file.

Some applications may need to be notified when changes occur with the indexed source file. However, they may only want to do this for a limited amount of time. This method provides a means to remove oneself from being notified of updates (it is the counterpart to the addFileLineIndexListener method).

Parameters:

flil - The listener to un-register (null is ignored).

Since:

1.0

See Also:

addFileLineIndexListener(com.ccg.io.FileLineIndexListener)