HPSF HOW-TOHow To Use the HPSF APIsThis HOW-TO is organized in three sections. You should read them sequentially because the later sections build upon the earlier ones.
Reading Standard PropertiesThis section explains how to read the most important standard properties of a Microsoft Office document. Standard properties are things like title, author, creation date etc. Chances are that you will find here what you need and don't have to read the other sections. The first thing you should understand is that properties are stored in separate documents inside the POI filesystem. (If you don't know what a POI filesystem is, read the POIFS documentation.) A document in a POI filesystem is also called a stream. The following example shows how to read a POI filesystem's "title" property. Reading other properties is similar. Consider the API documentation of org.apache.poi.hpsf.SummaryInformation to learn which methods are available! The standard properties this section focusses on can be found in a document called \005SummaryInformation located in the root of the POI filesystem. The notation \005 in the document's name means the character with the decimal value of 5. In order to read the title, an application has to perform the following steps:
Sounds easy, doesn't it? Here are the steps in detail. Open the document \005SummaryInformation in the root of the POI filesystemAn application that wants to open a document in a POI filesystem (POIFS) proceeds as shown by the following code fragment. (The full source code of the sample application is available in the examples section of the POI source tree as ReadTitle.java. import java.io.*; import org.apache.poi.hpsf.*; import org.apache.poi.poifs.eventfilesystem.*; // ... public static void main(String[] args) throws IOException { final String filename = args[0]; POIFSReader r = new POIFSReader(); r.registerListener(new MyPOIFSReaderListener(), "\005SummaryInformation"); r.read(new FileInputStream(filename)); } The first interesting statement is POIFSReader r = new POIFSReader(); It creates a org.apache.poi.poifs.eventfilesystem.POIFSReader instance which we shall need to read the POI filesystem. Before the application actually opens the POI filesystem we have to tell the POIFSReader which documents we are interested in. In this case the application should do something with the document \005SummaryInformation. r.registerListener(new MyPOIFSReaderListener(), "\005SummaryInformation"); This method call registers a org.apache.poi.poifs.eventfilesystem.POIFSReaderListener with the POIFSReader. The POIFSReaderListener interface specifies the method processPOIFSReaderEvent which processes a document. The class MyPOIFSReaderListener implements the POIFSReaderListener and thus the processPOIFSReaderEvent method. The eventing POI filesystem calls this method when it finds the \005SummaryInformation document. In the sample application MyPOIFSReaderListener is a static class in the ReadTitle.java source file. Now everything is prepared and reading the POI filesystem can start: r.read(new FileInputStream(filename)); The following source code fragment shows the MyPOIFSReaderListener class and how it retrieves the title. static class MyPOIFSReaderListener implements POIFSReaderListener { public void processPOIFSReaderEvent(POIFSReaderEvent event) { SummaryInformation si = null; try { si = (SummaryInformation) PropertySetFactory.create(event.getStream()); } catch (Exception ex) { throw new RuntimeException ("Property set stream \"" + event.getPath() + event.getName() + "\": " + ex); } final String title = si.getTitle(); if (title != null) System.out.println("Title: \"" + title + "\""); else System.out.println("Document has no title."); } } The line SummaryInformation si = null; declares a SummaryInformation variable and initializes it with null. We need an instance of this class to access the title. The instance is created in a try block: si = (SummaryInformation) PropertySetFactory.create(event.getStream()); The expression event.getStream() returns the input stream containing the bytes of the property set stream named \005SummaryInformation. This stream is passed into the create method of the factory class org.apache.poi.hpsf.PropertySetFactory which returns a org.apache.poi.hpsf.PropertySet instance. It is more or less safe to cast this result to SummaryInformation, a convenience class with methods like getTitle(), getAuthor() etc. The PropertySetFactory.create method may throw all sorts of exceptions. We'll deal with them in the next sections. For now we just catch all exceptions and throw a RuntimeException containing the message text of the origin exception. If all goes well, the sample application retrieves the title and prints it to the standard output. As you can see you must be prepared for the case that the POI filesystem does not have a title. final String title = si.getTitle(); if (title != null) System.out.println("Title: \"" + title + "\""); else System.out.println("Document has no title."); Please note that a Microsoft Office document does not necessarily contain the \005SummaryInformation stream. The documents created by the Microsoft Office suite have one, as far as I know. However, an Excel spreadsheet exported from StarOffice 5.2 won't have a \005SummaryInformation stream. In this case the applications won't throw an exception but simply does not call the processPOIFSReaderEvent method. You have been warned! Additional Standard Properties, Exceptions And Embedded ObjectsThis section focusses on reading additional standard properties. It also talks about exceptions that may be thrown when dealing with HPSF and shows how you can read properties of embedded objects. A couple of additional standard properties are not contained in the \005SummaryInformation stream explained above, for example a document's category or the number of multimedia clips in a PowerPoint presentation. Microsoft has invented an additional stream named \005DocumentSummaryInformation to hold these properties. With two minor exceptions you can proceed exactly as described above to read the properties stored in \005DocumentSummaryInformation:
And of course you cannot call getTitle() because DocumentSummaryInformation has different query methods. See the Javadoc API documentation for the details! In the previous section the application simply caught all exceptions and was in no way interested in any details. However, a real application will likely want to know what went wrong and act appropriately. Besides any IO exceptions there are three HPSF resp. POI specific exceptions you should know about:
Many Microsoft Office documents contain embedded objects, for example an Excel sheet on a page in a Word document. Embedded objects may have property sets of their own. An application can open these property set streams as described above. The only difference is that they are not located in the POI filesystem's root but in a nested directory instead. Just register a POIFSReaderListener for the property set streams you are interested in. For example, the POIBrowser application in the contrib section tries to open each and every document in a POI filesystem as a property set stream. If this operation was successful it displays the properties. Reading Non-Standard PropertiesThis section tells how to read non-standard properties. Non-standard properties are application-specific ID/type/value triples. OverviewNow comes the real hardcode stuff. As mentioned above, SummaryInformation and DocumentSummaryInformation are just special cases of the general concept of a property set. This concept says that a property set consists of properties and that each property is an entity with an ID, a type, and a value. Okay, that was still rather easy. However, to make things more complicated, Microsoft in its infinite wisdom decided that a property set shalt be broken into one or more sections. Each section holds a bunch of properties. But since that's still not complicated enough, a section may have an optional dictionary that maps property IDs to property names - we'll explain later what that means. The procedure to get to the properties is the following:
A Sample ApplicationLet's have a look at a sample Java application that dumps all property set streams contained in a POI file system. The full source code of this program can be found as ReadCustomPropertySets.java in the examples area of the POI source code tree. Here are the key sections: import java.io.*; import java.util.*; import org.apache.poi.hpsf.*; import org.apache.poi.poifs.eventfilesystem.*; import org.apache.poi.util.HexDump; The most important package the application needs is org.apache.poi.hpsf.*. This package contains the HPSF classes. Most classes named below are from the HPSF package. Of course we also need the POIFS event file system's classes and java.io.* since we are dealing with POI I/O. From the java.util package we use the List and Iterator class. The class org.apache.poi.util.HexDump provides a methods to dump byte arrays as nicely formatted strings. public static void main(String[] args) throws IOException { final String filename = args[0]; POIFSReader r = new POIFSReader(); /* Register a listener for *all* documents. */ r.registerListener(new MyPOIFSReaderListener()); r.read(new FileInputStream(filename)); } The POIFSReader is set up in a way that the listener MyPOIFSReaderListener is called on every file in the POI file system. The Property SetThe listener class tries to create a PropertySet from each stream using the PropertySetFactory.create() method: static class MyPOIFSReaderListener implements POIFSReaderListener { public void processPOIFSReaderEvent(POIFSReaderEvent event) { PropertySet ps = null; try { ps = PropertySetFactory.create(event.getStream()); } catch (NoPropertySetStreamException ex) { out("No property set stream: \"" + event.getPath() + event.getName() + "\""); return; } catch (Exception ex) { throw new RuntimeException ("Property set stream \"" + event.getPath() + event.getName() + "\": " + ex); } /* Print the name of the property set stream: */ out("Property set stream \"" + event.getPath() + event.getName() + "\":"); Creating the PropertySet is done in a try block, because not each stream in the POI file system contains a property set. If it is some other file, the PropertySetFactory.create() throws a NoPropertySetStreamException, which is caught and logged. Then the program continues with the next stream. However, all other types of exceptions cause the program to terminate by throwing a runtime exception. If all went well, we can print the name of the property set stream. The SectionsThe next step is to print the number of sections followed by the sections themselves: /* Print the number of sections: */ final long sectionCount = ps.getSectionCount(); out(" No. of sections: " + sectionCount); /* Print the list of sections: */ List sections = ps.getSections(); int nr = 0; for (Iterator i = sections.iterator(); i.hasNext();) { /* Print a single section: */ Section sec = (Section) i.next(); // See below for the complete loop body. } The PropertySet's method getSectionCount() returns the number of sections. To retrieve the sections, use the getSections() method. This method returns a java.util.List containing instances of the Section class in their proper order. The sample code shows a loop that retrieves the Section objects one by one and prints some information about each one. Here is the complete body of the loop: /* Print a single section: */ Section sec = (Section) i.next(); out(" Section " + nr++ + ":"); String s = hex(sec.getFormatID().getBytes()); s = s.substring(0, s.length() - 1); out(" Format ID: " + s); /* Print the number of properties in this section. */ int propertyCount = sec.getPropertyCount(); out(" No. of properties: " + propertyCount); /* Print the properties: */ Property[] properties = sec.getProperties(); for (int i2 = 0; i2 < properties.length; i2++) { /* Print a single property: */ Property p = properties[i2]; int id = p.getID(); long type = p.getType(); Object value = p.getValue(); out(" Property ID: " + id + ", type: " + type + ", value: " + value); } The Section's Format IDThe first method called on the Section instance is getFormatID(). As explained above, the format ID of the first section in a property set determines the type of the property set. Its type is ClassID which is essentially a sequence of 16 bytes. A real application using its own type of a custom property set should have defined a unique format ID and, when reading a property set stream, should check the format ID is equal to that unique format ID. The sample program just prints the format ID it finds in a section: String s = hex(sec.getFormatID().getBytes()); s = s.substring(0, s.length() - 1); out(" Format ID: " + s); As you can see, the getFormatID() method returns a ClassID object. An array containing the bytes can be retrieved with ClassID.getBytes(). In order to get a nicely formatted printout, the sample program uses the hex() helper method which in turn uses the POI utility class HexDump in the org.apache.poi.util package. Another helper method is out() which just saves typing System.out.println(). The PropertiesBefore getting the properties, it is possible to find out how many properties are available in the section via the Section.getPropertyCount(). The sample application uses this method to print the number of properties to the standard output: int propertyCount = sec.getPropertyCount(); out(" No. of properties: " + propertyCount); Now its time to get to the properties themselves. You can retrieve a section's properties with the method Section.getProperties(): Property[] properties = sec.getProperties(); As you can see the result is an array of Property objects. This class has three methods to retrieve a property's ID, its type, and its value. The following code snippet shows how to call them: for (int i2 = 0; i2 < properties.length; i2++) { /* Print a single property: */ Property p = properties[i2]; int id = p.getID(); long type = p.getType(); Object value = p.getValue(); out(" Property ID: " + id + ", type: " + type + ", value: " + value); } Sample OutputThe output of the sample program might look like the following. It shows the summary information and the document summary information property sets of a Microsoft Word document. However, unlike the first and second section of this HOW-TO the application does not have any code which is specific to the SummaryInformation and DocumentSummaryInformation classes. Property set stream "/SummaryInformation": No. of sections: 1 Section 0: Format ID: 00000000 F2 9F 85 E0 4F F9 10 68 AB 91 08 00 2B 27 B3 D9 ....O..h....+'.. No. of properties: 17 Property ID: 1, type: 2, value: 1252 Property ID: 2, type: 30, value: Titel Property ID: 3, type: 30, value: Thema Property ID: 4, type: 30, value: Rainer Klute (Autor) Property ID: 5, type: 30, value: Test (Stichw�rter) Property ID: 6, type: 30, value: This is a document for testing HPSF Property ID: 7, type: 30, value: Normal.dot Property ID: 8, type: 30, value: Unknown User Property ID: 9, type: 30, value: 3 Property ID: 18, type: 30, value: Microsoft Word 9.0 Property ID: 12, type: 64, value: Mon Jan 01 00:59:25 CET 1601 Property ID: 13, type: 64, value: Thu Jul 18 16:22:00 CEST 2002 Property ID: 14, type: 3, value: 1 Property ID: 15, type: 3, value: 20 Property ID: 16, type: 3, value: 93 Property ID: 19, type: 3, value: 0 Property ID: 17, type: 71, value: [B@13582d Property set stream "/DocumentSummaryInformation": No. of sections: 2 Section 0: Format ID: 00000000 D5 CD D5 02 2E 9C 10 1B 93 97 08 00 2B 2C F9 AE ............+,.. No. of properties: 14 Property ID: 1, type: 2, value: 1252 Property ID: 2, type: 30, value: Test Property ID: 14, type: 30, value: Rainer Klute (Manager) Property ID: 15, type: 30, value: Rainer Klute IT-Consulting GmbH Property ID: 5, type: 3, value: 3 Property ID: 6, type: 3, value: 2 Property ID: 17, type: 3, value: 111 Property ID: 23, type: 3, value: 592636 Property ID: 11, type: 11, value: false Property ID: 16, type: 11, value: false Property ID: 19, type: 11, value: false Property ID: 22, type: 11, value: false Property ID: 13, type: 4126, value: [B@56a499 Property ID: 12, type: 4108, value: [B@506411 Section 1: Format ID: 00000000 D5 CD D5 05 2E 9C 10 1B 93 97 08 00 2B 2C F9 AE ............+,.. No. of properties: 7 Property ID: 0, type: 0, value: {6=Test-JaNein, 5=Test-Zahl, 4=Test-Datum, 3=Test-Text, 2=_PID_LINKBASE} Property ID: 1, type: 2, value: 1252 Property ID: 2, type: 65, value: [B@c9ba38 Property ID: 3, type: 30, value: This is some text. Property ID: 4, type: 64, value: Wed Jul 17 00:00:00 CEST 2002 Property ID: 5, type: 3, value: 27 Property ID: 6, type: 11, value: true No property set stream: "/WordDocument" No property set stream: "/CompObj" No property set stream: "/1Table" There are some interesting items to note:
Property IDsProperties in the same section are distinguished by their IDs. This is similar to variables in a programming language like Java, which are distinguished by their names. But unlike variable names, property IDs are simple integral numbers. There is another similarity, however. Just like a Java variable has a certain scope (e.g. a member variables in a class), a property ID also has its scope of validity: the section. Two property IDs in sections with different section format IDs don't have the same meaning even though their IDs might be equal. For example, ID 4 in the first (and only) section of a summary information property set denotes the document's author, while ID 4 in the first section of the document summary information property set means the document's byte count. The sample output above does not show a property with an ID of 4 in the first section of the document summary information property set. That means that the document does not have a byte count. However, there is a property with an ID of 4 in the second section: This is a user-defined property ID - we'll get to that topic in a minute. So, how can you find out what the meaning of a certain property ID in the summary information and the document summary information property set is? The standard property sets as such don't have any hints about the meanings of their property IDs. For example, the summary information property set does not tell you that the property ID 4 stands for the document's author. This is external knowledge. Microsoft defined standard meanings for some of the property IDs in the summary information and the document summary information property sets. As a help to the Java and POI programmer, the class PropertyIDMap in the org.apache.poi.hpsf.wellknown package defines constants for the "well-known" property IDs. For example, there is the definition public final static int PID_AUTHOR = 4; These definitions allow you to use symbolic names instead of numbers. In order to provide support for the other way, too, - i.e. to map property IDs to property names - the class PropertyIDMap defines two static methods: getSummaryInformationProperties() and getDocumentSummaryInformationProperties(). Both return java.util.Map objects which map property IDs to strings. Such a string gives a hint about the property's meaning. For example, PropertyIDMap.getSummaryInformationProperties().get(4) returns the string "PID_AUTHOR". An application could use this string as a key to a localized string which is displayed to the user, e.g. "Author" in English or "Verfasser" in German. HPSF might provide such language-dependend ("localized") mappings in a later release. Usually you won't have to deal with those two maps. Instead you should call the Section.getPIDString(int) method. It returns the string associated with the specified property ID in the context of the Section object. Above you learned that property IDs have a meaning in the scope of a section only. However, there are two exceptions to the rule: The property IDs 0 and 1 have a fixed meaning in all sections:
Property typesA property is nothing without its value. It is stored in a property set stream as a sequence of bytes. You must know the property's type in order to properly interpret those bytes and reasonably handle the value. A property's type is one of the so-called Microsoft-defined "variant types". When you call Property.getType() you'll get a long value which denoting the property's variant type. The class Variant in the org.apache.poi.hpsf package holds most of those long values as named constants. For example, the constant VT_I4 = 3 means a signed integer value of four bytes. Examples of other types are VT_LPSTR = 30 meaning a null-terminated string of 8-bit characters, VT_LPWSTR = 31 which means a null-terminated Unicode string, or VT_BOOL = 11 denoting a boolean value. In most cases you won't need a property's type because HPSF does all the work for you. Property valuesWhen an application wants to retrieve a property's value and calls Property.getValue(), HPSF has to interpret the bytes making out the value according to the property's type. The type determines how many bytes the value consists of and what to do with them. For example, if the type is VT_I4, HPSF knows that the value is four bytes long and that these bytes comprise a signed integer value in the little-endian format. This is quite different from e.g. a type of VT_LPWSTR. In this case HPSF has to scan the value bytes for a Unicode null character and collect everything from the beginning to that null character as a Unicode string. The good new is that HPSF does another job for you, too: It maps the variant type to an adequate Java type.
The bad news is that there are still a couple of variant types HPSF does not yet support. If it encounters one of these types it returns the property's value as a byte array and leaves it to be interpreted by the application. An application retrieves a property's value by calling the Property.getValue() method. This method's return type is the abstract Object class. The getValue() method looks up the property's variant type, reads the property's value bytes, creates an instance of an adequate Java type, assigns it the property's value and returns it. Primitive types like int or long will be returned as the corresponding class, e.g. Integer or Long. DictionariesThe property with ID 0 has a very special meaning: It is a dictionary mapping property IDs to property names. We have seen already that the meanings of standard properties in the summary information and the document summary information property sets have been defined by Microsoft. The advantage is that the labels of properties like "Author" or "Title" don't have to be stored in the property set. However, a user can define custom fields in, say, Microsoft Word. For each field the user has to specify a name, a type, and a value. The names of the custom-defined fields (i.e. the property names) are stored in the document summary information second section's dictionary. The dictionary is a map which associates property IDs with property names. The method Section.getPIDString(int) not only returns with the well-known property names of the summary information and document summary information property sets, but with self-defined properties, too. It should also work with self-defined properties in self-defined sections. Codepage supportThe property with ID 1 holds the number of the codepage which was used to encode the strings in this section. The present HPSF codepage support is still very limited: When reading property value strings, HPSF distinguishes between 16-bit characters and 8-bit characters. 16-bit characters should be Unicode characters and thus be okay. 8-bit characters are interpreted according to the platform's default character set. This is fine as long as the document being read has been written on a platform with the same default character set. However, if you receive a document from another region of the world and want to process it with HPSF you are in trouble - unless the creator used Unicode, of course. Further ReadingThere are still some aspects of HSPF left which are not covered by this HOW-TO. You should dig into the Javadoc API documentation to learn further details. Since you've struggled through this document up to this point, you are well prepared. |