Digital Photo Slide Show 2005.2 serial key or number

Oracle® Outside In Clean Content Developer's Guide

Release 8.5

F11001-02

Skip Navigation Links

Introduction

Definitions

Features

File Formats

Support

Release Notes

SDK layout

Architecture

Java

C/C++

.NET

Using the API

Initialization

Basic Use

Request

Response

Document IO

Targets

Analysis and Scrubbing

Extraction

Embedding Recursion

Embedding Export

Embedding Replacement

PowerPoint Disassembly/Assembly

Threading

Exception Handling

Install and Coding Guidelines

Java

C/C++

.NET

Technical Notes

The Outside In Clean Content SDK provides all the components, documentation, samples and other resources required by third party developers to integrate Oracle's document analysis, scrubbing, extraction and export technology into their own applications.

Definitions

The following definitions are used throughout this documentation and the Clean Content API.

document
This term is used broadly and generically in the documentation and API to refer to any file such as a word processing document, a spreadsheet, a presentation, a PDF, etc.

target
Some feature or piece of information in a document that can be identified and in many cases removed (see scrub below). Most targets relate to well known security risks in popular file formats although some, like identifying a document as being encrypted, are more general.

analyze
To determine if a given target exists in the document

scrub
To remove a given target from a document

extract
To provide the developer with text, structure and other information in the document

export
To copy objects, images and other artifacts embedded in a document to standalone files

disassembly
To take a document with multiple parts (slides in PowerPoint is the only current example) and split it into multiple standalone documents, one per part

assembly
To take several documents and merge them into a single document

Features

The Clean Content API exposes the following major features;

• Discovery (analysis) and removal (scrubbing) of over 40 unique pieces of information (targets) inside Microsoft Word, Excel and PowerPoint documents
• Extraction of the text, structure and other information from Microsoft Office documents and Adobe Acrobat PDF files.
• Export of embedded objects and images from Microsoft Office documents and Adobe Acrobat PDFs
• Specialized scrubbing and modification of Fields in Microsoft Word documents
• Addition, modification and removal of Properties in Microsoft Office documents
• Recursion into embedded objects for both scrubbing and extraction
• Replacement of embedded images in Microsoft Office documents
• Assembly/Disassembly of PowerPoint presentations
• High performance
• High stability
• Multiple APIs including Java, C, C++ and .NET

File formats

TODO new FI!

Clean Content supports the following primary file formats. Many other formats (such as Windows Metafile) that are commonly associated with these primary formats are also supported. Note that hundreds of additional file formats are supported when using the optional integration with Outside In Search Export.

Adobe Acrobat (PDF)includes all versions
Support: Analyze, Extract, Export
Extensions: pdf

Adobe Forms Data Format
Support: Analyze, Extract, Export
Extensions: fdf

Compact Font Format
Support: Analyze, Scrub, Extract
Extensions: cff

Microsoft Docfileincludes formats such as Microsoft Visio, Microsoft Project, etc.
Support: Analyze (properties only), Scrub (properties only), Extract (properties only)

Microsoft Excel 2007 and above
Support: Analyze, Scrub, Extract, Export
Extensions: xlsx xlsb xlsm xltx xltm xlam xlsb xlsm xltx xltm xlam xlsb xlsm xltx xltm xlam

Microsoft Excel 2007 and above binary
Support: Analyze (limited), Scrub (properties and macros only), Extract
Extensions: xlsb

Microsoft Excel 2010 binary
Support: Analyze (limited), Scrub (properties and macros only), Extract
Extensions: xlsb

Microsoft Excel 2013/2016 binary
Support: Analyze (limited), Scrub (properties and macros only), Extract
Extensions: xlsb

Microsoft Excel 97 thru 2003
Support: Analyze, Scrub, Extract, Export
Extensions: xls

Microsoft PowerPoint 2007 and above
Support: Analyze, Scrub, Extract, Export, Assembly, Disassembly
Extensions: pptx pptm potx potm ppsx ppsm ppam pptm potx potm ppsx ppsm ppam pptm potx potm ppsx ppsm ppam

Microsoft PowerPoint 97 thru 2003
Support: Analyze, Scrub, Extract, Export, Assembly, Disassembly
Extensions: ppt pps pot ppa

Microsoft Word 2007 and above
Support: Analyze, Scrub, Extract, Export
Extensions: docx docm dotx dotm docm dotx dotm docm dotx dotm

Microsoft Word 97 thru 2003
Support: Analyze, Scrub, Extract, Export
Extensions: doc dot

Support

For pre-sales support please contact your Oracle sales representative.

Oracle customers have access to electronic support through My Oracle Support.  For information, visit http://www.oracle.com/support/contact.html or visit https://www.oracle.com/corporate/accessibility/learning-support.html#support-tab if you are hearing impaired.

Release Notes

Major changes and fixes for 8.5.0.01

• IMPORTANT: Added the boolean option BrokenPDFCorrection that tries to recover PDF if Clean Content fails to process the input PDF file.If this boolean option is set to "true" then correctectPDFs are by default placed at "scrubbedDir/CorrectedPDFs", else for extraction the corrected PDFs are placed at "extractDir/CorrectedPDFs".Considering the complex model of PDF and various errorneous struture, please note that Clean Content may not always be able to successfully correct the broken PDF.
• Multi-threading issue has been resolved in Clean Content
• Temporary files cleaned up after processing of PDF
• Clean content now successfully extract and scrub sensitive hyperlinks in 97 powerpoint files
• Clean Content now successfully completely scrub commentIds from Word 2016 files
• Issue numbers fixed in this release: 28707449, 25508010, 20357596,26269211,29036401 29338572,30426093,29028172,29488592,29338246,28534481,18482686,27773608,27117481
• CUSTOMER ALERT: This release of Clean Content requires Java 8

Major changes and fixes for 8.5.4.12

• Issue numbers fixed in this release: 27773608,27117481

Major changes and fixes for 2018.6.30

• Issue numbers fixed in this release: 27920409,25889084,25842315,26829340,25267266,27387738

Major changes and fixes for 2018.3.30

• Issue numbers fixed in this release: 25910751,27244381, 24390869, 23215828

Major changes and fixes for 2018.2.28

• Issue numbers fixed in this release: 25340818, 27117481, 25267266

Major changes and fixes for 2018.1.24

• Issue numbers fixed in this release: 25910751, 27244381

Major changes and fixes for 2015.1.4

• Assembly and disassembly now operates on PDF documents as well as PowerPoint documents.
• Issue numbers fixed in this release: 22050820, 23595853, 23749554

Major changes and fixes for 2015.1.3

• IMPORTANT: Added the boolean option TimeoutUsingThreadStop that allows timeout of requests in tight infinite loops. See the new Timeouts section in this document for details and warnings.
• CUSTOMER CODE CHANGE: Major changes to how file are identified and the FileFormat class are coming for Clean Content 2016.1. Hundreds of new formats will be identified. The change should be mostly backwards compatible but expect FileFormat to become an Enum.
• CUSTOMER ALERT: A display issue has been found when running the demo application under X-Windows. This will be resolved in Clean Content 2016.1.
• Scrubbing of Color Obfuscated Text in Microsoft Word 2007 and above when the ColorObfuscatedTextRemediation option is set to ColorObfuscatedTextRemediationOption.AdjustColor has been reworked to avoid setting all text in the document to "Auto" color.
• Added the boolean option SimulatePowerPointAnimationsDuringAssembly. This option applies to the assembly of PowerPoint 2007 and above (PPTX). When set to true, this option will cause slides that originally contained animation to be expanded into a series of slides that simulate the animations by hiding and restoring slide elements to simulate the entrance and exit of animated elements.
• PDF files embedded in Office 2007 and above will now behave correctly after being scrubbed.
  
• Issue numbers fixed in this release: 19064521, 19582411, 20663115, 20873121, 20898724, 21115231, 21305814, 21848936, 21954498, 22047140, 22293983, 22377985, 22385328, 22386086, 22390140, 22612040, 23215806 

Major changes and fixes for 2015.1.2

• Issue numbers fixed in this release: 20224163, 20639346, 20890404, 20906536, 21070673, 21084342, 21266459, 21385974, 21975599

Major changes and fixes for 2015.1.1

• Internal-only release
• Issue numbers fixed in this release: 20906434

Major changes and fixes for 2015.1

• CUSTOMER CODE CHANGE: In extracted output, the datacell element (and event) has been removed in favor of type specific elements including numbercell, textcell, datecell and durationcell.
• Extracted spreadsheet output now includes both the raw data value (in the value attribute) and the data as it appears formatted in the application (as text in the element). For example, a cell that was previously extracted like this, <datacell row="59" column="8" value="2238356.65"/>, is now extracted like this <numbercell row="59" column="8" value="2238356.65">$2,238,356.65</numbercell>.
• CUSTOMER ALERT: The next major release of Clean Content will require Java 7
• Removed Outside In Search Export integration. Customers can now use the Outside In Java and .NET APIs directly.
• Tweaked C include file extracthandlercpp.generated.h to support newer versions of GCC
• Added a scrub target called HybridExcel9597BookStream
• Added support for Microsoft Office files in Strict Open XML format
• Issue numbers fixed in this release: 8206503, 9285746, 9787417, 9787432, 13877473, 20122501, 20205999, 20402739, 20539633, 20561528

Major changes and fixes for 2013.1.6

• Issue numbers fixed in this release: 18461820, 19307501, 19504465, 19508023, 19516940, 19637681, 19773754, 19807436, 19815782

Major changes and fixes for 2013.1.5

• Fixed memory leak on Linux when using C/C++ or .NET APIs. The leak was most noticeable when using the BFSetFileOption function. For each call to this function a number of bytes equal to 2 times the length of the file name were being leaked.
• Issue numbers fixed in this release: 17866823, 18536642, 18697470, 18808601, 18808807, 18967956, 18995179, 18995293, 19074672, 19165003, 19180563

Major changes and fixes for 2013.1.4

• Added new BFStartupEx function to C API. This allows the path to CleanContent.jar, the path to the JRE, and the JRE options to all be set programatically. See the C/C++ Guidelines for details.
• Added a new version of SecureHelper.Startup to the .NET API which mirrors the new BEStartupEx function in the C API. See the .NET Guidelines for details.
• Issue numbers fixed in this release: 18614206, 18553874, 18116944

Major changes and fixes for 2013.1.3

• Issue numbers fixed in this release: 18420341, 18551449, 18298913, 18507762

Major changes and fixes for 2013.1.2

• Issue numbers fixed in this release: 18070491, 18298913

Major changes and fixes for 2013.1

• Customers will see a roughly 70% performance improvement in the analysis of XML-based Microsoft Office 2007-1013 documents, and significant increases in scrubbing and extraction performance.
• The following scrub or analysis Targets were added for this release: AppsForOffice, ExcelDataModel, UnknownXML, PDFAlternateImages, PDFDeprecatedPostscriptObjects, PDFAlternatePresentations, PDFWebCaptureInformation, PDFLegalAttestation, PDFDigitalSignatures, PDFThumbnailImages, PDFAnnotations, SensitiveContentLinks, XMPMetadataStreams, and GPSData.
• The following options were added for this release: PDFMinimumImageDimensionRequiredToProcess. See API for details.
• The option SensitiveHyperlinksRegex was renamed to SensitiveLinksRegex and now applies to both the SensitiveHyperlinks and SensitiveContentLinks targets.
• Extraction and scrubbing of the Open Office XML format (Microsoft Office) now includes processing of all choices in Alternate Content elements.
• Added support for XMP embedded in PDF documents.
• Added support for identification of XMP, RDF, WAV and AIFF file formats.
• CUSTOMER ALERT: The last public update of Java 6 was Feb 2013. An upcoming version of Clean Content will drop support for Java 6 and require Java 7. Please contact your Oracle sales representative if this change will create an issue for your product.
• Issue numbers fixed in this release: 9463630, 13377241, 13425435, 13475829, 13582954, 13956841, 14239187, 14509582, 14763715, 15893487, 16403744, 16561941, 17050811, 17337929, 17471338, 17633976

Major changes and fixes for 2012.1.2

• Added clarification on custom property modification to this document.
• Issue numbers fixed in this release: 16895155, 16914757

Major changes and fixes for 2012.1.1

• Verified against Microsoft Office 2013. Verification means that Office 2013 documents are correctly extracted and that scrubbed documents of any supported Microsoft Office version will open correctly in Office 2013. Support for Office 2013 schema changes, additional scrub targets, etc. will be available in a later version of Clean Content.
• Clarified the validity of output in exception and error cases. Details are here.
• Scrubbing of Unknown Fields in Word 2007-2013 files now works as documented
• The demo application now uses net.java.awt.Desktop class to open browser windows
• Issue numbers fixed in this release: 11677587, 16284559, 16297194, 16396281, 16515824, 16516432, 16601120, 16673293

Major changes and fixes for 2012.1

• Scrubbing of PDF documents as well as many PDF related targets have been added including ClippedText, PDFActions (and it's sub-targets like PDFJavaScriptActions), and PDFPrivateApplicationData.
• Integration with Oracle's Outside In Search Export has been added. Customers who choose to purchase that product will be able to extract text, metadata and structure from over 500 additional file formats through the Clean Content API. Details are here.
• XML processing is now more forgiving of XML that does not follow its XML Schema. An InvalidXML target and logged warnings have been added to alert developers to invalid XML elements in case they require strict schema conformance
• PDF seach hit highlighting support has been removed from the demo application. See related Customer Alert under 2011.2 changes
• Clean Content now requires Java 6 or higher
• The XML schema used for extraction has been extended with many additional elements and attributes in the message, archive and database areas to support the Outside In Search Export integration
• Fixes were made to how XML output is produced allowing use of Saxon and other third-party XSLT processors that plug-in to the Java JAXP infrastructure.
• Added UninitializedDocfileData target
• CUSTOMER ALERT: In an upcoming revision the Clean Content options that are enumerations will become real Java Enums in the Java API. This may end up being transparent to the customer's code but please expect this change.
• Issue numbers fixed in this release: 13636289, 13940798, 13987464, 14038681, 14051875, 14064291, 14120406, 14188819, 14193959, 14198083, 14251214, 14304898, 14527636, 14539010

Major changes and fixes for 2011.2.7:

• Tweaked the XML structure of scrubbed Word documents (.docx) to work around a bug in Apple's iPhone/iPad email attachment preview feature
• Normalized the definition and implementation of the Weak Protections target
• Added more descriptive reporting of invalid XML elements
• Resolved significant issue in PowerPoint .pptx disassembly
• Issue numbers fixed in this release: 13068853, 13796206, 13972531, 14019099, 14051609, 14078741, 14128837

Major changes and fixes for 2011.2.6:

• Scrubbing of ImplicitRef, Formula and Unknown fields now supported in Word
• Added Weak Protection support in Excel 2007/2010. See the Weak Protections target description for more information.
• Added Custom XML support for Word 97-2003. See the Custom XML target description for more information.
• Tracked Changes in Word 2007/2010 that indicate moves are now supported for scrubbing
• Issue numbers fixed in this release: 13041497, 13257351, 13388342, 13425420, 13576875, 13641062, 13642206, 13682781, 13682823, 13465121, 13776088

Major changes and fixes for 2011.2.5:

• CUSTOMER ALERT: The next major version of Clean Content will require Java 6 or above, Java 5 will no longer be supported
• Fields without structured data (PRIVATE, RD, TA, TC and XE) will be scrubbed correctly in Microsoft Word 97, 2000, XP & 2003 documents
• Unknown fields (that is fields that don't have a structured data type and don't match any field name) will be scrubbed in Microsoft Word 97, 2000, XP & 2003 documents based on the new SecureOptions.Fields.Unknown field type
• Added missing GetEnumResult method to BFSecureResponse class in the C/C++ API
• Added missing GetResult(EnumOption) method to the SecureResponse class in the .NET API
• Fixed declaration of .NET enumeration option values from 'static' to 'const' so they can be used in a switch statement
• Updated C/C++ and .NET sample code in the Response section of this document to use the new ProcessingStatus option
• In previous versions the SecureResponse class had references to other large objects trees causing the collection and later processing of a large number of these objects to be very memory intensive. The code has be refactored to produce a very light SecureResponse object allowing collection of a large number of responses without huge memory overhead.
• In version 2011.2.4 the UNKNOWN FileFormat was accidentally moved to be a sibling of the ALL FileFormat instead of its child, this has been corrected
• Issue numbers fixed in this release: 13345226, 13332027, 13325655, 13253127, 13103312, 13087267, 13082906, 13042424, 13361562, 13345226

Major changes and fixes for 2011.2.4:

• CUSTOMER ALERT: The behavior of property modification has been altered so that the actions None and Scrub when specified for individual properties will propogate down into embedded documents. The Replace and AddOrReplace actions will behave as before in that they will not affect properties in embedded documents.
• Changed processing of Word and PowerPoint XML element types to allow certain unusual documents to be processed correctly.
• Changed processing of Microsoft Office 2007-2010 documents to recover from malformed VML
• Fixed the C function BFHelperGetDateString
• Fixed infinite loop cases in certain malformed Microsoft Office documents

Major changes and fixes for 2011.2.3:

• Changed methodology for scrubbing Comments in Word 97-2003 documents. The earlier method could, in exceptional circumstances, produce documents unreadable by specific versions of Word. The new method fixes this issue while still passing Word 2010's Office File Validation test.
• Fixed small issues in property categorization and scrubbing

Major changes and fixes for 2011.2.2:

• Added array allocation checks in generated code to avoid OutOfMemoryError being thrown in cases where malformed documents drive the code with unexpected data
• Removed all use of the String object's intern method to avoid filling PermGen space and causing OutOfMemoryError

Major changes and fixes for 2011.2.1:

• Small, document specific fixes to the PowerPoint 97-2003, PowerPoint 2007/2010, Word 2007/2010 and PDF transforms
• BMP is now a valid image replacement format in Office 2007/2010 documents

Major changes and fixes for 2011.2:

• CUSTOMER ALERT: Two fixes have been made to the C API (and therefore the C++ and .NET APIs built on top) to solve issues where certain documents processed through these API's could cause untrapped segmentation faults in the Java Virtual Machine. All customers using the C, C++ or .NET APIs on Clean Content 2010.1 or above should upgrade ASAP to 2011.2. A patch to Clean Content 2011.1 for this issue is also available on My Oracle Support at Doc ID 1307556.1. This is not an issue in the Java API.
• CUSTOMER ALERT: In Acrobat X, Adobe Systems has chosen to remove the "hit highlighting" feature Clean Content can leverage to highlight search terms in PDF documents. This feature was available but disabled in Acrobat 9. These changes call into question the usefulness of the GenerateAcrobatHighlightPositions option in Clean Content.
• Added OfficeXMLPartValidation option and associated analysis targets: OfficeXMLUnanalyzedParts, OfficeXMLUnexpectedParts, OfficeXMLAlternateContentParts and OfficeXMLRogueParts. Details may be found in the Microsoft Office Open XML Support Technical Note.
• CUSTOMER CODE CHANGE: The little used getCharBuffer method in the ElementHandler interface has been modified. It now takes a single parameter specifying the minimum allowable size of the returned character buffer.
• In Microsoft Word and PowerPoint 2007/2010 formats the line break element is now being handled correctly, avoiding cases where text separated by line breaks world run together in the extracted output.
• Several enhancements have been made to PDF extraction in the areas of vertical text and space inference.
• Added ability to scrub of PRIVATE and BIDIOUTLINE fields in all supported versions of Microsoft Word
• Fixed issue where images and embeddings with the hidden character property were not being scrubbed when the HiddenText target is set to SCRUB in Microsoft Word 2007/2010.

Major changes and fixes for 2011.1:

• Added support for encrypted Microsoft Office and PDF documents and associated PasswordList and DecryptionStatus options. Details may be found in the Encrypted Document Support Technical Note.
• Added ToTextEncoding option allowing either UTF-16 or UTF-8 encoding when extracting text with OutputType set to ToText. In addition, some small improvements were made to the extracted text itself.
• Added ProcessingProblem response to make response handling easier to write. The WasProcessed, WasIdentified, WasSupported, WasTimeout and WasException responses are still available but deprecated. The Response section has been updated to reflect these changes.
• Added support for field modification in Office 2007/2010 documents.
• Added ability to change the starting page number of a Microsoft Word 2007/2010 document through the new ChangeStartingPageNumber (boolean) and StartingPageNumber (integer) options.
• Added analysis and scrubbing of document variables in Microsoft Word 2007/2010 under the DocumentVariables target.
• Moved analysis and scrubbing of document variables in Microsoft Word 97-2003 from the MacrosAndCode target to the DocumentVariables target.
• CUSTOMER ALERT: Microsoft Office 2010 added a security feature called Office File Validation. This feature (enabled by default) adds numerous document integrity tests when opening pre-Office 2007 binary formats for Word, Excel and PowerPoint. This is an effort by Microsoft to prevent specially crafted pre-Office 2007 documents from compromising system security. Unfortunately this feature will also flag some documents that have been scrubbing by earlier versions of Clean Content causing the Office 2010 application to enter Protected View and warn the user. Note that such documents open correctly and with no warning in all previous versions of Microsoft Office. The Clean Content scrubbing process has been modified so that pre-Office 2007 documents scrubbed by this version will not receive this warning in Office 2010.
• CUSTOMER CODE CHANGE: Customers are able to apply an XSLT transform to the extracted XML output using the TransformResult (boolean) and ResultTransform (file) options. Up until this version the XSLT provided was required to treat XML attributes in the extracted output as "qualified" even through the untransformed XML output shows them as "unqualified". This issue has been fixed and XSLT passed to the ResultTransform option should be modified to treat XML attributes as "unqualified".

Major changes and fixes for 2010.2:

• Full support for Office 2010. Details may be found in the Microsoft Office Open XML Support Technical Note.
• Changed the way the Comments target is scrubbed out of Microsoft Word 97-2003 documents to avoid a warning when opening such scrubbed documents in Microsoft Word 2010.

Major changes and fixes for 2010.1:

• Added C/C++ API support for 64 bit versions of Windows and Linux on the x86-64 architecture.
• Added .NET API support for 64 bit versions of Windows on the x86-64 architecture.
• Numerous PDF extraction and analysis features have been added with this release. These include the detection of JavaScript (Macros and Code) and Incremental Updates (Fast Save Data) and the extraction of Document Outlines, Thumbnail Images, Article Thread Information, Interactive Forms, and XFA Forms. Enhancements have also been made for AES Encrypted documents, PDF Portfolio support, and Malformed PDF document parsing. Additionally, numerous heuristic enhancements have been made to the line detection and character mapping algorithms that further improve extraction from poorly crafted PDF documents. Details may be found in the PDF Extraction and Analysis Support Technical Note.
• Processing of PDF documents with AES 256 bit encryption requires the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files. See the PDF Extraction and Analysis Support Technical Note for details.
• CUSTOMER CODE CHANGE: The C interface (and by extension the C++ and .NET interfaces built on top) has been largely rewritten to be thinner, faster and more portable. As part of this rewrite several small changes have been made to the C API itself, these include...
  • Access to actions and new values for scrubbing and changing Properties and Word Fields has been changed. The former API defined a structure like SecureOptions_Properties_Author which contained option id's for the action and newValue on the Author Property. The new API simply defines static constants for the two option ids. So all a developer needs to do in their code is change things like SecureOptions_Properties_Author.newValue to SecureOptions_Properties_Author_newValue.
  • Arrays listing the options of various types used to be directly available through arrays like AllScrubOptions or AllStringOptions. These arrays and the count of their elements are now available only through function calls like BFGetAllScrubOptions and BFGetAllStringOptions.
  • The two changes above (and other changes not visible in the API) were undertaken to allow multiple source files to include secureapi.h without the cumbersome #define BFDONTDEFINE method used in earlier versions.
• The C interface library is now buildable by the OEM customer in order to support the C/C++ API on any reasonable unix-like platform with a Java 1.5 JDK and a recent GNU toolchain. This also allows the customers using Linux to relink libCleanContentAPI.so with standard library versions of their choice.
• This release ships static (libCleanContentAPI.a) versions of the C interface library for Linux as well as the shared (libCleanContentAPI.so) versions.
• As of Sun JVM version 1.5 the requirement on Linux to add the Java library paths (like /usr/java/j2re1.4.2_05/lib/i386 and /usr/java/j2re1.4.2_05/lib/i386/client) to an applications library search path (either using -rpath at link time or LD_LIBRARY_PATH at run time) has been removed.
• The granularity of the RequestTimeout option has been improved to roughly half a second. Previous versions had a timeout resolution of about 10 seconds.

Major changes and fixes for 2009.1.1:

• Fixed problem with certain PDF documents were never timing out
• Fixed .NET API issue where dates out of a certain range cause an untrapped exception
• Fixed issue where the Content Properties target was not being set for Microsoft Office 2007 files containing content properties
• Fixed issue where the Headers and Footers target was not being set for Microsoft Word 2007 documents containing headers or footers
• Fixed issue where Open Office 3.1 would not open Microsoft Office 2007 documents scrubbed by Clean Content
• Updated the Targets section of this guide to more clearly list the formats that can contain each target
• CUSTOMER CODE CHANGE: The ExportMaximumReplacementSize option retrieved during the startEmbeddedContent and processEmbeddedContent methods may now be 0 (zero) indicating that any size replacement is allowable.
• Fixed issues with C, C++ and .NET documentation

Major changes and fixes for 2009.1:

• Full support for extraction, analysis and scrubbing of Office 2007 Word (.docx, .docm, .dotx, .dotm), Excel (.xlsx, .xlsm, .xltx, .xltm, .xlam), PowerPoint (.pptx, .pptm, .potx, .potm, .ppsx, .ppsm, .ppam) documents.
• Full support for Microsoft PowerPoint 2007 assembly and disassembly.
• Full support for extraction and extensive analysis support of Office 2007 Excel Binary (.xlsb) documents. Property and Macro scrubbing is also included for this format.
• CUSTOMER CODE CHANGE: Added ScrubbedFormat option and new code examples in the Response section. It is critical that existing customers that would like to correctly scrub Office 2007 formats read this new information and update their code accordingly. In short, Office 2007 files may need to have their extensions changed when scrubbed. For example, when scrubbing macros from a .docm file it must be changed to a .docx file or it will not open in Word. The ScrubbedFormat option allows the developer to understand when this is required and change the name accordingly. See the Response section for details.
• Added the CustomXML and StructuredDocumentTags scrub targets.
• CUSTOMER CODE CHANGE: The hyperlinkbegin and hyperlinkend elements in the extraction schema have been changed as follows. The url and sensitive attributes have been removed in favor of always using a contentref or linkedcontent child. This normalizes how hyperlinking and external references are accomplished in the schema.
• CUSTOMER CODE CHANGE: The signatures of the StartEmbeddedContent and ProcessEmbeddedContent methods in BFBaseElementHandler have changed. In both cases the exportOptions parameter is now part of the BFEmbeddedContentElement structure instead of being an additional parameter. In the case of C++ users extending BFBaseElementHandler the exportOptions handle will now need to be used with the C option functions or used to create a BFOptionSet. See the Embedding Replacement sample code for an example of this.
• Java 1.4 is no longer supported. Clean Content now supports only Java 1.5 and above.
• The demo application now includes an improved single file processing UI and allows setting of nearly all options available in the API.
• In a change in behavior, SecureRequest will now delete the newly created scrubbed document when processing fails. This avoids leaving partially constructed, corrupt documents floating around. This is only the case where the ScrubbedDocument option is set using a File object in Java, a path in C/C++ or a FileInfo object in .NET.
• FEATURE CHANGE: The ScrubInPlace option has been removed. Unlike the binary Microsoft Office file formats, the XML Microsoft Office file formats need to be completely rewritten in order to be correctly scrubbed. This requirement makes ScrubInPlace impossible or at least highly inefficient to implement. This, along with other more technical issues, have led us to remove this feature from the API.

Major changes and fixes for 2008.1.5:

• Fixed PowerPoint 2000 assembly issue

Major changes and fixes for 2008.1.4:

• Added image extraction from Adobe Acrobat PDFs
• Various fixes and enhancements for Adobe Acrobat PDF (see Technical Notes)
• CUSTOMER CODE FLASH: The 2008.2 release will operate only on Java 1.5 and above. Java 1.4 will no longer be supported.

Major changes and fixes for 2008.1.3:

• Various fixes for PowerPoint assembly/disassembly

Major changes and fixes for 2008.1.2:

• Added property extraction support for Office 2007 XML (MSOOX) files
• This document now meets Oracle Global HTML Accessibility Guidelines
• Miscellaneous fixes for a number of file formats including several for PDF
• CUSTOMER CODE CHANGE: This release includes an update to SimpleChannel and its C, C++ derivatives to include a truncate method. Any developer providing or consuming data through a SimpleChannel (or its C, C++ derivatives) will need to update their implementation and recompile to work with the 2008.1.2 release. The equivalent functionality in C# is provided through the .NET Stream class and C# developers should need no changes to their code.

Major changes and fixes for 2008.1.1 (Oracle internal only):

• Added property modification support for Office 2007 XML (MSOOX) files. Currently only supports addition/deletion/modification of custom properties
• Added PowerPoint slide fingerprinting support (see technical note at end of this document)
• Added extended support for identifying Office documents with incorrect CLSIDs
• Demo application now supports additional PDF, PowerPoint and Excel options
• CUSTOMER CODE FLASH: The 2008.2 release will update SimpleChannel and its C, C++ and C# derivatives to include a truncate method. Any developer providing or consuming data through a SimpleChannel (or its C, C++ or C# derivatives) will need to update their implementation and recompile to work with the 2008.2 release.

Major changes and fixes for 2008.1:

• Added PDF extraction with hit highlighting support
• Added logging support in Java
• Published XML schema for extracted XML and ElementHandler events
• CUSTOMER CODE CHANGE: Addition of a published XML Schema has lead us to do some cleanup on the XML output (and consequently the ElementHandler events). These changes should not affect most customers but anyone post-processing the XML or coding for specific series of ElementHandler events should review the new XML Schema and modify their code, xslt, etc. accordingly. In the future we may refine and tweak the XML Schema but no drastic changes are envisioned in the short to medium term.
• Fast Save data in Word can now be exported for processing
• Added PDF technical notes

Major changes and fixes for 2007.2.1:

• Added PowerPoint Assembly/Disassembly

Major changes and fixes for 2007.2:

• Added ExtremeCells, ExtremeIndenting, ExtremeObjects and OverlappedObjects targets
• Added regex-based Header/Footer removal and modification
• Added .NET API
• Property modification and scrubbing will now work on any Microsoft Docfile format document not just the formats (Word, Excel and PowerPoint) where more comprehensive scrubbing occurs. This mean that property modification can now occur on formats like Microsoft Visio and others that use the DocFile container format.
• Reworked documentation and extended sample code

Major changes and fixes for 2007.1:

• CUSTOMER CODE CHANGE: Property and Field scrubbing/modification API has been reworked to avoid specialized methods/functions and decrease the API footprint. If you use Property or Field scrubbing/modification you will need to change your code to conform to the new API. This is true of both the C/C++ and Java APIs.
• JAVA CUSTOMER CODE CHANGE: All trapped and many untrapped exceptions are now being caught and reported through a TransformException (a subclass of IOException). As a result, developers now need only catch IOException on the request's execute method. Existing code will work if it followed the earlier guidelines but many of the catch blocks will never be reached.
• Added Size Obfuscated Text scrub target
• Added Color Obfuscated Text scrub target
• Added table in table support in Microsoft Word
• Removed Scenario Comments scrub target in favor of new Scenarios scrub target
• The WasIdentified, WasSupported, WasException and WasTimeout boolean results have been added to improve error reporting.
• Demo application now has GUI support for all Clean Content options.
• Demo application now provides more statistics about groups of documents being processed.
• Demo application now provides better reporting of exceptions and other errors.
• A timeout mechanism has been added to interrupt very long or looping requests. See the RequestTimeout option for details.
• The C/C++ API now has a setting that allows easier debugging of the Java startup process that occurs during BFStartup function or BFSecureRequest.Startup method.
• In Microsoft Word the bit that indicates a document is Fast Save is now cleared when Fast Save Data is scrubbed.
• Rebranded to Oracle
• SDKs now ship with the Sun's 1.4.2_13 JRE which resolved some vulnerabilities in earlier JREs.
  See http://www.us-cert.gov/cas/techalerts/TA07-022A.html

Major changes and fixes for 2006.2:

• Added ability to remove, modify and add document properties in Microsoft Office documents.
• Added ability to remove hyperlinks based regular expression matching.
• Rebranded to Stellent

Major changes and fixes for 2006.1:

• CUSTOMER BUILD CHANGE: Rebranded to Clean Content SDK. JAR, DLL and SO file names have changed. Customers should inspect and update their build process.
• Performance on all formats has roughly doubled.
• Added the ability to extract the text and structure from documents as they are processed.
• Added the ability to export embedded objects for further processing or display.
• Added the ability to recur into embedded objects for scrubbing, analysis, extraction and/or export purposes.
• Added the ability to replace images and embedded objects.
• Added option to extract only properties in order to increase performance for those customers needing only document properties.
• Extracted XML now includes better delineation and referencing of items like headers, footers, footnotes, etc.
• Several new scrub targets have been added including: Alternative Text, DocumentVariables, Fields (including individual field type removal and text replacement) and Meeting Minutes.
• Added the ability to 'unhide' hidden cells in Excel.
• Added the ability to define a global default scrub behavior and changed the default value of all scrub target options to use this default.
• The Secure sample application can now extract to XML as well as produce an HTML report.
• JAVA CUSTOMER CODE CHANGE: In order to avoid difficulty in writing Java code all options available in Secure are now available in the SecureOptions class. The ExtractOptions and SharedOptions classes still exist but Stellent recommends using SecureOptions class instead.
• C/C++ CUSTOMER CODE CHANGE: In order to support both Secure's and Extract's options in the C/C++ interface the option names which were formerly things like SourceDocument, AuthorHistory and OutputType have now been namspaced by prefixing with 'SecureOptions_'. So the above are now called SecureOptions_SourceDocument, SecureOptions_AuthorHistory, SecureOptions_OutputType. In addition, a C++ cover class on these options has also been added so C++ customers can use BFSecureOptions::SourceDocument, BFSecureOptions::AuthorHistory and BFSecureOptions::OutputType. Customer's C/C++ code will need to be modified to reflect this change.

Major changes and fixes for 2005.2:

• Added C/C++ API.
• Added several new ways to provide the SourceDocment and ResultDocument options including as a block of memory, as a FileChannel and as a SimpleChannel (defined in this SDK) which allows arbitrary IO redirection.
• Many small fixes to analysis, scrubbing and reporting behavior.

Major changes and fixes for 2005.1:

• The information content and look of the reports has been improved substantially.
• Many small fixes to analysis, scrubbing and reporting behavior.
• Many small fixes to the BitformSecureSDK application.
• Performance on Microsoft Word documents has been almost doubled.
• The TransformReport option now has its documented effect.

The SDK's directory structure provides easy access to all the components, samples, documentation and other files needed to integrate the Clean Content SDK into your application.

CleanContentSDK

The root directory of the SDK

CleanContentSDKDemoWin32.exe
CleanContentSDKDemoWin64.exe
CleanContentSDKDemoLinux32.sh
CleanContentSDKDemoLinux64.sh

OS specific launchers for the Clean Content SDK demo application. This Java application is designed to demonstrate the full potential of the Clean Content API and allow developers to explore the analysis, scrubbing, extraction and export behavior of this SDK in a full featured GUI environment.

CleanContentSDKDemoGeneric.sh

A generic launcher for the Clean Content SDK demo application on Unix style operating systems. It will use the JAVA_HOME environment variable followed by locate bin/java to find an appropriate Java Runtime Environment to use. Requires the bash shell.

index.html

The Clean Content Developer's Guide

app

Directory containing components and documentation for the CleanContentSDKDemo application

c

Directory containing libraries, include files, samples and other files required to use the Clean Content C/C++ API.

include

Directory containing include files required to use the C/C++ API. Most importantly it contains secureapi.h which is the only file your code needs to include.

lib

Directory containing native code libraries needed to use the C/C++ API plus the test and sample app executables.

Windows

Directory containing DLLs and LIBs needed to use the C/C++ API on Microsoft Windows plus the test and sample application EXEs . This directory will include one or more sub-directories that correspond to the processor architecture for which Clean Content is available. Currently x86 and x64 architectures are available.

Linux

Directory containing library archives and shared objects needed to use the C/C++ API on Linux plus the test and sample applications. This directory will include one or more sub-directories that correspond to the processor architecture for which Clean Content is available. Currently x86 and x64 architectures are available.

api

Directory containing the full source code to the C/C++ API.

apitest

Directory containing a cross-platform, pure C, test application designed to exercise the C API.

sanitytest

Directory containing a cross-platform, C++, test application that uses the files in CleanContentSDK/samplefiles/targets to verify that basic document analysis is working correctly.

csample

Directory containing a cross-platform, pure C, sample application.

cppsample

Directory containing a cross-platform, C++, sample application.

dumptext

Directory containing a cross-platform, C++ sample application that shows how to retrieve the text out of a document using an element handler.

docs

Directory containing documentation for the SDK

cdoc

Directory containing C/C++ API documentation

javadoc

Directory containing Java API documentation

dotnetdoc

Directory containing .NET API documentation

technotes

Directory containing technical notes

java

Directory containing components, samples and other files required to use the Clean Content Java API

lib

Directory containing CleanContent.jar that should be shipped with your application. See Install and Coding Guidelines.

sample

Directory containing Java API sample applications. Sample directories include batch files and shell scripts to build and run each application.

AnalyzeDirectorySample

Directory containing a command line sample application that analyzes all the documents in a given directory.

dotnet

Directory containing components, samples and other files required to use the Clean Content .NET API

lib

Directory containing CleanContentNET.dll that should be shipped with your application plus the test app executables. Just shipping this dll is not enough, please see the .NET Install and Coding Guidelines.

apitest

Directory containing a .NET test application designed to exercise the basics of the .NET API.

sanitytest

Directory containing a .NET test application that uses the files in CleanContentSDK/samplefiles/targets to verify that basic document analysis is working correctly.

jres

Directory containing four Java Runtime Environments (Win32, Win64, Linux32 and Linux64) needed to run the CleanContentSDKDemo application and the Java sample applications on the supported operating systems. These JREs may also be distributed with the developer's application. Oracle chooses to ship these JREs along with its SDK instead of requiring developers to "install Java" before using the demo app.

samplefiles

Directory containing documents that can be used to test Clean Content's behavior and your application. See the readme.txt file in this directory for detailed information.

samples

Directory containing the original set of Clean Content sample documents.

targets

Directory containing documents that exercise all the targets Clean Content can identify for the various supported file formats. Oracle uses these documents internally as one part of Clean Content's automated QA process.

exception

Directory containing a series of Microsoft Word documents built specifically to trigger Clean Content to generate certain exceptions, including null pointer exception and out of memory exception. The document names indicate the Java exception they generate. These documents were developed to help customers build QA processes that include exception testing. It should be noted that these documents do not exercise flaws in Clean Content rather certain bytes have been modified and are tested by Clean Content's Microsoft Word transform which in turn triggers these specific exceptions on purpose.

Java

The core of the SDK is a set of Java classes that perform the actual analysis, scrubbing, extraction and export. These classes are delivered as CleanContent.jar. If your application is written in Java or has direct access to Java classes (a web site using Java Server Pages for example) the jar can be used directly through the Clean Content Java API.

What Java runtime to use?

If you are already using Java then you probably have an existing Java Runtime Environment (JRE) that you use and/or require. If you plan on using Clean Content's C/C++ or .NET interfaces then this might be the first time you've been exposed to the JRE choices available to you. Clean Content requires a Java Standard Edition 6 compatible JRE and ships with four version of Oracle's JRE 6 (in the jres subdirectory of the SDK).

C/C++

Clean Content's C/C++ API is built on top of the Clean Content Java API allowing your C or C++ application to run Clean Content "in process" for maximum performance while getting all the stability and safety features of Java. This is accomplished by providing a native code library (CleanContentAPI.dll or libCleanContentAPI.so for example) that does all the work of loading the Java VM into your process and interfacing with Clean Content's Java core. This is done without requiring that you or your customer "install Java" on the target system. The Java components (CleanContent.jar and the JRE subdirectory) may be local to your application with no impact on the rest of the system. In this instance, Java is simply a number of extra DLLs or SOs that are being dynamically loaded into your process.

This architecture was selected in order to meet the requirement of high performance in-process parsing while still protecting your process from the problems often caused by the limitless variations of complex, malformed, hacked and truncated documents. The C and C++ APIs provide the interfaces that meet your application's needs while the Java VM provides a stable and well tested platform that protects your application from wild pointers and buffer overflows that plague parsers written in native code. Running these documents inside a VM protects your applications while avoiding the complexity and performance problems of "out of process" solutions.

.NET

Clean Content's .NET API is built on top of the C API using .NET's interop services. As with the C/C++ API the .NET API runs Clean Content "in process" (the Common Language Runtime and the Java Virtual Machine can coexist in the same process) for maximum performance and ease of integration all without requiring you or your customer to "install Java". Please review the C/C++ section above for further details.

Initialization

The Java API requires no per-process or per-thread initialization and your code may immediately begin creating SecureRequest objects. The C/C++ and .NET APIs however require per-process and per-thread initialization in order to interface correctly with the underlying Java VM. In these environments the following guidelines must be followed for initialization...

• BFStartup (in C/C++) or SecureHelper.Startup (in .NET) must be called before any other calls to this API.
• BFShutdown or SecureHelper.Shutdown must be called after all other calls to the API and must be called in the same thread that called BFStartup/SecureHelper.Startup
• Each new thread that wants to process documents must call BFAttachThread or SecureHelper.AttachThread before any other calls to this API. This is not required for the thread that calls BFStartup/SecureHelper.Startup.
• A thread that called BFAttachThread/SecureHelper.AttachThread must call BFDetachThread or SecureHelper.DetachThread after its last call to this API. This is not required for the thread that calls BFStartup/SecureHelper.Startup.
• Each thread should create its own SecureRequest and use it only in that thread. This is not a hard and fast rule but it is simplest and safest. For example, an application that has thousands of threads might choose instead to create a small pool of SecureRequest objects to service those threads as long as each SecureRequest is used only in one thread at a time and the thread follows the rules above.
• Please contact Oracle pre or post sales support if you need additional clarification on the API's threading behavior.

Basic Use

A developer's primary interaction in this API is with a SecureRequest object or handle in the case of the C API (from now on this document will use object/class/method semantics, C API users should be aware that a SecureRequest handle is equivalent to a SecureRequest object). This class contains mostly methods that allow the developer to get and set a collection of typesafe options found in the SecureOptions object. In addition, SecureRequest contains methods for executing the request and for getting the results. This follows Clean Content's basic design philosophy for long term APIs which favors extensible, typesafe, self-describing options over more concrete methods attached directly to the SecureRequest.

The basic execution flow of a simple application that needs to process multiple documents is as follows.

1. Call Startup (C, C++ and .NET only)
2. Create a new SecureRequest object
3. Use that object's various setOption methods with the options in SecureOptions to define what parts of the each document should be analyzed or scrubbed and/or to set extraction, export and other processing parameters.
4. Set the SourceDocument option to define the next document to be processed. If all documents are done go to step 8.
5. Call the SecureRequest's execute method.
6. Use the SecureRequest's getResponse method to determine the outcome of the operation.
7. Return to step #4
8. Call Shutdown (C, C++ and .NET only)

Below are code samples for complete Java, C++ and C# programs that show how to analyze a single document for targets. Notice that the .NET API requires explicit Close methods for SecureRequest and SecureResponse objects. For more details see the .NET Install and Coding Guidelines.

Request

The SecureRequest object represents a reusable request to perform actions on a document. A single SecureRequest is created and reused to process as many documents as necessary within a single thread (see Threading for details).

A rhodium-catalyzed tandem enantioselective C–H silylation/alkene hydrosilylation of dihydrosilanes, which enables the streamlined construction of a wide range of silicon-stereogenic silanes, is successfully developed. This process involves a SiH₂-steered highly enantioselective C–H silylation to furnish the corresponding desymmetric monohydrosilanes, which are subsequently trapped with alkenes in a stereospecific fashion to build functionally diverse asymmetrically tetrasubstituted silanes. This general strategy combines readily available dihydrosilanes and alkenes to construct various enantioenriched silicon-stereogenic silanes, including 9-silafluorenes, Si-bridged ladder compounds, and benzosilolometallocenes, in a single step with good to excellent yields and enantioselectivities.