What encoding is used by javax.xml.transform.Transformer?

Question

Please can you answer a couple of questions based on the code below (excludes the try/catch blocks), which transforms input XML and XSL files into an output XSL-FO file:

File xslFile = new File("inXslFile.xsl");
File xmlFile = new File("sourceXmlFile.xml");
TransformerFactory tFactory = TransformerFactory.newInstance();
Transformer transformer = tFactory.newTransformer(new StreamSource(xslFile));
FileOutputStream fos = new FileOutputStream(new File("outFoFile.fo");
transformer.transform(new StreamSource(xmlFile), new StreamResult(fos));

inXslFile is encoded using UTF-8 - however there are no tags in file which states this. sourceXmlFile is UTF-8 encoded and there may be a metatag at start of file indicating this.

am currently using Java 6 with intention of upgrading in the future.

1. What encoding is used when reading the xslFile?
2. What encoding is used when reading the xmlFile?
3. What encoding will be applied to the FO outfile?
4. How can I obtain the info (properties) for 1 - 3? Is there a method call?
5. How can the properties in 4 be altered - using configuration and dynamically?
6. if known - Where is there info (web site) on this that I can read - I have looked without much success.

Answer 1

XML is defined to be UTF-8 or UTF-16 with BOM in the absence of an explicit encoding in the prologue: http://www.w3.org/TR/REC-xml/#charencoding

If you're reading/writing an XML file using a standard parser/serializer, you should always use FileInputStream / FileOutputStream and rely on the library doing the right thing.

If you're getting input from an HTTP server, you have to pay attention to the encoding specified in the Content-Type header. In this case you should use an InputStreamReader to wrap the socket with the correct encoding. If you don't have an explicit encoding specification, just pass the socket's stream to the parser.

Similarly, if you're writing XML on a server, make sure that you're not accidentally creating an incorrect Content-Type. You should explicitly set application/xml without encoding, and rely on the serializer doing the right thing.

You should never write XML to a string and then write that string to a file. Nor should you ever write XML to a FileWriter (you shouldn't use FileWriter or FileReader for anything, not just XML, because they use the platform default encoding).

Unfortunately, a lot of people do that (or worse, don't use a serializer), so you may be getting a file that looks like XML but isn't.

Answer 2

If you know that the encoding is always going to be UTF-8, you could use the InputStreamReader class which allow you to specify an encoding, and pass those instances to StreamSource. Otherwise, I'd expect the system to use the encoding specified in the XML file, and if it's not specified then either use the system default encoding or make a best guess (that should not be depended on to be correct). The best solution would be to ensure that the XML files have correct encoding metadata.

The output encoding can be controlled using the <xsl:output> tag. From Java, this can be done using the setOutputProperty() method.

Answer 3

What encoding is used when reading the xslFile?

What encoding is used when reading the xmlFile?

How can I obtain the info (properties) for 1 - 3? Is there a method call?

There is no clear way to retrieve this information from the Transformer API. Source encoding is only relevant to a parser, and then only while parsing a particular document or fragment from a stream of bytes.

I imagine you ask this because you've run into issues with encoding and are looking for a way to debug it. A common issue is that XML documents are marked as being in encoding ABC, but are actually encoded through XYZ, so... verify that your XML documents are stored in the encoding they claim.

If you're forced to a guess, it's probably encoded in Windows-1252 / Cp1252 and the XML document will either claim to be UTF-8 or won't have a declaration.

There are two ways around this:

1. convert the file with iconv or a text editor like Notepad++, or

2. add encoding="windows-1252" to the <?xml instruction.

If you're going to deploy to non-Windows machines, keep in mind that a JVM is not required to support windows-1252 / cp1252 (though they support its close relativeISO-8859-1):

Standard charsets

Every implementation of the Java platform is required to support the following standard charsets. Consult the release documentation for your implementation to see if any other charsets are supported. The behavior of such optional charsets may differ between implementations.

• US-ASCII Seven-bit ASCII, a.k.a. ISO646-US, a.k.a. the Basic Latin block of the Unicode character set
• ISO-8859-1 ISO Latin Alphabet No. 1, a.k.a. ISO-LATIN-1
• UTF-8 Eight-bit UCS Transformation Format
• UTF-16BE Sixteen-bit UCS Transformation Format, big-endian byte order
• UTF-16LE Sixteen-bit UCS Transformation Format, little-endian byte order
• UTF-16 Sixteen-bit UCS Transformation Format, byte order identified by an optional byte-order mark

Answer 4

To answer part of the question following code works for me. This can take input encoding and convert the data into output encoding.

ByteArrayInputStream inStreamXMLElement = new ByteArrayInputStream(strXMLElement.getBytes(input_encoding));
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder(); 
Document docRepeat = db.parse(new InputSource(new InputStreamReader(inStreamXMLElement, input_encoding)));
Node elementNode = docRepeat.getElementsByTagName(strRepeat).item(0);

TransformerFactory tFactory = null;
Transformer transformer = null;
DOMSource domSourceRepeat = new DOMSource(elementNode);
tFactory = TransformerFactory.newInstance();
transformer = tFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
transformer.setOutputProperty(OutputKeys.ENCODING, output_encoding);

ByteArrayOutputStream bos = new ByteArrayOutputStream();
StreamResult sr = new StreamResult(new OutputStreamWriter(bos, output_encoding));

//          StringWriter sW = new StringWriter();
//          StreamResult resultRepeat = new StreamResult(sW);

transformer.transform(domSourceRepeat, sr);
byte[] outputBytes = bos.toByteArray();
strRepeatString = new String(outputBytes, output_encoding);