Splunk Apps » Website Input

http://answers.splunk.com/answers/144939/website-input-charset-problem

This is how html5lib does it:

"If no encoding is specified, the parser will attempt to detect the encoding from a <meta> element in the first 512 bytes of the document (this is only a partial implementation of the current HTML 5 specification).

If no encoding can be found and the chardet library is available, an attempt will be made to sniff the encoding from the byte pattern.

If all else fails, the default encoding will be used. This is usually Windows-1252, which is a common fallback used by Web browsers."