index.html 3.73 KB
Newer Older
marijnh's avatar
marijnh committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <title>Trivial UTF-8</title>
  <link rel="stylesheet" type="text/css" href="style.css"/>
  <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
</head>

<body>
 <div class="header">
   <h1>Trivial UTF-8</h1>
 </div>

<p>Trivial UTF-8 is a small library for doing UTF-8-based in- and
output <em>on a Lisp implementation that already supports Unicode</em>
-- meaning <tt>char-code</tt> and <tt>code-char</tt> deal with Unicode
character codes.</p>

<p>The rationale for the existence of this library is that while
Unicode-enabled implementations usually do provide some kind of
interface to dealing with character encodings, these are typically not
flexible or efficient enough. Specifically, SBCL's
<tt>sb-ext:octets-to-string</tt> and <tt>string-to-octets</tt> are 10
times slower than the equivalents in this library (and not easily
optimized because of the way they are defined -- the
<tt>use-value</tt> restart in particular), and do not provide a way to
directly read or write UTF-8 from or to a stream.</p>

<h2>Download and installation</h2>

<p>The latest release of trivial-utf-8 can be downloaded from <a
href="http://common-lisp.net/project/trivial-utf-8/trivial-utf-8.tgz">http://common-lisp.net/project/trivial-utf-8/trivial-utf-8.tgz</a>,
or installed with <a
href="http://www.cliki.net/ASDF-Install">asdf-install</a>.</p>

marijnh's avatar
marijnh committed
38
39
40
41
<p>A <a href="http://www.darcs.net/">darcs</a> repository with the most recent changes can be checked out with:</p>

<p><pre>&gt; darcs get http://common-lisp.net/project/trivial-utf-8/darcs/trivial-utf-8</pre></p>

marijnh's avatar
marijnh committed
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
<h2>Support and mailing lists</h2>

<p>The <a href="http://common-lisp.net/mailman/listinfo/trivial-utf-8-devel">trivial-utf-8-devel</a> mailing list can be used for any questions, discussion, bug-reports, patches, or anything else relating to this library.</p>

<h2>Reference</h2>

<p class="def">function <tt>string-to-utf-8-bytes</tt> (string) => array of (unsigned-byte 8)</p>

<p class="desc">Convert a string into an array of unsigned bytes containing its
utf-8 representation.</p>

<p class="def">function <tt>utf-8-bytes-to-string</tt> (bytes) => string</p>

<p class="desc">Convert a byte array containing utf-8 encoded characters into the
string it encodes.</p>

<p class="def">function <tt>write-utf-8-bytes</tt> (string)</p>

<p class="desc">Write a string to a byte-stream, encoding it as utf-8.</p>

<p class="def">function <tt>read-utf-8-string</tt> (input &amp;key null-terminated stop-at-eof char-length byte-length)</p>

<p class="desc">Read utf-8 encoded data from a byte stream and construct a string
with the characters found. When null-terminated is given it will stop
reading at a null character, stop-at-eof tells it to stop at the end
of file without raising an error, and the char-length and byte-length
parameters can be used to specify the max amount of characters or
bytes to read.</p>

<p class="def">function <tt>utf-8-byte-length</tt> (string) => integer</p>

<p class="desc">Calculate the amount of bytes needed to encode a string.</p>

<p class="def">function <tt>utf-8-group-size</tt> (byte) => integer</p>

<p class="desc">Determine the amount of bytes that are part of the character
starting with a given byte.</p>

<p class="def">condition <tt>utf-8-decoding-error</tt></p>

<p class="desc">A condition of this type is raised whenever an incorrectly encoded
character is encountered.</p>

<hr/>

<p>Back to <a href="http://common-lisp.net/">Common-lisp.net</a>.</p>

 <div class="check">
   <a href="http://validator.w3.org/check/referer">Valid XHTML 1.0 Strict</a>
 </div>
</body>
</html>