- Jul 19, 2010
-
-
rtoy authored
reference fd-stream-char-to-octets-error and don't call %make-fd-stream with :char-to-octets-error and :octets-to-char-error since these don't exist without unicode.
-
- Jul 05, 2010
- Jul 03, 2010
-
-
rtoy authored
o Update comments for the various slots in DEFINE-EXTERNAL-FORMAT. fd-stream.lisp: o Declare the types for the CHAR-TO-OCTETS-ERROR and OCTETS-TO-CHAR-ERROR slots in FD-STREAM. o Update docstrings for MAKE-FD-STREAM and OPEN for :DECODING-ERROR and :ENCODING-ERROR parameters.
-
rtoy authored
variable that was shadowing another.
-
rtoy authored
where the external format does not have a flush-state method.
-
rtoy authored
o Add error parameter to flush-state in external format definition so we can handle errors when flushing the state to a stream. o Add optional error parameter to flush-state macro. code/fd-stream.lisp: o For the case where an external format has flush method, EF-FLUSH was not calling it correctly. Update so the output function actuall works. o Add error handler to call to flush-state. o For the case where an external format does not have a flush method, output the state value instead of a replacement character so the external format can handle any errors.
-
- Jul 02, 2010
-
-
rtoy authored
o Pass the error handler on for composed external formats. code/fd-stream.lisp: o Forgot to pass the error-handler to char-to-octets in EF-COUT. o In MAKE-FD-STREAM slightly change handling of encoding-error and decoding-error: - If :encoding-error is a character, then the external format will use that character whenever an encoding error happens instead of the default (internally specified by the external format). - If :decoding-error is a character, then the external format will use that character whenever an encoding error happens instead of the default (internally specified by the external format). If :decoding-error is T, then a cerror is signaled; if continued, the Unicode replacement character (#\U+FFFD) is used. o Fix bug in OPEN: The :decoding-error and :encoding-error keyword parameter was placed in the &aux section by mistake.
-
rtoy authored
o The optional error parameter doesn't need to be optional in DEFINE-EXTERNAL-FORMAT, EF-STRING-TO-OCTETS, EF-OCTETS-TO-STRING, EF-ENCODE and EF-DECODE. code/fd-stream.lisp: o Update comments for char-to-octets-error and octets-to-char-error. o Forgot to pass the error handler to char-to-octets in EF-SOUT and EF-STRLEN.
-
rtoy authored
-
rtoy authored
code/extfmts.lisp o Call the error handler for iso8859-1 output. o In OCTETS-TO-CODEPOINT and CODEPOINT-TO-OCTETS, call the external format with the error argument. o In OCTETS-TO-CHAR - Call OCTETS-TO-CODEPOINT with the error handler. - For all of the error conditions, call the error handler if defined. o Add error parameter to EF-STRING-TO-OCTETS and EF-ENCODE so we can handle errors. Call CHAR-TO-OCTETS with the error handler. o Add error parameter to STRING-TO-OCTETS and use it. o Add error parameter to EF-OCTETS-TO-STRING and EF-DECODE so we can handle errors. Call OCTETS-TO-CHAR with the error handler. o Add error parameter to OCTETS-TO-STRING and use it. o In STRING-ENCODE and STRING-DECODE, call the ef function with the error handler. o Change STRING-ENCODE to use keyword args instead of optional args. Add error parameter and use it. code/fd-stream-extfmt.lisp: o Tell OCTETS-TO-STRING about the error handler stored in the fd-stream. code/fd-stream.lisp: o OPEN, MAKE-FD-STREAM, and OPEN-FD-STREAM get DECODING-ERROR and ENCODING-ERROR keyword arguments for specifying how to handle decoding and encoding errors in external formats. code/stream.lisp: o Make sure the error handler is called in FAST-READ-CHAR-STRING-REFILL. pcl/simple-streams/external-formats/utf-8.lisp: o Initial cut at calling the error handler for the various possible invalid octet streams for a utf-8 encoding.
-
- Jun 30, 2010
-
-
rtoy authored
when converting octets to characters or characters to octets. These aren't currently being used, but will be. A basic cross-compile is needed. When CMUCL asks about incompatible structures, select the CLOBBER-IT restart in all cases.
-
- Jun 07, 2010
-
-
rtoy authored
o OPEN merges the filename with *DEFAULT-PATHNAME-DEFAULTS*, as required by CLHS sec 19.2.3. code/filesys.lisp: o DELETE-FILE, ENSURE-DIRECTORIES-EXIST, and FILE-AUTHOR merge the filename with *DEFAULT-PATHNAME-DEFAULTS*, as required by CLHS sec 19.2.3. general-info/release-20b.txt: o Update.
-
- Apr 20, 2010
-
-
rtoy authored
may get confused with source locations if the reader macros are installed.
-
- Apr 19, 2010
-
-
rtoy authored
-
- Mar 19, 2010
-
-
rtoy authored
boot-2010-02-1 as the bootstrap file. You should probably also use the new -P option for build.sh to generate and update the po files while building.
-
- Jan 25, 2010
-
-
rtoy authored
up another string.
-
- Jan 23, 2010
-
-
rtoy authored
everything is defined yet. Hence, add dummy %SET-FD-STREAM-EXTERNAL-FORMAT and move the real one to fd-stream-extfmt.lisp. This builds. code/fd-stream.lisp: o Always call %SET-FD-STREAM-EXTERNAL-FORMAT, even if LISP::*ENABLE-STREAM-BUFFER-P* is NIL. code/stream.lisp: o Move %SET-FD-STREAM-EXTERNAL-FORMAT to fd-stream-extfmt.lisp. o Add dummy implementation of %SET-FD-STREAM-EXTERNAL-FORMAT. code/fd-stream-extfmt.lisp: o %SET-FD-STREAM-EXTERNAL-FORMAT moved here.
-
rtoy authored
set-routines so that the stream has the specified external format.
-
- Jan 22, 2010
-
-
rtoy authored
file position, we need to convert what's left in the buffer into octets to figure out how many octets haven't been delivered yet.
-
rtoy authored
inadvertently broken when the fast external format work was done. The position did not take into account the number of octets that had been read in but not yet processed.
-
- Oct 18, 2009
-
-
rtoy authored
faster reads on external-formats. This is done by adding an additional buffer to streams so we can convert the entire in-buffer into characters all at once. To build this change, you need to do a cross-compile using boot-2009-10-1-cross.lisp. Using that build, do a normal build with these sources. For a non-unicode build use boot-2009-10-01.lisp with a 20a non-unicode build. code/extfmts.lisp: o Add another slot to the extfmts for copying the state. o Modify EF-OCTETS-TO-STRING and OCTETS-TO-STRING to support the necesssary changes for fast formats. This is incompatible with the previous version because the string is not grown if needed. code/fd-stream-extfmt.lisp: o Set *enable-stream-buffer-p* to T so we have fast streams. code/fd-stream.lisp: o Add new slots to support fast strams. o In SET-ROUTINES, initialize the new slots appropriately. o Update UNREAD-CHAR to be able to back up in the string buffer to unread. o Add implementation to copy the state of an external format. code/stream.lisp: o Change %SET-FD-STREAM-EXTERNAL-FORMAT to be able to change formats even if we've already converted the buffer with a different format. We reconvert the buffer with the old format until we reach the current character. Then the remaining octets are converted using the new format and stored in the string buffer. o Add FAST-READ-CHAR-STRING-REFILL to refill the string buffer, like FAST-READ-CHAR-REFILL does for the octet in-buffer. code/struct.lisp: o Add new slots to hold the string buffer, the current index, and length. These are needed for the fast formats. code/sysmacs.lisp: o Update PREPARE-FOR-FAST-READ-CHAR, DONE-WITH-FAST-READ-CHAR, and FAST-READ-CHAR to support the string buffer. code/string.lisp: o Microoptimization of SURROGATEP to reduce the number of branchs. general-info/release-20b.txt: o Update with these changes pcl/simple-streams/external-formats/utf-16-be.lisp: pcl/simple-streams/external-formats/utf-16-le.lisp: pcl/simple-streams/external-formats/utf-16.lisp: o These formats actually have state, so update them to take a handle an initial state. These are needed if the string buffer ends with a leading surrogate and the next string buffer starts with a trailing surrogate. The conversion needs to combine the surrogates together.
-
- Sep 09, 2009
-
-
rtoy authored
-
- Aug 26, 2009
-
-
rtoy authored
closing an output stream. This causes things like (with-open-file (s "foo" :direction :output :external-format :utf-8) (write-char #\u+d800 s)) to output the replacement character instead of creating an empty file. code/extfmts.lisp: o Add new slot to efx structure to hold the function to flush the state in an external format. o Add accessor for the flush-state slot. o Update DEFINE-EXTERNAL-FORMAT to allow specifying the flush function. o Add macro to call the flush-state function. o Added +EF-FLUSH+ o Use vm::defenum to name the constants instead of the hand-written values. o Export +REPLACEMENT-CHARACTER-CODE+ o Document the slots in an efx stucture. code/fd-stream.lisp: o Add ef-flush def-ef-macro to flush the state of an external format when closing an output file. If ef-flush-state is NIL, we just call EF-COUT to send out the replacement character. Otherwise, the flush-state function is called to handle it. o When closing an output character stream, call ef-flush to flush any state before flushing the buffers of the stream. o Document the unicode slots in an fd-stream. code/exports.lisp: o Export +REPLACEMENT-CHARACTER-CODE+
-
- Aug 10, 2009
-
-
rtoy authored
o Standard streams no longer change formats when *default-external-format* changes. Use stream:set-system-external-format instead, or (setf external-format). o char-to-octets properly handles surrogate characters being written. o Makes simple-streams work again. This change needs to be cross-compiled. 2009-07 binaries work for cross-compiling using the 19e/boot-2008-05-cross-unicode-*.lisp cross-compile script.
-
- Jun 25, 2009
-
-
rtoy authored
o Handle the surrogate pairs in our UTF-16 strings.
-
- Jun 24, 2009
-
-
rtoy authored
-
- Jun 11, 2009
-
-
rtoy authored
unicode-utf16-extfmt-2009-06-11.
-
- Nov 05, 2007
-
-
rtoy authored
simple-streams buffers) into LISP and removes them from simple-streams (and from EXT). Use boot-2007-11-1.lisp to bootstrap the necessary symbol changes. Patch from Paul Foley.
-
- Feb 27, 2006
-
-
rtoy authored
original. This is the solution from Carl, for a bug reported by Madhu, cmucl-imp, 2006-02-19.
-
- Apr 04, 2005
-
-
rtoy authored
is done, based on a suggestion from Lynn Quam. o Add BINARY-TEXT-STREAM structure for streams that support binary and character I/O. This is basically an FD-STREAM. o Export BINARY-TEXT-STREAM from the EXTENSIONS package. o OPEN will recognize :class 'ext:binary-text-stream to enable such streams. o Removed *FD-STREAM-ENABLE-CHARACTER-AND-BINARY-INPUT*, since we use OPEN to specify that now. Use boot-2005-03-2.lisp to bootstrap this change.
-
- Feb 21, 2005
-
-
rtoy authored
streams opened with element-type (unsigned-byte 8) to be also opened for character input. This gives simple-streams semantics to fd-streams. * code/fd-stream.lisp o Make FD-STREAM-IN use PICK-INPUT-ROUTINE to select the correct type of input. o Add *FD-STREAM-ENABLE-CHARACTER-AND-BINARY-INPUT* to allow binary and character input. Currently defaults to NIL, but will change to T. * code/stream.lisp o New READ-INTO-SIMPLE-STRING to allow stream element-type of (unsigned-byte 8). o Support reading and writing simple arrays of single-float and double-float. o Strings can be written to streams of element-type '(unsigned-byte 8). * tools/worldcom.lisp o Compile stream-vector-io.lisp. * tools/worldload.lisp o Load stream-vector-io. * code/stream-vector-io.lisp o New file implementing READ-VECTOR and WRITE-VECTOR.
-
- Feb 10, 2005
-
-
rtoy authored
connection is reset. Based on code from Scott Burson, cmucl-imp, 2005-01-31.
-
- Sep 23, 2004
-
-
rtoy authored
it always DPB'ed the result into -1, which always results in a negative number. We need to check the most significant bit to decide what to do.
-
- Apr 23, 2004
- Apr 15, 2004
-
-
rtoy authored
o FILE-STRING-LENGTH can be used on BROADCAST-STREAM's where the result is 1. o Graphic characters are not printed using the character name (hence #\space is printed #\ ) o Make INPUT-STREAM-P and OUTPUT-STREAM-P work correctly on synonym streams. o MAKE-STRING-INPUT-STREAM was not accepting NIL for the optional END argument. o Close string-input streams. (We weren't before.)
-
- Apr 06, 2004
-
-
emarsden authored
- fix defaulting behaviour of :if-not-exist in OPEN when used with :direction :io. Corrects a number of the OPEN.IO.* ANSI compliance tests.
-
- Mar 26, 2004
-
-
emarsden authored
- add a SIMPLE-STREAM-ERROR class, and use it when signaling errors from fd-streams DO-OUTPUT (noted by Matthew Danish) - fix bug in fd-stream error handling (patch from Ole Rohne)
-
- Nov 05, 2003
-
-
gerd authored
as last change, which fixed write-char etc. only.
-