Skip to content
  • rtoy's avatar
    Merge changes from unicode-string-buffer-impl-branch which gives · 392d3e59
    rtoy authored
    faster reads on external-formats.  This is done by adding an
    additional buffer to streams so we can convert the entire in-buffer
    into characters all at once.
    
    To build this change, you need to do a cross-compile using
    boot-2009-10-1-cross.lisp.  Using that build, do a normal build with
    these sources.
    
    For a non-unicode build use boot-2009-10-01.lisp with a 20a
    non-unicode build.
    
    code/extfmts.lisp:
    o Add another slot to the extfmts for copying the state.
    o Modify EF-OCTETS-TO-STRING and OCTETS-TO-STRING to support the
      necesssary changes for fast formats.  This is incompatible with the
      previous version because the string is not grown if needed.
    
    code/fd-stream-extfmt.lisp:
    o Set *enable-stream-buffer-p* to T so we have fast streams.
    
    code/fd-stream.lisp:
    o Add new slots to support fast strams.
    o In SET-ROUTINES, initialize the new slots appropriately.
    o Update UNREAD-CHAR to be able to back up in the string buffer to
      unread.
    o Add implementation to copy the state of an external format.
    
    code/stream.lisp:
    o Change %SET-FD-STREAM-EXTERNAL-FORMAT to be able to change formats
      even if we've already converted the buffer with a different format.
      We reconvert the buffer with the old format until we reach the
      current character.  Then the remaining octets are converted using
      the new format and stored in the string buffer.
    o Add FAST-READ-CHAR-STRING-REFILL to refill the string buffer, like
      FAST-READ-CHAR-REFILL does for the octet in-buffer.
    
    code/struct.lisp:
    o Add new slots to hold the string buffer, the current index, and
      length.  These are needed for the fast formats.
    
    code/sysmacs.lisp:
    o Update PREPARE-FOR-FAST-READ-CHAR, DONE-WITH-FAST-READ-CHAR, and
      FAST-READ-CHAR to support the string buffer.
    
    code/string.lisp:
    o Microoptimization of SURROGATEP to reduce the number of branchs.
    
    general-info/release-20b.txt:
    o Update with these changes
    
    pcl/simple-streams/external-formats/utf-16-be.lisp:
    pcl/simple-streams/external-formats/utf-16-le.lisp:
    pcl/simple-streams/external-formats/utf-16.lisp:
    o These formats actually have state, so update them to take a handle
      an initial state.  These are needed if the string buffer ends with a
      leading surrogate and the next string buffer starts with a trailing
      surrogate.  The conversion needs to combine the surrogates together.
    392d3e59