Commit 23485475 authored by Eric Timmons's avatar Eric Timmons
Browse files

Rename to tar-file

parent 22a64d96
Copyright (c) 2004, Nathan Froyd. All rights reserved.
Copyright (c) 2021, Eric Timmons.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
-*- mode: outline -*-
* Version 0.9, released 16-12-2012
** bugfixes
Handling of absolute pathnames when creating tar archives was fixed.
(Thanks to Vsevolod Dymokin.)
Handling of *PRINT-CASE* when interning symbols was fixed. (Thanks to
Orivej Desh.)
* Version 0.8.1, release 19-07-2010
** new features
Support was adding for writing directories and recursively adding files
in directories. (Thanks to Vsevolod Dyomkin.)
A condition hierarchy was added.
* Version 0.8, released 18-12-2009
** bugfixes
ARCHIVE can now read POSIX extended header entries, such as those
created by git. The data from them are currently ignored.
Fixed extraction of files on SBCL, CMUCL, and Lispworks. (Thanks to
Elliot Slaughter.)
Fixed compilation error in CLISP. (Thanks to Elliot Slaughter.)
Fixed calling out to _stat on Lispworks/Win32. (Thanks to Luke Crook.)
* Version 0.7, released 09-02-2008
** bugfixes
Due to a packaging snafu, the Lispworks/Win32 support contributed by
Sean Ross was not included in previous versions. This oversight has
been fixed.
ARCHIVE now works correctly with archives created by BSD tar variants.
(Thanks to Matthew Kennedy.)
ARCHIVE now works much better on Win32 SBCL. (Thanks to Matthew
* Version 0.6, released 30-04-2007
** incompatible changes
renamed to :STREAM and its semantics changed. Please see the
documentation string for details about the semantics of :STREAM.
The dependence on FLEXI-STREAMS has been removed; ENTRY-STREAM now
returns a stream with element-type (UNSIGNED-BYTE 8). This was done to
facilitate using implementation-dependent translation streams.
** bugfixes
The names of the archive classes (TAR-ARCHIVE, ODC-CPIO-ARCHIVE,
SVR4-CPIO-ARCHIVE) have been exported from ARCHIVE for easier access.
has been tightened.
** new features
ARCHIVE now knows how to extract directories.
WITH-OPEN-ARCHIVE now accepts either a pathname or a stream from which
to open an archive. (Thanks to Sean Ross.)
ARCHIVE can now create archives on Lispworks/Win32. (Thanks to Sean
A new generic function, (SETF NAME) has been added for setting the
name of an archive entry.
A family of functions, ENTRY-<foo>-P, have been added for determining
what sort of file an entry is associated with. Please see the
documentation strings for these functions for details.
* Version 0.5, released 12-01-2006
The README contains a few examples of the library's use.
** bugfixes
Several bugs have been fixed with this release. The file buffering
code, a source of bugs in the past, has been completely re-written.
** new features
ARCHIVE now understands GNU-style "long filename" and "long linkname"
entries in tar files.
CLISP is now a supported implementation for creating archives. Other
relevant fixes for CLISP have been made as well. (Thanks to Magnus
CMUCL is now a supported implementation for creating archives.
Accessing the data associated with an archive entry has been revamped
thanks to the use of Edi Weitz's FLEXI-STREAMS library. Please see
the documentation string for ENTRY-STREAM for an explanation and the
example of its use in the README file.
* Version 0.3.1, released 11-02-2005
** new features
Sean Ross contributed a compatibility layer for Lispworks as well as
some other small fixes.
* Version 0.2, released 05-01-2004
ARCHIVE no longer comes in a single 'tar.lisp' file, instead being
split across multiple files and tied together with an ASDF file, as
is standard for Common Lisp projects nowadays.
** new features
Support for reading cpio archives. The input layer was rewritten
somewhat to support this, but the output layer has not yet had the
corresponding changes made. Old-style and SVR4-style cpio archives are
* Version 0.1, released 15-12-2004
Initial release
This is the README for ARCHIVE, a package for reading an writing disk-based
file archives such as those generated by the 'tar' program on Unix. This
package aspires to be a pure Common Lisp replacement for the 'tar' program.
Current functionality includes basic extraction from and creation of tar
ASDF packaging is provided; (asdf:oos 'asdf:load-op :archive) should be
all you need to get started. Once you have the system loaded, you can
try the following small example, which replicates the functionality of
'tar tf':
(defun list-archive-entries (pathname)
(archive:with-open-archive (archive pathname :direction :input)
(archive:do-archive-entries (entry archive)
(format t "~A~%" (archive:name entry)))))
Access to the data for individual entries is also provided.
ENTRY-STREAM returns a stream that provides access to the raw bytes of
data for the entry. If you want to access it as text, you need to use a
library to encode the bytes into characters, or wrap the stream using
Edi Weitz's FLEXI-STREAMS. The following example prints out the first
line of every file in the archive (assuming that the entry is a text
file, of course):
(defun first-line-of-archive-entries (pathname)
(archive:with-open-archive (archive pathname :direction :input)
(archive:do-archive-entries (entry archive)
(when (archive:entry-regular-file-p entry)
(let ((stream (flexi-streams:make-flexi-stream (entry-stream entry))))
(format t "~A~%" (read-line stream)))))))
Another thing to do is create archives:
;;; This function is actually included in ARCHIVE.
(defun create-tar-file (pathname filelist)
(archive:with-open-archive (archive pathname :direction :output
:if-exists :supersede)
(dolist (file filelist (archive:finalize-archive archive))
(let ((entry (archive:create-entry-from-pathname archive file)))
(archive:write-entry-to-archive archive entry)))))
Note that you need to call FINALIZE-ARCHIVE once you are done adding
Comments, criticisms, additions, and optimizations are welcome at the
below email address.
Nathan Froyd
#+TITLE: Tar File
This project provides functionality to read and write a variety of tar archive
formats. Currently supported are the POSIX ustar, PAX (ustar with a few new
entry types), GNU, and v7 (very old) formats.
This project is rather low level and is focused exclusively on reading and
writing tar file entries using streams. Therefore, it contains no functionality
to automatically build archives from a set of files on the filesystem or write
the contents of a file to the filesystem. Additionally, there are no smarts
that read multiple physical entries and combine them into a single logical
entry (e.g., with PAX extended headers or GNU long link/path name
support). This approach is taken to both improve portability (the CL spec has
no facility for things like detecting symlinks or modifying/reading file
owners, etc.) and properly separate concerns. Another system will be written
that does the non portable bits and other systems can use tar-file directly if
they have a need to perform low level operations on tar archives.
This project is a fork of [[][Nathan Froyd's archive library]]. Much code remains,
but the non-portable bits have been stripped, better support for multiple
archive types (and autodetection) has been added, better blocking support added
(tar readers/writers are supposed to read/write in multiples of 512 bytes),
cpio support removed, and a test suite added, along with other miscellaneous
fixes and improvements.
One major user visible difference between this library and the original archive
is that there is no need to discard entries when you are finished with them and
support has been added for seeking within a stream using FILE-POSITION. This
means you can do something like iterate over all entries in one go and then get
a stream containing the contents of an arbitrary entry.
However, care must be taken if you do this as it will only work if the
underlying stream containing the tar archive is seekable. Typically streams
backed by files *are* seekable, but all other input streams (including stdin)
*are not*. Therefore if there is any chance your code that uses =tar-file= will
read files from non-seekable streams, you should process entries sequentially
and completely before moving on to the next one. Alternatively, you can set the
blocking factor high enough that the entire file is read into memory at once
-*- mode: outline -*-
* Support other CL implementations
Any CL implementation should be able to read archives
SBCL/CMUCL/CLISP/Lispworks are the only implementations capable of
writing archives at this point.
* Improving extraction
Extraction really only supports plain files. Directories, at the very
least, should be supported. Symbolic and hard links would be good.
Better restoration of metadata (keyword arguments to the EXTRACT-ENTRY
call) would also be desirable.
* Make buffering conform to the POSIX standard
Tar files, for example, are supposed to read and write data in blocks
of 10240 bytes, which ARCHIVE does not currently do.
* Read/write support for pax-style archives
* Support for writing cpio archives
Self-explanatory, although people are probably not clamoring for this.
* Autodetection of archive formats
* Appending to archives
This support will be tricky to add.
* Robust error handling, particularly with truncated archives
;;;; archive.lisp
(in-package #:tar)
(in-package #:tar-file)
(defvar *archive-type-detectors* nil
"A list of functions, that when called with a header buffer must return a
......@@ -2,7 +2,7 @@
;;;; This is part of cl-tar. See README and LICENSE for more information.
(in-package #:tar)
(in-package #:tar-file)
(define-condition blocked-stream-error (stream-error)
(in-package #:tar)
(in-package #:tar-file)
(defclass bounded-input-stream (trivial-gray-streams:trivial-gray-stream-mixin
;;;; buffer-ops.lisp
(in-package #:tar)
(in-package #:tar-file)
(defun read-number-from-buffer (buffer &key (start 0) end (radix 10))
(declare (type (simple-array (unsigned-byte 8) (*)) buffer))
;;;; conditions.lisp -- conditions signalled by the library
(in-package #:tar)
(in-package #:tar-file)
(define-condition archive-error (error)
;;;; constants.lisp -- tar constants
(in-package #:tar)
(in-package #:tar-file)
;;; constants and class definitions
(defconstant +tar-n-block-bytes+ 512
;;;; entry.lisp -- abstract entry classes
(in-package #:tar)
(in-package #:tar-file)
(defclass entry ()
(in-package #:tar)
(in-package #:tar-file)
(defun list-archive-entries (pathname)
(with-open-archive (archive pathname)
;;;; macros.lisp -- various external macros
(in-package #:tar)
(in-package #:tar-file)
(defun call-with-open-archive (thunk pathname &key (direction :input)
(if-exists nil)
(in-package #:tar)
(in-package #:tar-file)
(defgeneric (setf name) (value entry)
;;;; gnu.lisp -- header format for GNU archives
(in-package #:tar)
(in-package #:tar-file)
(defparameter *gnu-magic-vector*
(coerce `(,@(map 'list #'char-code "ustar "))
;;;; macros.lisp -- various internal macros
(in-package #:tar)
(in-package #:tar-file)
(defun extractor-function-name (entry-name field-name)
(intern (with-standard-io-syntax (format nil "~A-READ-~A-FROM-BUFFER" entry-name field-name))))
(uiop:define-package #:tar
(:use :cl #+(and lispworks (not win32)) :sys)
(uiop:define-package #:tar-file
(:use :cl)
(:export ;; types
#:archive #:tar-archive
;;;; ustar.lisp -- header formts for ustar
(in-package #:tar)
(in-package #:tar-file)
(defparameter *ustar-magic-vector*
(coerce `(,@(map 'list #'char-code "ustar") 0)
(in-package #:tar)
(in-package #:tar-file)
(defparameter *default-header-encoding* :latin-1)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment