Add FSet/Jzon subsystem
This MR adds a new Quicklisp-loadable system, FSet/Jzon, wrapping the Jzon JSON parser/printer. It parses into FSet collections — JSON arrays parse into FSet seqs, and by default, JSON objects parse into FSet replay maps — and can also print these in JSON format. Two other FSet representations for JSON objects are also supported, plain maps and dynamic tuples.
FSet tuple keys have a new feature: you can optionally specify the type of the values stored under a key; FSet will signal an error if you try to assign a value to a key in a tuple that is not of the specified type. It will also signal an error if the default provided for the key is not of that type. (FSet/Jzon uses this feature to control how nil is printed in JSON; see below.)
Parsing
The FSet/Jzon parsing API exports many of the same names as Jzon, and is mostly used in the same way. The exception is when the input contains, potentially, more than one top-level JSON value. In this case, pass :allow-multiple-content t to make-parser or with-parser, as you would have with Jzon, but then call parse-top-level once for each top-level value you wish to parse. FSet/Jzon does not export parse-next.
As mentioned, JSON arrays parse into FSet seqs. For JSON objects, you have a choice. The default is to parse them into replay maps; these preserve the order of the keys of each object for printing purposes. Using the :object-type keyword argument to with-parser or make-parser, you can alternatively request plain maps, or tuples. The plain maps use the CHAMP implementation, so the keys will (usually) be reordered on printing, but they use less space than replay maps. In these cases the keys will be strings by default, though you can use other types by passing an appropriate :key-fn.
Tuples are FSet's most space-efficient representation of JSON objects, but require a bit more work to set up. You will need a tuple key for each key in the JSON you're parsing. The name of each tuple key should contain the corresponding JSON key string, normally upcased and with a prefix and/or suffix added; the use of a prefix and/or suffix is optional, but highly recommended. For example, if your JSON contains keys "size" and "color", the corresponding tuple key names could be +JSON-SIZE+ and +JSON-COLOR+ (written here in uppercase because that's how they'll print, but of course you can normally write them in lowercase in source code). Thus the prefix is "+JSON-" and the suffix is "+". With that in mind, the steps are:
- For each key that you want to access in your program, write an
fset2:define-tuple-keyform. - To
with-parserormake-parser, pass:object-type 'tuple; for:key-package, the package containing the tuple key names; and for:key-prefixand:key-suffix, the prefix and suffix to use. (If you're using a nonstandard reader case mode, or have mixed-case keys in your JSON input, you can also pass a:key-case-modeof:downcaseor:preserve;:upcaseis the default.)
On printing, the keys will be in the order in which they were declared, or if some keys were found in the input that had not been declared, the order in which they were first encountered. (You can still call fset2:define-tuple-key on them after parsing, if you hadn't done it beforehand.)
There is one other reason you might want to use tuples for objects, which has to do with how nil is printed; see below.
- Type
json-atom - Re-exported from Jzon.
- Type
json-element (or json-atom seq map tuple)- Macro
with-parser(var in &rest kwd-args) &body body - Binds
vararoundbodyto an FSet/Jzon parser that will read fromin, which can be a vector, a stream, or a pathname. The keyword arguments are passed unchanged tomake-parser. - Function
make-parserin &key allow-comments allow-trailing-comma allow-multiple-content (max-string-length (min 1048576 (1- array-dimension-limit))) (max-depth 128) (key-fn (make-fset-string-pool)) (object-type 'replay-map) key-package (key-case-mode ':upcase) (key-prefix "") (key-suffix "") - If
allow-commentsis true, comments as in C (//for single-line or/* ... */for block) will be accepted. Ifallow-trailing-comma, a comma will be permitted after the last element of an array or object.max-string-lengthlimits the length of strings, andmax-depththe maximum nesting depth; these help protect against DoS attacks.key-fncontrols whether and how string pooling is done; the default uses an FSet set. - The returned parser should be closed after use; see
close-parser, and macrowith-parserwhich closes it automatically. - If
allow-multiple-contentis true, the input may contain multiple top-level JSON values, which you can read by callingparse-top-levelmultiple times. - JSON arrays are returned as FSet seqs; JSON objects are returned according to
object-type, which can bereplay-map,map, ortuple(symbols in packagefset2). Ifkey-packageis supplied, keys will be interned in this package; this option overrideskey-fn. Ifobject-typeistuple,key-packageis required, and may not be the CL keyword package. Ifkey-packageis supplied: (a)key-case-modecan be any of:upcase,:downcase, or:preserve; the default is:upcase. (b) If supplied,key-prefixis prepended andkey-suffixis appended to the case-converted key before interning.
Other parsing-related symbols re-exported from Jzon are: json-atom, json-error, json-limit-error, json-parse-error, json-parse-limit-error, and json-eof-error.
Printing
For printing, FSet/Jzon mostly just re-exports symbols from Jzon; you invoke the printer in the same way, but methods have been added to print the FSet collections that the parser can produce. So the tree you supply can contain anything that Jzon knows how to print, possibly with FSet collections mixed in. The printing-related Jzon symbols re-exported are: stringify, json-write-error, json-write-limit-error, json-recursive-write-error, coerced-fields, coerce-key, writer, make-writer, close-writer, and with-writer.
The one new printing-related function concerns tuple keys. As noted above, you may wish, when parsing JSON objects into tuples, to wrap the key strings with a prefix and/or suffix before interning them. When printing, you will probably want to strip these off. FSet/Jzon exports the generic function coerce-tuple-key (see below), which gives you a package-specific way to do that.
Jzon has three different ways it can print nil, depending on the type of the class slot holding the value: [] for a list; false for a boolean; or null for all other types. If you're using tuples to represent JSON objects, you can get the same effect by using the new :type keyword argument to fset2:define-tuple-key. (Of course, an empty seq will always print as [].) Examples:
(define-tuple-key +json-color+ :type (or null string))
(define-tuple-key +json-visible?+ :type boolean)
- Function
coerce-tuple-keykey key-name-package-as-keyword - Generic function called to turn a `tuple-key' into a string. The first argument is the key; the second is the key's name's package name, as a keyword symbol. So you can write methods like
(defmethod coerce-tuple-key (key (pkg (eql :my-package)))
...)