Commit 05b63ccd authored by Dave Cooper's avatar Dave Cooper

Added directories for more HTTP server options

parent 4fe9881c
all: build
serve: static favicon
sbcl --load test-serve.lisp
static:
mkdir -p output/static
......@@ -12,3 +9,25 @@ build: static
clean:
rm -rf output
mkdir output
serve: static
sbcl --load test-serve.lisp
paserve: static
sbcl --load servers/paserve/serve.lisp
zaserve: static
sbcl --load servers/zaserve/serve.lisp
htoot: static
sbcl --load servers/htoot/serve.lisp
woo: static
sbcl --load servers/woo/serve.lisp
clack: static
sbcl --load servers/clack/serve.lisp
cl-http: static
sbcl --load servers/cl-http/serve.lisp
......@@ -2,9 +2,7 @@
## Directory structure
The source code for the site generator is located in the
root of the repository. The `t/` directory contains tests
for the site generator.
### Content
The site content is divided over the directories `content/`
and `layout`.
......@@ -36,6 +34,21 @@ If a file by the same name is stored in both
`layout/static/` and `content/static/`, then the one
in `layout/static/` takes precedence.
### Generator Source Code
The source code for the site generator is located in the
root of the repository. The `t/` directory contains tests
for the site generator.
### Server Setup Code (optional)
The `servers/` directory contains code for running various CL HTTP
servers to be able to serve the generated content locally (e.g. while
authoring/editing). It has [its own
README](https://gitlab.common-lisp.net/dcooper/cl-site/blob/master/servers/README.md).
## Dependencies
* CL-MUSTACHE for HTML templating
......@@ -57,7 +70,7 @@ The Makefile contains various useful commands.
Among them:
* `make` - to generate the site.
* `make serve` - to generate the site and start up a test server.
* `make serve` - to generate the site and start up a default test server.
* `make clean` - to remove the generated files from output directory
......@@ -76,13 +89,12 @@ Among them:
(load ".../cl-site/test-serve.lisp") ;; replace with your actual path to cl-site/
This will start serving the site on port 8008 (or subsequent port
numbers, if 8008 is busy).
This will start serving the site. The toplevel URL should be printed to the console.
### Known Problems
The test server crashes on load with `invalid number of arguments` on
certain 1.3.1 releases of SBCL (apparently a problem with the
`ironclad` library from quicklisp with this sbcl version). If you get
this error, try upgrading your SBCL or use CCL.
At least one of the test servers crashes on load with `invalid number
of arguments` on certain 1.3.1 releases of SBCL (apparently a problem
with the `ironclad` library from quicklisp with this sbcl
version). If you get this error, try upgrading your SBCL or use CCL.
......@@ -3,6 +3,8 @@
(push (make-pathname :defaults *load-truename* :name nil :type nil)
quicklisp:*local-project-directories*)
(ql:register-local-projects)
(ql:quickload :cl-site)
(cl-site:make-site)
......
......@@ -37,5 +37,5 @@
wrapped with calls to add-content. Global value should always be
nil.")
(defvar *aserve-port* 8008)
(defvar *aserve-listeners* 5)
(defvar *paserve-port* 8008)
;; FLAG add different ports for other webservers when activated
(in-package :cl-user)
(defpackage #:cl-site
(:use :cl)
(:export #:make-site #:make-clean #:add-content #:wmd #:with-cl-who-string
#:start-aserve #:publish-cl-site))
#:*output-dir*
#:*paserve-port* ;; FLAG export port globals for other webservers when activated
))
(defpackage #:cl-site-content
......
# Common-Lisp.net test webserver suite
## Background
Common Lisp Webservers have a long and storied history going back to
the beginning of the World Wide Web in the mid 1990s. CL servers and
frameworks continue to evolve rapidly today.
There is a somewhat outdated list of CL webservers
[here](https://www.cliki.net/HTTP%20server). Consider volunteering to
help bring this list up to date.
## Purpose
This set of directories is provided as a service to clnet volunteers
as well as a simple testbed for a variety of Common Lisp HTTP servers.
Using the `serve.lisp` file in each of these directories, it should be
possible to start a local webserver which publishes the freshly built
local Common-Lisp.Net content and allows you to browse the site from
your local host.
## Support/Maintenance
Note that the Common-Lisp.Net maintainers cannot commit to maintaining
all these webservers in guaranteed working order at all times. It is
hoped that individual volunteers may step up and take responsibity for
each server (i.e. each subdirectory).
Some of these subdirectories contain only stubs (i.e. placeholders) at
present, and if it represents a server you are interested in, please
consider volunteering to fill in the missing code and maintain that
server for the CLNet community.
If there is a CL webserver which you care about or are curious about,
and it's not listed here, please feel free to add it.
## Usage
### Command Line
Use `make` followed by the desired server name. Currently existing
make targets are (note that many of these are still placeholders):
make serve # starts a default server (from list below) which is expected to work.
make clack # Aims to provide a server-agnostic veneer on top of any actual webserver.
make cl-http # The world's first HTTP1.1 server; powered whitehouse.gov under President Clinton.
make htoot # Hunchentoot (the "800lb Gorilla" of CL webservers).
make paserve # Legacy Portable AllegroServe (reverts to native aserve on Allegro CL).
make woo # The Fastest Webserver in the World.
make zaserve # the new "ZACL compatible" AllegroServe fork (reverts to native aserve on Allegro CL).
### From Lisp
Load the following files to get the corresponding servers running from
within Lisp (note: if you try to do more than one at a time, your
mileage may vary. It'd be interesting though. Who can get them all
running at the same time in the same running CL session? (on distinct
ports, of course, as defined below) (substitute the path to your
`cl-site/` direcotry for the `...`):
(load ".../cl-site/servers/clack/serve")
(load ".../cl-site/servers/cl-http/serve")
(load ".../cl-site/servers/htoot/serve")
(load ".../cl-site/servers/paserve/serve")
(load ".../cl-site/servers/woo/serve")
(load ".../cl-site/servers/zaserve/serve")
## Directory structure
(the following information will mainly be of interest if you want to
volunteer to maintain the code for one of the servers.)
Each subdirectory should contain the following:
* `package.lisp` package definition for `:cl-site-<server>`,
exporting `#:start-server` and `#:publish-cl-site`
* `cl-site-<server>.asd` should contain an ADSF system definition,
including `:depends-on` for the server in question.
* `publish.lisp` should contain code in `:cl-site-<server>` package,
defining the functions `publish-cl-site`, `client-test`, and
`start-server`. See completed examples and documentation in
placeholder code to see what these should do.
* `serve.lisp` should load the `cl-site/build.lisp` file (which loads
core site content and renders it to output), then call the
corresponding servers's `start-server` and `publish-cl-site`
functions. Finally a message should be printed informing the
user how to visit the locally served site in their browser.
Also, for each server there should be a corresponding distinct default
starting port number defined as a global parameter in
`cl-site/globals.lisp` with its symbol exported in
`cl-site/package.lisp` -- see existing one(s) for the expected
nomenclature.
;;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
(in-package :cl-user)
(asdf:defsystem cl-site-test-serve
:name "cl-site-test-serve"
(asdf:defsystem cl-site-cl-http
:name "cl-site-cl-http"
:version "0.0.1"
:maintainer "Common-Lisp.Net maintainers"
:author "Dave Cooper & Common-Lisp.Net maintainers"
:licence "TBD"
:description "Test server for common-lisp.net, written in CL"
:depends-on (:cl-site :aserve)
:components ((:module source
:pathname ""
:serial t
:components ((:file "publish")))))
:description "Test server for common-lisp.net for cl-http"
:depends-on (:cl-site) ;; :cl-http ;; FLAG -- add cl-http when/if available in Quicklisp.
:serial t
:components ((:file "package")
(:file "publish")))
(in-package :cl-user)
(defpackage #:cl-site-cl-http
(:use :cl)
(:export #:start-server #:publish-cl-site))
(in-package :cl-site-cl-http)
(defparameter *listeners* 5)
(defun print-unimplemented-message ()
(format t "
**********************************
Thank you for trying the clnet site cl-Http test server!
cl-http itself is not available in Quicklisp so it is not loaded
yet. So, the first step would be to get it added to Quicklisp (Reinier
Joswig might be able to help with this; when it's fit for inclusion,
you can make a request at
https://github.com/quicklisp/quicklisp-projects/issues
Next, the specific code for serving the cl-site pages for cl-http has
not been filled in yet.
If you'd like to volunteer to do this, and maintain it, please fill in
the `publish-cl-site,' `client-test,' and `start-server' functions in
the file cl-site/servers/cl-http/publish.lisp.
Any changes/additions for cl-http support should be contained in
this directory (cl-site/servers/cl-http/), except for the definition and
exporting of a global variable specifying a distinct cl-http
default starting port `cl-site-cl-http:*port*,' which can go in
cl-site/globals.lisp and cl-site/package.lisp respectively.
For implementing the functions below, it may help to follow the
examples in cl-site/servers/paserve/publish.lisp.
**********************************
"))
(defun publish-cl-site (&key (output-dir cl-site:*OUTPUT-DIR*))
"This function should publish all the site content from the output-dir, both with and without their .html suffices"
(declare (ignore ouput-dir)))
(defun client-test (port)
"This function should check whether the given port is free for listening, and return it if so, nil if not."
(declare (ignore port)))
(defun start-server (&key (port ;;cl-site:*cl-http-port* ;; FLAG Define this global and uncomment this when activated.
) (listeners *listeners*))
"This function should check for the next available port, starting with the given port, and start a webserver
on that port. If starting on that port throws an error, then subsequent ports should be tried, with randomly
increasing wait times in between (a la Ethernet collision avoidance strategy)."
(declare (ignore port listeners))
(print-unimplemented-message))
(in-package :cl-user)
(uiop:with-current-directory (*load-truename*)
(load "../../build"))
(ql:quickload :cl-site-cl-http)
;;
;; Start aserve, publish site directory & pages, set default package.
;;
(cl-site-cl-http:start-server)
(cl-site-cl-http:publish-cl-site)
(setq *package* (find-package :cl-site))
(format t "
This message from cl-site/servers/cl-http/serve.lisp should report the url & port
that the user may visit to browse the site.
")
;;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
(in-package :cl-user)
(asdf:defsystem cl-site-clack
:name "cl-site-clack"
:version "0.0.1"
:maintainer "Common-Lisp.Net maintainers"
:author "Dave Cooper & Common-Lisp.Net maintainers"
:licence "TBD"
:description "Test server for common-lisp.net for Clack"
:depends-on (:cl-site :clack)
:serial t
:components ((:file "package")
(:file "publish")))
(in-package :cl-user)
(defpackage #:cl-site-clack
(:use :cl)
(:export #:start-server #:publish-cl-site))
(in-package :cl-site-clack)
(defparameter *listeners* 5)
(defun print-unimplemented-message ()
(format t "
**********************************
Thank you for trying the clnet site Clack test server!
Clack itself is loaded and ready to go! But, the specific code
for serving the cl-site pages for Clack has not been filled in
yet.
If you'd like to volunteer to do this, and maintain it, please fill in
the `publish-cl-site,' `client-test,' and `start-server' functions in
the file cl-site/servers/clack/publish.lisp.
Any changes/additions for Clack support should be contained in
this directory (cl-site/servers/clack/), except for the definition and
exporting of a global variable specifying a distinct Clack
default starting port `cl-site-clack:*port*,' which can go in
cl-site/globals.lisp and cl-site/package.lisp respectively.
For implementing the functions below, it may help to follow the
examples in cl-site/servers/paserve/publish.lisp.
**********************************
"))
(defun publish-cl-site (&key (output-dir cl-site:*OUTPUT-DIR*))
"This function should publish all the site content from the output-dir, both with and without their .html suffices"
(declare (ignore ouput-dir)))
(defun client-test (port)
"This function should check whether the given port is free for listening, and return it if so, nil if not."
(declare (ignore port)))
(defun start-server (&key (port ;;cl-site:*clack-port* ;; FLAG Define this global and uncomment this when activated.
) (listeners *listeners*))
"This function should check for the next available port, starting with the given port, and start a webserver
on that port. If starting on that port throws an error, then subsequent ports should be tried, with randomly
increasing wait times in between (a la Ethernet collision avoidance strategy)."
(declare (ignore port listeners))
(print-unimplemented-message))
(in-package :cl-user)
(uiop:with-current-directory (*load-truename*)
(load "../../build"))
(ql:quickload :cl-site-clack)
;;
;; Start aserve, publish site directory & pages, set default package.
;;
(cl-site-clack:start-server)
(cl-site-clack:publish-cl-site)
(setq *package* (find-package :cl-site))
(format t "
This message from cl-site/servers/clack/serve.lisp should report the url & port
that the user may visit to browse the site.
")
;;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
(in-package :cl-user)
(asdf:defsystem cl-site-htoot
:name "cl-site-htoot"
:version "0.0.1"
:maintainer "Common-Lisp.Net maintainers"
:author "Dave Cooper & Common-Lisp.Net maintainers"
:licence "TBD"
:description "Test server for common-lisp.net for Hunchentoot"
:depends-on (:cl-site :hunchentoot)
:serial t
:components ((:file "package")
(:file "publish")))
(in-package :cl-user)
(defpackage #:cl-site-htoot
(:use :cl)
(:export #:start-server #:publish-cl-site))
(in-package :cl-site-htoot)
(defparameter *listeners* 5)
(defun print-unimplemented-message ()
(format t "
**********************************
Thank you for trying the clnet site Hunchentoot test server!
Hunchentoot itself is loaded and ready to go! But, the specific code
for serving the cl-site pages for Hunchentoot has not been filled in
yet.
If you'd like to volunteer to do this, and maintain it, please fill in
the `publish-cl-site,' `client-test,' and `start-server' functions in
the file cl-site/servers/htoot/publish.lisp.
Any changes/additions for Hunchentoot support should be contained in
this directory (cl-site/servers/htoot/), except for the definition and
exporting of a global variable specifying a distinct Hunchentoot
default starting port `cl-site-htoot:*port*,' which can go in
cl-site/globals.lisp and cl-site/package.lisp respectively.
For implementing the functions below, it may help to follow the
examples in cl-site/servers/paserve/publish.lisp.
**********************************
"))
(defun publish-cl-site (&key (output-dir cl-site:*OUTPUT-DIR*))
"This function should publish all the site content from the output-dir, both with and without their .html suffices"
(declare (ignore ouput-dir)))
(defun client-test (port)
"This function should check whether the given port is free for listening, and return it if so, nil if not."
(declare (ignore port)))
(defun start-server (&key (port ;;cl-site:*htoot-port* ;; FLAG Define this global and uncomment this when activated.
) (listeners *listeners*))
"This function should check for the next available port, starting with the given port, and start a webserver
on that port. If starting on that port throws an error, then subsequent ports should be tried, with randomly
increasing wait times in between (a la Ethernet collision avoidance strategy)."
(declare (ignore port listeners))
(print-unimplemented-message))
(in-package :cl-user)
(uiop:with-current-directory (*load-truename*)
(load "../../build"))
(ql:quickload :cl-site-htoot)
;;
;; Start aserve, publish site directory & pages, set default package.
;;
(cl-site-htoot:start-server)
(cl-site-htoot:publish-cl-site)
(setq *package* (find-package :cl-site))
(format t "
This message from cl-site/servers/htoot/serve.lisp should report the url & port
that the user may visit to browse the site.
")
;;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
(in-package :cl-user)
(asdf:defsystem cl-site-paserve
:name "cl-site-paserve"
:version "0.0.1"
:maintainer "Common-Lisp.Net maintainers"
:author "Dave Cooper & Common-Lisp.Net maintainers"
:licence "TBD"
:description "Test server for common-lisp.net for Legacy Portable AllegroServe"
:depends-on (:cl-site :aserve) ;; FLAG system name may be updated to :paserve in future QL.
:serial t
:components ((:file "package")
(:file "publish")))
(in-package :cl-user)
(defpackage #:cl-site-paserve
(:use :cl)
(:export #:start-server #:publish-cl-site))
(in-package :cl-site)
(in-package :cl-site-paserve)
(defparameter *listeners* 5)
(defun publish-cl-site ()
(dolist (file (directory (make-pathname :defaults *OUTPUT-DIR* :name :wild :type "html")))
(defun publish-cl-site (&key (output-dir cl-site:*OUTPUT-DIR*))
(dolist (file (directory (make-pathname :defaults output-dir :name :wild :type "html")))
(dolist (format-string (list "/~a" "/~a.htm" "/~a.html"))
(net.aserve:publish-file :path (format nil format-string (pathname-name file))
:file (namestring file))))
(net.aserve:publish-directory :prefix "/"
:destination (namestring *output-dir*)))
:destination (namestring output-dir)))
;;
......@@ -51,7 +51,7 @@
(defun start-aserve (&key (port *aserve-port*) (listeners *aserve-listeners*)
(defun start-server (&key (port cl-site:*paserve-port*) (listeners *listeners*)
;;
;; FLAG -- figure out correct external-format for the other Lisps.
;;
......
(in-package :cl-user)
(uiop:with-current-directory (*load-truename*)
(load "../../build"))
(ql:quickload :cl-site-paserve)
;;
;; Start paserve, publish site directory & pages, set default package.
;;
(cl-site-paserve:start-server)
(cl-site-paserve:publish-cl-site)
(setq *package* (find-package :cl-site))
(let ((wserver net.aserve:*wserver*))
(when wserver
(let ((port (acl-compat.socket::port(net.aserve:wserver-socket wserver))))
(format t "You may now visit http://localhost:~a to test the site.~%" port))))
;;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
(in-package :cl-user)
(asdf:defsystem cl-site-woo
:name "cl-site-woo"
:version "0.0.1"
:maintainer "Common-Lisp.Net maintainers"
:author "Dave Cooper & Common-Lisp.Net maintainers"
:licence "TBD"
:description "Test server for common-lisp.net for Woo"
:depends-on (:cl-site :woo)
:serial t
:components ((:file "package")
(:file "publish")))
(in-package :cl-user)
(defpackage #:cl-site-woo
(:use :cl)
(:export #:start-server #:publish-cl-site))
(in-package :cl-site-woo)
(defparameter *listeners* 5)
(defun print-unimplemented-message ()
(format t "
**********************************
Thank you for trying the clnet site Woo test server!
Woo itself is loaded and ready to go! But, the specific code
for serving the cl-site pages for Woo has not been filled in
yet.
If you'd like to volunteer to do this, and maintain it, please fill in
the `publish-cl-site,' `client-test,' and `start-server' functions in
the file cl-site/servers/woo/publish.lisp.
Any changes/additions for Woo support should be contained in
this directory (cl-site/servers/woo/), except for the definition and
exporting of a global variable specifying a distinct Woo
default starting port `cl-site-woo:*port*,' which can go in
cl-site/globals.lisp and cl-site/package.lisp respectively.
For implementing the functions below, it may help to follow the
examples in cl-site/servers/paserve/publish.lisp.
**********************************
"))
(defun publish-cl-site (&key (output-dir cl-site:*OUTPUT-DIR*))
"This function should publish all the site content from the output-dir, both with and without their .html suffices"
(declare (ignore ouput-dir)))
(defun client-test (port)
"This function should check whether the given port is free for listening, and return it if so, nil if not."
(declare (ignore port)))
(defun start-server (&key (port ;;cl-site:*woo-port* ;; FLAG Define this global and uncomment this when activated.
) (listeners *listeners*))
"This function should check for the next available port, starting with the given port, and start a webserver
on that port. If starting on that port throws an error, then subsequent ports should be tried, with randomly
increasing wait times in between (a la Ethernet collision avoidance strategy)."
(declare (ignore port listeners))
(print-unimplemented-message))
(in-package :cl-user)
(uiop:with-current-directory (*load-truename*)
(load "../../build"))
(ql:quickload :cl-site-woo)
;;
;; Start aserve, publish site directory & pages, set default package.
;;
(cl-site-woo:start-server)
(cl-site-woo:publish-cl-site)
(setq *package* (find-package :cl-site))
(format t "
This message from cl-site/servers/woo/serve.lisp should report the url & port
that the user may visit to browse the site.
")
;;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
(in-package :cl-user)
(asdf:defsystem cl-site-zaserve
:name "cl-site-zaserve"
:version "0.0.1"
:maintainer "Common-Lisp.Net maintainers"
:author "Dave Cooper & Common-Lisp.Net maintainers"
:licence "TBD"
:description "Test server for common-lisp.net for :zaserve AllegroServe port"
:depends-on (:cl-site :aserve) ;; FLAG -- change to :zaserve when available in QL.
:serial t
:components ((:file "package")
(:file "publish")))