Commit 809db4fe authored by Erik Huelsmann's avatar Erik Huelsmann
Browse files

Add processing of Markdown input files

Note that the 'markdown.cl' file doesn't support the extensions
required to generate the content section.

Note also that the 'cl-markdown' library doesn't actually generate
the content section we want. However, with a ton of tweaks, it
likely will.
Or, if it won't, we can create our own extension to cl-markdown.
Or, if that's too much work, we can do the same thing as currently
happens in the newsbox processing: add our own processing of the
HTML output...
parent db5965bc
......@@ -8,7 +8,7 @@
:author "C. Yang & Common-Lisp.Net maintainers"
:licence "TBD"
:description "Static site generator for common-lisp.net, written in CL"
:depends-on (:cl-mustache :plump :markdown.cl :closer-mop :cl-who :cl-yaml)
:depends-on (:cl-mustache :plump :cl-markdown :closer-mop :cl-who :cl-yaml)
:components ((:module source
:pathname ""
:serial t
......
......@@ -2,7 +2,7 @@
<img src="http://i.imgur.com/jLVXhpc.png">
</div>
# Awesome Common Lisp [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) [![Assertible status](https://assertible.com/apis/102e334d-f9a8-4565-9353-7572de775cae/status?api_token=8b55a286830323effb)](https://assertible.com/docs/guide/deployments)
<!-- Awesome Common Lisp [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) [![Assertible status](https://assertible.com/apis/102e334d-f9a8-4565-9353-7572de775cae/status?api_token=8b55a286830323effb)](https://assertible.com/docs/guide/deployments) -->
A curated list of _awesome_ Common Lisp libraries.
......@@ -20,95 +20,7 @@ is provided in the LICENSE file. This repository is also mirrored on
alternative to Github. Preference is given to [free software][13] and
sellers who aren't evil for physical resources.
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-generate-toc again -->
**Table of Contents**
- [Build Systems](#build-systems)
- [Compilers, code generators](#compilers-code-generators)
- [C, C++](#c-c)
- [Javascript](#javascript)
- [Crypto](#crypto)
- [Cryptocurrencies](#cryptocurrencies)
- [Database](#database)
- [Data Structures](#data-structures)
- [Foreign Function Interface](#foreign-function-interface)
- [C](#c)
- [Java](#java)
- [Miscellaneous](#miscellaneous)
- [Objective-C](#objective-c)
- [Python](#python)
- [Game Development](#game-development)
- [Graphics](#graphics)
- [GUI](#gui)
- [JSON](#json)
- [Language extensions](#language-extensions)
- [CLOS extensions](#clos-extensions)
- [Non-deterministic programming](#non-deterministic-programming)
- [Learning and Tutorials](#learning-and-tutorials)
- [Online](#online)
- [Beginner](#beginner)
- [Intermediate](#intermediate)
- [Advanced](#advanced)
- [Reference](#reference)
- [Offline](#offline)
- [Beginner](#beginner)
- [Intermediate](#intermediate)
- [Advanced](#advanced)
- [Library Manager](#library-manager)
- [Interfaces to other package managers](#interfaces-to-other-package-managers)
- [Machine Learning](#machine-learning)
- [Natural Language Processing](#natural-language-processing)
- [Network and Internet](#network-and-internet)
- [HTTP clients](#http-clients)
- [HTTP Servers](#http-servers)
- [Clack plugins](#clack-plugins)
- [Web frameworks](#web-frameworks)
- [Assets management](#assets-management)
- [Parsing html](#parsing-html)
- [Querying HTML/DOM](#querying-htmldom)
- [HTML generators and templates](#html-generators-and-templates)
- [URI handling](#uri-handling)
- [Javascript](#javascript)
- [Websockets](#websockets)
- [Static site generators](#static-site-generators)
- [Third-party APIs](#third-party-apis)
- [Others](#others)
- [Numerical and Scientific](#numerical-and-scientific)
- [Parallelism and Concurrency](#parallelism-and-concurrency)
- [Regex](#regex)
- [Scripting](#scripting)
- [Text Editor Resources](#text-editor-resources)
- [Emacs](#emacs)
- [Vim](#vim)
- [Lem](#lem)
- [Atom](#atom)
- [Sublime Text](#sublime-text)
- [Notebooks](#notebooks)
- [REPLs](#repls)
- [Text Processing](#text-processing)
- [Tools](#tools)
- [Unit Testing](#unit-testing)
- [Utilities](#utilities)
- [Caching](#caching)
- [Configuration](#configuration)
- [CSV](#csv)
- [Date and time](#date-and-time)
- [Data validation](#data-validation)
- [Documentation builders](#documentation-builders)
- [Files and directories](#files-and-directories)
- [Git](#git)
- [i18n](#i18n)
- [Linting](#linting)
- [Logging](#logging)
- [Markdown](#markdown)
- [Plotting](#plotting)
- [Text Processing](#text-processing)
- [Other](#other)
- [XML](#xml)
- [Contributing](#contributing)
<!-- markdown-toc end -->
{table-of-contents :depth 2 :start 1 :label "Table of contents" }
Build Systems
=============
......@@ -746,7 +658,7 @@ Configuration
* [py-configparser](https://common-lisp.net/project/py-configparser/) - reads and writes Python's ConfigParser-like configuration files. [MIT][200].
CSV
---
----
* [cl-csv](https://github.com/AccelerationNet/cl-csv) - A library for parsing CSV files. [3-clause BSD][15].
......@@ -788,7 +700,7 @@ Files and directories
* [archive](https://github.com/froydnj/archive) - a library for reading and creating archive (tar, cpio) files. [BSD_3Clause][15]. A pure Common Lisp replacement for the 'tar' program.
Git
---
----
* [legit](https://shinmera.github.io/legit/) - an interface to the Git
binary. [Artistic License 2.0][51].
......@@ -865,9 +777,6 @@ XML
* [xmls](http://quickdocs.org/xmls/) - A small, simple, non-validating XML parser. [3-clause BSD][15].
* [cl-feedparser](https://github.com/TBRSS/cl-feedparser) - A Common Lisp (RSS, Atom) feed parser. [LLGPL][8]
Contributing
============
Your contributions are always welcome! Please submit a pull request or create
an issue to add a new framework, library or software to the list.
......
......@@ -27,6 +27,7 @@
;;
(append
(directory (make-pathname :defaults *PAGES-DIR* :name :wild :type "html"))
(directory (make-pathname :defaults *PAGES-DIR* :name :wild :type "md"))
(directory (make-pathname :defaults *PAGES-DIR* :name :wild :type "lisp")))))
......
;;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
;;;; -*-Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*-
(in-package :cl-site)
......@@ -51,7 +51,7 @@
(split-content-from-header page-path)
(let* ((header-text (format nil "~{~A~%~}" header))
(content-text (format nil "~{~A~%~}" content))
(parsed-header (yaml:parse header-text))
(parsed-header (when header (yaml:parse header-text)))
(page-context
(when parsed-header
(loop for k being each hash-key of parsed-header
......@@ -64,28 +64,38 @@
(append (acons :page-content (render-template* content-text) page-context) *GLOBAL-CONTEXT*)
output-stream))))))
(defun preprocess-md-pages (pages)
(format t "Preprocessing Markdown pages..~%")
(dolist (page pages)
(let* ((md-file (make-path *pages-dir* page t)))
(when (string= (pathname-type md-file) "md")
(let ((html-file (make-pathname :defaults md-file :type "html")))
(format t "Transforming ~A to ~A~%" md-file html-file)
(multiple-value-bind
(header content)
(split-content-from-header md-file)
(let ((md-content (with-output-to-string (out) (cl-markdown:markdown (format nil "~{~A~%~}" content) :stream out))))
(with-open-file (f html-file
:direction :output
:if-exists :supersede)
(when header
(format f "~{~A~%~}---~%" header))
(format f "~A~%" md-content)))))))))
(defun process-pages (pages)
(format t "Processing pages...~%")
(preprocess-md-pages pages)
(preprocess-lisp-pages pages)
(generate-news)
(format t "Processing pages..~%")
(let ((template-path (pathname (merge-pathnames *templates-dir* *DEFAULT-PAGE-TEMPLATE*))))
(ensure-directories-exist *OUTPUT-DIR*)
(setq pages (populate-pages)) ;; to pick up any newly generated .html files.
(dolist (page pages)
(let ((page-pathname (merge-pathnames (page-content page) *pages-dir*)))
(when (string-equal (pathname-type page-pathname) "html")
(format t "Rendering ~A~%" (cdr (assoc :content page)))
(render-page page template-path)
(when (probe-file (make-pathname :defaults page-pathname :type "lisp"))
(format t "Deleting temporary generated ~a~%" (file-namestring page-pathname))
(delete-file page-pathname)
))))
(loop for page in pages
when (string= (pathname-type (cdr (assoc :content page))) "html")
do
(format t "Rendering ~A~%" (cdr (assoc :content page)))
(render-page page template-path))
(format t "Done.~%")))
......@@ -119,7 +129,7 @@
;;
(defun generate-news ()
(let ((html
(markdown.cl:parse-file (merge-pathnames "news.md" *PAGES-DIR*))))
(with-output-to-string (out) (cl-markdown:markdown (merge-pathnames "news.md" *PAGES-DIR*) :stream out))))
(with-open-file (f (merge-pathnames "news.html" *PAGES-DIR*)
:direction :output
:if-does-not-exist :create
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment