1. 19 Feb, 2019 1 commit
    • Didier Verna's avatar
      Make parallel code a separate system. · fd3f25f3
      Didier Verna authored
      2019-02-19  Didier Verna  <didier@didierverna.net>
      
      	* parallel/parallel.lisp: Move here from ../core/src/.
      	* parallel/fr.epita.lrde.quickref.parallel.asd: New file.
      	* setup/setup.lisp (restrict-because):
      	(setup-parallel): New functions.
      	(*configuration*): Update docstring.
      	(:fr.epita.lrde.quickref.setup): Export SETUP-PARALLEL.
      	* setup/fr.epita.lrde.quickref.setup.asd:
      	(:fr.epita.lrde.quickref.setup/parallel): New system.
      	* core/src/util.lisp: Add one comment on top of the configuration
      	variables related to parallelism.
      	* core/src/quickref.lisp (build): Conditionalize parallel
      	execution on the availability of parallel support.
      	* core/fr.epita.lrde.quickref.core.asd
      	(:fr.epita.lrde.quickref.core): Remove dependency on
      	src/parallel.lisp.
      	* fr.epita.lrde.quickref.asd (:fr.epita.lrde.quickref): Load the
      	setup/parallel system.
      fd3f25f3
  2. 18 Feb, 2019 10 commits
    • Didier Verna's avatar
      Update copyright years. · 387462e0
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp:
      	* core/src/texinfo.lisp:
      	* core/src/quickref.lisp:
      	* core/src/parallel.lisp:
      	* core/src/html.lisp:
      	* core/meta.lisp:
      	* core/fr.epita.lrde.quickref.core.asd:
      	* bin/declt: Add 2019 to the copyright years.
      387462e0
    • Didier Verna's avatar
      Comment out two functions not used anymore. · 037c7ad3
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/texinfo.lisp: Comment out BUILD-TEXINFO-MANUAL.
      	* core/src/html.lisp: Comment out BUILD-HTML-MANUAL.
      037c7ad3
    • Didier Verna's avatar
      README update. · 99e1cf8e
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* README.md: Rephrase advertisement for parallelism (which is not
      	rudimentary anymore).
      99e1cf8e
    • Didier Verna's avatar
      Improve one docstring. · e4d3e3f6
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (*empty-cache*): Improve docstring.
      e4d3e3f6
    • Didier Verna's avatar
      Parametrize the number of task-specific threads in parallel solutions. · 9836810e
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (*declt-threads*):
      	(*makeinfo-threads*): New parameters.
      	* core/meta.lisp (:fr.epita.lrde.quickref): Export them.
      	* core/src/parallel.lisp (build-html-manuals-//-2):
      	(build-texinfo-manuals-//-2):
      	(build-texinfo-manuals-//-3): Rename THREAD parameter to
      	THREAD-NUMBER.
      	(build-manuals-//-2):
      	(build-manuals-//-3): Rename TEXINFO-THREADS to DECLT-THREADS.
      	Rename HTML-THREADS to MAKEINFO-THREADS. Accept new DECLT-THREADS
      	and MAKEINFO-THREADS arguments.
      	* core/src/quickref.lisp (build): Accept and propagate new
      	:DECLT-THREADS and :MAKEINFO-THREADS arguments.
      9836810e
    • Didier Verna's avatar
      Synchronize I/O in WITH-PROGRESSION. · fdad1092
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (with-progression): Finish outputing up to
      	the ellision before executing the body.
      fdad1092
    • Didier Verna's avatar
      Improve reporting in parallel solutions. · fa2120fe
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-manuals-//-2):
      	(build-manuals-//-3): Pass the thread number as an argument to the
      	thread function.
      	(build-html-manuals-//-2):
      	(build-texinfo-manuals-//-2):
      	(build-texinfo-manuals-//-3): Update accordingly. Use it in
      	reporting, along with the number of successfully built manuals.
      fa2120fe
    • Didier Verna's avatar
      Change default number of threads in parallel solution 2. · f92de93d
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-manuals-//-2): Change the default
      	number of threads.
      f92de93d
    • Didier Verna's avatar
      *caching-behavior* -> *cache-policy*. · df3ab91a
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (*caching-behavior*): Rename to
      	*cache-policy*.
      	(cache-directory):
      	(ensure-quickref-directory):
      	* core/src/texinfo.lisp (run-declt):
      	* core/src/quickref.lisp (build):
      	(build):
      	* core/src/parallel.lisp (build-manuals-//-2):
      	(build-manuals-//-3):
      	* core/meta.lisp (:fr.epita.lrde.quickref):
      	* bin/declt: Update accordingly.
      df3ab91a
    • Didier Verna's avatar
      Small reorganization. · 8b147b27
      Didier Verna authored
      2019-02-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp: Move some code around.
      8b147b27
  3. 13 Feb, 2019 2 commits
    • Didier Verna's avatar
      Reporting improvements. · c783ef31
      Didier Verna authored
      This patch essentially removes the clumsy :silent option from various
      functions. There is now a clean distinction between low level functions (not
      printing messages) used both by the sequential and parallel code, and the
      higher level wrappers than perform reporting.
      
      As a side effect, some former functions are almost ready to become new entry
      points to the library.
      
      2019-02-13  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (with-progression): New macro.
      	(library-name): Move to parallel.lisp.
      	* core/src/texinfo.lisp (run-declt): Rename from DECLT.
      	(build-texinfo-manual): Remove SILENT option. Use
      	WITH-PROGRESSION. Not used anymore (future new entry point).
      	* core/src/html.lisp (run-makeinfo): Rename from MAKEINFO.
      	(build-html-manual): Remove SILENT option. Use WITH-PROGRESSION.
      	Not used anymore (future new entry point).
      	* core/src/quickref.lisp (build-manual): Use RUN-DECLT,
      	RUN-MAKEINFO, and INSERT-README-CONTENTS wrapped in
      	WITH-PROGRESSION directly.
      	(build): Small format string update.
      	* core/src/parallel.lisp (library-name): Move from util.lisp.
      	(build-html-manuals-//-1):
      	(build-texinfo-manuals-//-1):
      	(build-manuals-//-1):
      	(build-html-manuals-//-2):
      	(build-texinfo-manuals-//-2):
      	(build-texinfo-manuals-//-3): Use RUN-DECLT, RUN-MAKEINFO, and
      	INSERT-README-CONTENTS directly.
      c783ef31
    • Didier Verna's avatar
      Small code reorganization. · 8275431f
      Didier Verna authored
      2019-02-13  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp: Reorder code sections.
      8275431f
  4. 12 Feb, 2019 3 commits
    • Didier Verna's avatar
      Move format-// to parallel.lisp. · af8d606f
      Didier Verna authored
      2019-02-12  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (format-//): Move from here...
      	* core/src/parallel.lisp (format-//): ...to here.
      af8d606f
    • Didier Verna's avatar
      Comment update. · e5da3c09
      Didier Verna authored
      2019-02-12  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp: Remove obsolete header comment.
      e5da3c09
    • Didier Verna's avatar
      Small typo fix. · 9f9bc17d
      Didier Verna authored
      2019-02-12  Didier Verna  <didier@didierverna.net>
      
      	* bin/declt: Fix one type in a comment.
      9f9bc17d
  5. 24 Jan, 2019 3 commits
    • Didier Verna's avatar
      Fix 3rd parallel implementation. · f3c22be1
      Didier Verna authored
      We cannot rely on the queue being empty to push a new batch, because some
      Texinfo threads might still be running. So instead, we use a counter,
      decremented /after/ processing is finished. The counter reaching 0 is the
      actual signal to start processing the next batch of libraries.
      
      2019-01-24  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (*remaining-libraries-count*): New
      	variable.
      	(build-texinfo-manuals-//-3):
      	(build-manuals-//-3): Handle it.
      f3c22be1
    • Didier Verna's avatar
      First implementation of parallel version 3. · de654b58
      Didier Verna authored
      2019-01-24  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (while): New macro.
      	* core/src/parallel.lisp (build-texinfo-manuals-//-3):
      	* core/src/parallel.lisp (build-manuals-//-3): New functions.
      	* core/src/quickref.lisp (build): Handle parallel version 3.
      de654b58
    • Didier Verna's avatar
      Various small code improvements. · 1eeca57b
      Didier Verna authored
      2019-01-24  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-html-manuals-//-1):
      	(build-manuals-//-1):
      	(build-manuals-//-1):
      	(build-texinfo-manuals-//-2):
      	(build-manuals-//-2): Improve some format strings. Use &aux.
      1eeca57b
  6. 23 Jan, 2019 4 commits
    • Didier Verna's avatar
      Benefit from the dependency graph in the first parallel solution. · f268a428
      Didier Verna authored
      2019-01-23  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-manuals-//-1): Sort libraries by
      	dependency order before processing.
      f268a428
    • Didier Verna's avatar
      Distinguish the processing of non-quicklisp systems. · fa12c143
      Didier Verna authored
      It doesn't hurt much to build them first and sequentially because there's
      only a couple of them, and this is required to be able to process a
      dependency graph.
      
      2019-01-23  Didier Verna  <didier@didierverna.net>
      
      	* core/src/quickref.lisp (build): Always build the contents of
      	*OTHER-SYSTEMS-NAMES* first, and sequentially.
      fa12c143
    • Didier Verna's avatar
      Simplify the dependency graph implementation. · 31f04fbc
      Didier Verna authored
      2019-01-23  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (dgnode): Remove structure. Store
      	dependency lists directly as the hash table values.
      	(cull-nodes):
      	(ordered-primary-systems): Update accordingly.
      31f04fbc
    • Didier Verna's avatar
      First implementation of the dependency graph. · f81e0b67
      Didier Verna authored
      2019-01-23  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (dgnode): New structure.
      	(cull-nodes):
      	(ordered-primary-systems): New functions.
      f81e0b67
  7. 16 Jan, 2019 1 commit
    • Didier Verna's avatar
      Fix parallel code wrt user options. · cfae074c
      Didier Verna authored
      Because threads don't see dynamic bindings of special variables,
      we need to recreate them in every threaded function.
      
      2019-01-16  Didier Verna  <didier@didierverna.net>
      
      	* core/src/quickref.lisp (build): Reorganize and remove one
      	spurious special variable binding.
      	* core/src/parallel.lisp (build-manuals-//-1):
      	(build-manuals-//-2): Pass user options as thread function
      	arguments, so that they can in turn recreate dynamic bindings.
      cfae074c
  8. 14 Jan, 2019 2 commits
    • Didier Verna's avatar
      Minor typo fix. · 2d940345
      Didier Verna authored
      2019-01-14  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-html-manuals-//-2): Fix one typo
      	in a comment.
      2d940345
    • Didier Verna's avatar
      Fix bug in second parallel version. · b4d1f3a3
      Didier Verna authored
      We need to awaken /all/ waiting threads for them to see the :done marker.
      
      2019-01-14  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-manuals-//-2): Use
      	condition-broadcast instead of condition-notify after pushing
      	:done.
      b4d1f3a3
  9. 19 Dec, 2018 6 commits
    • Didier Verna's avatar
      Code improvements. · 261f1404
      Didier Verna authored
      2018-12-19  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-html-manuals-//-1): Improve LOOP
      	code.
      261f1404
    • Didier Verna's avatar
      Implement second parallel algorithm. · e41f513b
      Didier Verna authored
      2018-12-19  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (*library-queue*): New variable.
      	(build-html-manuals-//-2):
      	(build-texinfo-manuals-//-2):
      	(build-manuals-//-2): New functions.
      	* core/src/quickref.lisp (build): Handle paralell option #2.
      e41f513b
    • Didier Verna's avatar
      Print the current number of Texinfo files processed in parallel solution #1. · 89a9beda
      Didier Verna authored
      2018-12-19  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (build-manuals-//-1): Pass the total
      	number of libraries to BUILD-HTML-MANUALS-//-1.
      	(build-html-manuals-//-1): Update accordingly. Print it.
      89a9beda
    • Didier Verna's avatar
      Make current parallel solution #1. · b577b2cd
      Didier Verna authored
      2018-12-19  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (*parallel*): Update docstring.
      	* core/src/parallel.lisp (build-html-manuals-//):
      	(build-texinfo-manuals-//):
      	(build-manuals-//): Rename by appending -1. Update callers.
      	* core/src/quickref.lisp (build): Update accordingly.
      b577b2cd
    • Didier Verna's avatar
      Naming improvements in parallel code. · 65b47b33
      Didier Verna authored
      2018-12-19  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp (*texinfo-queue*): Rename from
      	*libraries-queue*.
      	(build-manuals-//): Update accordingly. Rename mutex to
      	texinfo-queue and new-manual to new-texinfo.
      	(build-texinfo-manuals-//):
      	(build-html-manuals-//): Update accordingly.
      65b47b33
    • Didier Verna's avatar
      Move parallel code in its own file. · 699aa1e4
      Didier Verna authored
      2018-12-19  Didier Verna  <didier@didierverna.net>
      
      	* core/src/parallel.lisp: New file.
      	* core/fr.epita.lrde.quickref.core.asd
      	(:fr.epita.lrde.quickref.core): Add it.
      	* core/src/texinfo.lisp:
      	* core/src/html.lisp:
      	* core/src/quickref.lisp:  Move parallel code there.
      699aa1e4
  10. 18 Dec, 2018 1 commit
    • Didier Verna's avatar
      Add the empty-cache option. · 68245e35
      Didier Verna authored
      2018-12-18  Didier Verna  <didier@didierverna.net>
      
      	* core/src/util.lisp (*empty-cache*): New parameter.
      	(ensure-quickref-directory): Handle it.
      	* core/meta.lisp (:fr.epita.lrde.quickref): Export it.
      	* core/src/quickref.lisp (build): Handle it.
      68245e35
  11. 13 Dec, 2018 1 commit
  12. 12 Dec, 2018 1 commit
    • Didier Verna's avatar
      Fix one missing export. · a12f5f6e
      Didier Verna authored
      2018-12-08  Didier Verna  <didier@didierverna.net>
      
      	* core/meta.lisp (:fr.epita.lrde.quickref): Export
      	*CROSS-REFERENCE-VALIDATION*.
      a12f5f6e
  13. 11 Dec, 2018 3 commits
  14. 08 Dec, 2018 1 commit
  15. 05 Dec, 2018 1 commit