several documentation improvements to theming, templating, and extens…

…ion development

doc/contents.rst

@@ -10,7 +10,6 @@ Sphinx documentation contents
    development/index
    man/index
 
-   theming
    templating
    latex
    extdev/index

doc/development/tutorials/builders.rst

@@ -0,0 +1,32 @@
+Configuring builders
+====================
+
+Discover builders by entry point
+--------------------------------
+
+.. versionadded:: 1.6
+
+:term:`Builder` extensions can be discovered by means of `entry points`_ so
+that they do not have to be listed in the :confval:`extensions` configuration
+value.
+
+Builder extensions should define an entry point in the ``sphinx.builders``
+group. The name of the entry point needs to match your builder's
+:attr:`~.Builder.name` attribute, which is the name passed to the
+:option:`sphinx-build -b` option. The entry point value should equal the
+dotted name of the extension module. Here is an example of how an entry point
+for 'mybuilder' can be defined in the extension's ``setup.py``::
+
+    setup(
+        # ...
+        entry_points={
+            'sphinx.builders': [
+                'mybuilder = my.extension.module',
+            ],
+        }
+    )
+
+Note that it is still necessary to register the builder using
+:meth:`~.Sphinx.add_builder` in the extension's :func:`setup` function.
+
+.. _entry points: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins

doc/development/tutorials/index.rst

@@ -1,12 +1,26 @@
+.. _extension-tutorials-index:
+
 Extension tutorials
 ===================
 
 Refer to the following tutorials to get started with extension development.
 
+.. toctree::
+   :caption: General extension tutorials
+
+   overview
+   builders
+
 .. toctree::
    :caption: Directive tutorials
    :maxdepth: 1
 
    helloworld
    todo
    recipe
+
+.. toctree::
+   :caption: Theming
+   :maxdepth: 1
+
+   theming-dev

doc/development/tutorials/overview.rst

@@ -0,0 +1,32 @@
+Developing extensions overview
+==============================
+
+This page contains general information about developing Sphinx extensions.
+
+Make an extension depend on another extension
+---------------------------------------------
+
+Sometimes your extension depends on the functionality of another
+Sphinx extension. Most Sphinx extensions are activated in a
+project's :file:`conf.py` file, but this is not available to you as an
+extension developer.
+
+.. module:: sphinx.application
+    :noindex:
+
+To ensure that another extension is activated as a part of your own extension,
+use the :meth:`Sphinx.setup_extension` method. This will
+activate another extension at run-time, ensuring that you have access to its
+functionality.
+
+For example, the following code activates the "recommonmark" extension:
+
+.. code-block:: python
+
+    def setup(app):
+        app.setup_extension("recommonmark")
+
+.. note::
+
+   Since your extension will depend on another, make sure to include
+   it as a part of your extension's installation requirements.

doc/theming.rst → doc/development/tutorials/theming-dev.rst

@@ -1,7 +1,5 @@
-.. highlight:: python
-
-HTML theming support
-====================
+HTML theme development
+======================
 
 .. versionadded:: 0.6
 
@@ -20,6 +18,11 @@ the theme's look and feel.
 Themes are meant to be project-unaware, so they can be used for different
 projects without change.
 
+.. note::
+
+   See :ref:`dev-extensions` for more information that may
+   be helpful in developing themes.
+
 
 Creating themes
 ---------------
@@ -125,7 +128,7 @@ If your theme package contains two or more themes, please call
 Templating
 ----------
 
-The :doc:`guide to templating <templating>` is helpful if you want to write your
+The :doc:`guide to templating </templating>` is helpful if you want to write your
 own templates.  What is important to keep in mind is the order in which Sphinx
 searches for templates:
 
@@ -138,6 +141,9 @@ name as an explicit directory: ``{% extends "basic/layout.html" %}``.  From a
 user ``templates_path`` template, you can still use the "exclamation mark"
 syntax as described in the templating document.
 
+
+.. _theming-static-templates:
+
 Static templates
 ~~~~~~~~~~~~~~~~
 
@@ -154,6 +160,137 @@ templating to put the color options into the stylesheet.  When a documentation
 is built with the classic theme, the output directory will contain a
 ``_static/classic.css`` file where all template tags have been processed.
 
+
+Use custom page metadata in HTML templates
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Any key / value pairs in :doc:`field lists </usage/restructuredtext/field-lists>`
+that are placed *before* the page's title will be available to the Jinja template when
+building the page within the :data:`meta` attribute. For example, if a page had the
+following text before its first title:
+
+.. code-block:: rst
+
+    :mykey: My value
+
+    My first title
+    --------------
+
+Then it could be accessed within a Jinja template like so:
+
+.. code-block:: jinja
+
+    {%- if meta is mapping %}
+        {{ meta.get("mykey") }}
+    {%- endif %}
+
+Note the check that ``meta`` is a dictionary ("mapping" in Jinja
+terminology) to ensure that using it in this way is valid.
+
+
+Defining custom template functions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Sometimes it is useful to define your own function in Python that you wish to
+then use in a template. For example, if you'd like to insert a template value
+with logic that depends on the user's configuration in the project, or if you'd
+like to include non-trivial checks and provide friendly error messages for
+incorrect configuration in the template.
+
+To define your own template function, you'll need to define two functions
+inside your module:
+
+* A **page context event handler** (or **registration**) function. This is
+  connected to the :class:`.Sphinx` application via an event callback.
+* A **template function** that you will use in your Jinja template.
+
+First, define the registration function, which accepts the arguments for
+:event:`html-page-context`.
+
+Within the registration function, define the template function that you'd like to use
+within Jinja. The template function should return a string or Python objects (lists,
+dictionaries) with strings inside that Jinja uses in the templating process
+
+.. note::
+
+    The template function will have access to all of the variables that
+    are passed to the registration function.
+
+At the end of the registration function, add the template function to the
+Sphinx application's context with ``context['template_func'] = template_func``.
+
+Finally, in your extension's ``setup()`` function, add your registration
+function as a callback for :event:`html-page-context`.
+
+.. code-block:: python
+
+   # The registration function
+    def setup_my_func(app, pagename, templatename, context, doctree):
+        # The template function
+        def my_func(mystring):
+            return "Your string is %s" % mystring
+        # Add it to the page's context
+        context['my_func'] = my_func
+    # Your extension's setup function
+    def setup(app):
+        app.connect("html-page-context", setup_my_func)
+
+Now, you will have access to this function in jinja like so:
+
+.. code-block:: jinja
+
+   <div>
+   {{ my_func("some string") }}
+   </div>
+
+
+Inject javsacript based on user configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If your extension makes use of JavaScript, it can be useful to allow users
+to control its behavior using their Sphinx configuration. However, this can
+be difficult to do if your JavaScript comes in the form of a static library
+(which will not be built with Jinja).
+
+There are two ways to inject variables into the JavaScript space based on user
+configuration.
+
+First, you may append ``_t`` to the end of any static files included with your
+extension. This will cause Sphinx to process these files with the templating
+engine, allowing you to embed variables and control behavior. See
+:ref:`theming-static-templates` for more information.
+
+Second, you may use the :meth:`Sphinx.add_js_file` method without pointing it
+to a file. Normally, this method is used to insert a new JavaScript file
+into your site. However, if you do *not* pass a file path, but instead pass
+a string to the "body" argument, then this text will be inserted as JavaScript
+into your site's head. This allows you to insert variables into your project's
+javascript from Python.
+
+For example, the following code will read in a user-configured value and then
+insert this value as a JavaScript variable, which your extension's JavaScript
+code may use:
+
+.. code-block:: python
+
+    # This function reads in a variable and inserts it into JavaScript
+    def add_js_variable(app):
+        # This is a configuration that you've specified for users in `conf.py`
+        js_variable = app.config['my_javascript_variable']
+        js_text = "var my_variable = '%s';" % js_variable
+        app.add_js_file(None, body=js_text)
+    # We connect this function to the step after the builder is initialized
+    def setup(app):
+        # Tell Sphinx about this configuration variable
+        app.add_config_value('my_javascript_variable')
+        # Run the function after the builder is initialized
+        app.connect('builder-inited', add_js_variable)
+
+As a result, in your theme you can use code that depends on the presence of
+this variable. Users can control the variable's value by defining it in their
+:file:`conf.py` file.
+
+
 .. [1] It is not an executable Python file, as opposed to :file:`conf.py`,
        because that would pose an unnecessary security risk if themes are
        shared.

doc/extdev/index.rst

@@ -3,54 +3,41 @@
 Developing extensions for Sphinx
 ================================
 
-Since many projects will need special features in their documentation, Sphinx is
-designed to be extensible on several levels.
-
-This is what you can do in an extension: First, you can add new
-:term:`builder`\s to support new output formats or actions on the parsed
-documents.  Then, it is possible to register custom reStructuredText roles and
-directives, extending the markup.  And finally, there are so-called "hook
-points" at strategic places throughout the build process, where an extension can
-register a hook and run specialized code.
-
-An extension is simply a Python module.  When an extension is loaded, Sphinx
-imports this module and executes its ``setup()`` function, which in turn
-notifies Sphinx of everything the extension offers -- see the extension tutorial
-for examples.
-
-The configuration file itself can be treated as an extension if it contains a
-``setup()`` function.  All other extensions to load must be listed in the
-:confval:`extensions` configuration value.
-
-Discovery of builders by entry point
-------------------------------------
-
-.. versionadded:: 1.6
-
-:term:`builder` extensions can be discovered by means of `entry points`_ so
-that they do not have to be listed in the :confval:`extensions` configuration
-value.
-
-Builder extensions should define an entry point in the ``sphinx.builders``
-group. The name of the entry point needs to match your builder's
-:attr:`~.Builder.name` attribute, which is the name passed to the
-:option:`sphinx-build -b` option. The entry point value should equal the
-dotted name of the extension module. Here is an example of how an entry point
-for 'mybuilder' can be defined in the extension's ``setup.py``::
-
-    setup(
-        # ...
-        entry_points={
-            'sphinx.builders': [
-                'mybuilder = my.extension.module',
-            ],
-        }
-    )
-
-Note that it is still necessary to register the builder using
-:meth:`~.Sphinx.add_builder` in the extension's :func:`setup` function.
-
-.. _entry points: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins
+Since many projects will need special features in their documentation, Sphinx
+is designed to be extensible on several levels.
+
+Here are a few things you can do in an extension:
+
+* Add new :term:`builder`\s to support new output formats or actions on the
+  parsed documents.
+* Register custom reStructuredText roles and directives, extending the markup
+  using the :doc:`markupapi`.
+* Add custom code to so-called "hook points" at strategic places throughout the
+  build process, allowing you to register a hook and run specialized code.
+  For example, see the :ref:`events`.
+
+An extension is simply a Python module with a ``setup()`` function. A user
+activates the extension by placing the extension's module name
+(or a sub-module) in their :confval:`extensions` configuration value.
+
+When :program:`sphinx-build` is executed, Sphinx will attempt to import each
+module that is listed, and execute ``yourmodule.setup(app)``. This
+function is used to prepare the extension (e.g., by executing Python code),
+linking resources that Sphinx uses in the build process (like CSS or HTML
+files), and notifying Sphinx of everything the extension offers (such
+as directive or role definitions). The ``app`` argument is an instance of
+:class:`.Sphinx` and gives you control over most aspects of the Sphinx build.
+
+.. note::
+
+    The configuration file itself can be treated as an extension if it
+    contains a ``setup()`` function.  All other extensions to load must be
+    listed in the :confval:`extensions` configuration value.
+
+The rest of this page describes some high-level aspects of developing
+extensions and various parts of Sphinx's behavior that you can control.
+For some examples of how extensions can be built and used to control different
+parts of Sphinx, see the :ref:`extension-tutorials-index`.
 
 .. _important-objects:
 
@@ -192,6 +179,11 @@ as metadata of the extension.  Metadata keys currently recognized are:
 APIs used for writing extensions
 --------------------------------
 
+These sections provide a more complete description of the tools at your
+disposal when developing Sphinx extensions. Some are core to Sphinx
+(such as the :doc:`appapi`) while others trigger specific behavior
+(such as the :doc:`i18n`)
+
 .. toctree::
    :maxdepth: 2