diff --git a/README.adoc b/README.adoc index db374e3..98dd38f 100644 --- a/README.adoc +++ b/README.adoc @@ -1,4 +1,6 @@ = LiquiDoc +:toc: preamble + // tag::overview[] LiquiDoc is a documentation build utility for true single-sourcing of technical content and data. It is especially suited for documentation projects with various required output formats, but it is intended for any project with complex, versioned input data for use in docs, user interfaces, and even back-end code. @@ -877,8 +879,12 @@ See <>. === Deploy Operations -It's not clear how deeply we will delve into deploy operations, since other build systems (such as rake) would seem far more suitable. -For testing purposes, however, spinning up a local webserver with the same stroke that you build a site is pretty rewarding and time saving. +Mainstream deployment platforms are probebly better suited to tying all your operations together, but we plan to bake a few common operations in to help you get started. +For true build-and-deployment control, consider build tools such as Make, Rake, and Gradle, or deployment tools like Travis CI, CircleCI, and Jenkins. + +==== Jekyll Serve + +For testing purposes, however, spinning up a local webserver with the same stroke that you build a site is pretty rewarding and time saving, so we'll start there. For now, this functionality is limited to adding a `--deploy` flag to your `liquidoc` command. This will attempt to serve files from the *destination* set for the associated Jekyll build. @@ -886,6 +892,70 @@ This will attempt to serve files from the *destination* set for the associated J [WARNING] Deployment of Jekyll sites is both limited and untested under nonstandard conditions. +==== Algolia Search Indexing for Jekyll + +If you're using Jekyll to build sites, LiquiDoc makes indexing your files with the Algolia cloud search service a matter of configuration. +The heavy lifting is performed by the jekyll-algolia plugin, but LiquiDoc can handle indexing even a complex site by using the same configuration that built your HTML content (which is what Algolia actually indexes). + +[NOTE] +You will need a free community (or premium) link:https://www.algolia.com/users/sign_up/hacker[Algolia account] to take advantage of Algolia's indexing service and REST API. +Simply create a named index, then visit the API Keys to collect the rest of the info you'll need to get going. + +Two hard-coding steps are required to prep your source to handle Algolia index pushes. + +. Add a block to your main Jekyll configuration file. ++ +.Example Jekyll Algolia configuration +[source,yaml] +---- +algolia: + application_id: 'your-application-id' # <1> + search_only_api_key: 'your-search-only-api-key' # <2> + extensions_to_index: [adoc] # <3> +---- ++ +<1> From the top bar of your Algolia interface. +<2> From the API Keys screen of your Algolia interface. +<3> List as many extensions as apply, separated by commas. + +. Add a block to your build config. ++ +[source,yaml] +---- + - action: render + data: globals.yml + builds: + - backend: jekyll + properties: + files: + - _configs/jekyll-global.yml + - _configs/jekyll-portal-1.yml + arguments: + destination: build/site/user-basic + attributes: + portal_term: Guide + search: + index: 'portal-1' +---- ++ +The `index:` parameter is for the name of the index you are pushing to. +(An Algolia “app” can have multiple “indices”.) +If you have + +Now you can call your same LiquiDoc build command with the `--search-index-push` or `--search-index-dry` flags along with the `--search-api-key='your-admin-api-key-here'` argument in order to invoke the indexing operation. +The `--search-index-dry` flag merely tests content packaging, whereas `--search-index-push` connects to the Algolia REST API and attempt to push your content for indexing and storage. + +.Example Jekyll Algolia deployment +[source,shell] +---- +bundle exec liquidoc -c _configs/build-docs.yml --search-index-push --search-index-api-key='90f556qaa456abh6j3w7e8c10t48c2i57' +---- + +This operation performs a complete build, including each render operation, before the Algolia plugin processes content and pushes each build to the indexing service, in turn. + +[TIP] +To add modern site search for your users, add link:https://community.algolia.com/instantsearch.js/[Algolia's InstantSearch functionality] to your front end! + === Config Settings Matrix Here is a table of all the established configuration settings, as they pertain to each key LiquiDoc action. @@ -992,6 +1062,12 @@ s| properties | N/A | Optional | + +s| search +| N/A +| N/A +| Optional +| |=== pass:[*]The `output` setting is considered optional for render operations because static site generations target a directory set in the SSG's config file. diff --git a/lib/liquidoc.rb b/lib/liquidoc.rb index b5f06a5..d401397 100755 --- a/lib/liquidoc.rb +++ b/lib/liquidoc.rb @@ -1,7 +1,7 @@ require 'liquidoc' +require 'optparse' require 'yaml' require 'json' -require 'optparse' require 'liquid' require 'asciidoctor' require 'asciidoctor-pdf' @@ -48,6 +48,8 @@ @verbose = false @quiet = false @explicit = false +@search_index = false +@search_index_dry = '' # Instantiate the main Logger object, which is always running @logger = Logger.new(STDOUT) @@ -276,7 +278,7 @@ def message text = ". #{stage}Draws data from `#{self.data[0]}`" end else - text = ". #{stage}Draws data from `#{self.data['file']}`" + text = ". #{stage}Draws data from `#{self.data}`" end text.concat("#{reason},") if reason text.concat(" and parses it as follows:") @@ -422,6 +424,18 @@ def prop_files_array # props['files'].force_array if props['files'] # end + def search + props['search'] + end + + def add_search_prop! prop + begin + self.search.merge!prop + rescue + raise "PropertyInsertionError" + end + end + # NOTE this section repeats in Class.AsciiDocument def attributes @build['attributes'] @@ -586,11 +600,7 @@ def get_data datasrc # Pull in a semi-structured data file, converting contents to a Ruby hash def ingest_data datasrc -# Must be passed a proper data object (there must be a better way to validate arg datatypes) - unless datasrc.is_a? Object - raise "InvalidDataObject" - end - # This proc should really begin here, once the datasrc object is in order + raise "InvalidDataObject" unless datasrc.is_a? Object case datasrc.type when "yml" begin @@ -654,7 +664,7 @@ def parse_regex data_file, pattern end end end - output = {"data" => records} + output = records rescue Exception => ex @logger.error "Something went wrong trying to parse the free-form file. #{ex.class} thrown. #{ex.message}" raise "Freeform parse error" @@ -665,8 +675,10 @@ def parse_regex data_file, pattern # Parse given data using given template, generating given output def liquify datasrc, template_file, output, variables=nil input = get_data(datasrc) - nested = { "data" => get_data(datasrc)} - input.merge!nested + unless input['data'] + nested = { "data" => input.dup } + input.merge!nested + end validate_file_input(template_file, "template") if variables vars = { "vars" => variables } @@ -878,8 +890,8 @@ def generate_site doc, build when "jekyll" attrs = doc.attributes build.add_config_file("_config.yml") unless build.prop_files_array - jekyll_config = YAML.load_file(build.prop_files_array[0]) # load the first Jekyll config file locally - attrs.merge! ({"base_dir" => jekyll_config['source']}) # Sets default Asciidoctor base_dir to == Jekyll root + jekyll = load_jekyll_data(build) # load the first Jekyll config file locally + attrs.merge! ({"base_dir" => jekyll['source']}) # Sets default Asciidoctor base_dir to == Jekyll root # write all AsciiDoc attributes to a config file for Jekyll to ingest attrs.merge!(build.attributes) if build.attributes attrs = {"asciidoctor" => {"attributes" => attrs} } @@ -892,14 +904,31 @@ def generate_site doc, build if build.props['arguments'] opts_args = build.props['arguments'].to_opts_args end - command = "bundle exec jekyll build --config #{config_list} #{opts_args} #{quiet}" + base_args = "--config #{config_list} #{opts_args}" + command = "bundle exec jekyll build #{base_args} #{quiet}" + if @search_index + # TODO enable config-based admin api key ingest once config is dynamic + command = algolia_index_cmd(build, @search_api_key, base_args) + @logger.warn "Search indexing failed." unless command + end + end + if command + @logger.info "Running #{command}" + @logger.debug "AsciiDoc attributes: #{doc.attributes.to_yaml} " + system command end - @logger.info "Running #{command}" - @logger.debug "AsciiDoc attributes: #{doc.attributes.to_yaml} " - system command jekyll_serve(build) if @jekyll_serve end +def load_jekyll_data build + data = {} + build.prop_files_array.each do |file| + settings = YAML.load_file(file) + data.merge!settings if settings + end + return data +end + # === # DEPLOY procs # === @@ -914,6 +943,20 @@ def jekyll_serve build system command end +def algolia_index_cmd build, apikey=nil, args + unless build.search and build.search['index'] + @logger.warn "No index configuration found for build; jekyll-algolia operation skipped for this build." + return false + else + unless apikey + @logger.warn "No Algolia admin API key passed; skipping jekyll-algolia operation for this build." + return false + else + return "ALGOLIA_INDEX_NAME='#{build.search['index']}' ALGOLIA_API_KEY='#{apikey}' bundle exec jekyll algolia #{@search_index_dry} #{args} " + end + end +end + # === # Text manipulation Classes, Modules, procs, etc # === @@ -1081,7 +1124,7 @@ def regexreplace input, regex, replacement='' @quiet = true end - opts.on("--explicit", "Log explicit step descriptions to console as build progresses. (Otherwise writes to file at #{@build_dir}/pre/config-explainer.adoc .)") do |n| + opts.on("--explain", "Log explicit step descriptions to console as build progresses. (Otherwise writes to file at #{@build_dir}/pre/config-explainer.adoc .)") do |n| explainer_init("STDOUT") @explainer.level = Logger::INFO @logger.level = Logger::WARN # Suppress all those INFO-level messages @@ -1096,6 +1139,19 @@ def regexreplace input, regex, replacement='' @jekyll_serve = true end + opts.on("--search-index-push", "Runs any search indexing configured in the build step and pushes to Algolia.") do + @search_index = true + end + + opts.on("--search-index-dry", "Runs any search indexing configured in the build step but does NOT push to Algolia.") do + @search_index = true + @search_index_dry = "--dry-run" + end + + opts.on("--search-api-key=STRING", "Passes Algolia Admin API key (which you should keep out of Git).") do |n| + @search_api_key = n + end + opts.on("-h", "--help", "Returns help.") do puts opts exit