Refactor ".py" files into drush scripts

[komlenic]

Due to inertia and how the predecessor to Drubs worked, additional configuration (beyond what can be accomplished in a drush make file) is currently accomplished in a single python file per node. These files are generally just called "dot P.Y." files in conversation. For the rest of this description, when referring to .py files, we're referring not to generic python scripts, but to the node specific configuration files that Drubs uses.

The types of tasks that are commonly needed in these files are:

• drush commands (particularly module enables/disables, drush scr, drush sql etc)
• linux/shell commands (move files, extract archives, create symlinks, etc.)

These tasks seem quite suited for inclusion in a simple shell script (or scripts). This could accomplish several things:

• remove the need to use/write python to use Drubs (a few simple commands are basically all that is used presently, but it would be better to stick to shell/php)
• remove the need for the current boilerplate code that exists in .py files (using shell scripts would mean that every line in the files was written by the user)
• would solve/simplify Solve escaping issues with difficult py file commands #12

These limitations may all be able to be resolved:

• current usage of .py files keeps everything in python/fabric scope
• using shell/drush scripts may not allow for precise failure handling that is available in fabric/python
• currently several environment variables are referenced (mostly for pathing issues) within .py files (can these be passed somehow from fabric to shell scripts?)

[komlenic]

todo: Update/remove https://github.com/komlenic/drubs/blob/master/docs/update_legacy_projects.md

[komlenic]

Example of how to perform some common actions in a drush script:

drush_invoke('updb');
drush_invoke('rr');
module_enable(array('syslog'), TRUE);
module_disable(array('dblog'), TRUE);
drupal_uninstall_modules(array('dblog'), TRUE);
variable_set('image_toolkit', 'imagemagick');
variable_set('imagemagick_convert', 'convert');

[komlenic]

todo: update or close #37

[komlenic]

Working on this feature and posting some findings:

Creating a new file named something like localdev.drush (file extension is unimportant, though it seems .drush is common), containing this:

#!/usr/bin/env drush
<?php
$action = drush_shift();
$action();

function setup() {
// Tasks to be executed on install AND update go here.
}

function setup_install() {
// Tasks to be executed ONLY after install go here.
}

function setup_update() {
// Tasks to be executed ONLY after update go here.
}

...will execute the function named the same as the first passed argument. This will allow for the division of setup tasks that should be executed on drubs install AND drubs update, as well as only on drubs install, and only on drubs update. This follows the previously used model from .py files.

To call the drush script and run the setup_install tasks, for example:

./localdev.drush setup_install --root=/var/www/html/

How exactly this all works may/will change and be refined.

---

The task/command examples provided previously do work for common tasks, however it appears to be best to call drush_invoke() or drush_invoke_process() instead of drupal functions like module_enable() directly. The main reason for this is that there is no output by default when using drupal functions, however drush commands tend to output "[ok]" or "[success]" or "[error]" etc when executed.

It appears that drush_invoke_process() is preferred over drush_invoke(), particularly because it seems that options (command line arguments starting with "-" or "--") are not able to be passed to drush_invoke(). Will need to confirm this better.

To greatly simplify the syntax needed in the new ".drush" file, when calling drush_invoke_process(), and to make "new" drush script commands essentially follow the same syntax as old .py file commands (with the minor addition of a semicolon to the end, since this is now written in PHP instead of Python), the following function was written:

/**
 * Wrapper for drush_invoke_process() that accepts a simple string command.
 *
 * @param string $full_command
 *   The full drush command to execute, as it would be written on the command
 *   line, without the first word 'drush'.  Example: if on the command line
 *   the command would be 'drush pm-enable mymodule', the $full_command argument
 *   for this function would be 'pm-enable mymodule'.
 */
function drush($full_command) {

  // Parse the supplied full command into pieces.
  $pieces = str_getcsv($full_command, ' ');

  // Get the actual drush command to execute.
  $command = array_shift($pieces);

  // Split/parse remaining pieces into arguments and options.
  $args = array();
  $options = array();
  if (count($pieces) > 0) {
    foreach($pieces as $piece) {
      if(preg_match('/^-/', $piece)) {
        $options[] = $piece;
      }
      else {
        $args[] = $piece;
      }
    }
  }

  drush_invoke_process('@self', $command, $args, $options);
}

With the above function defined, the following all now seem to work exactly as desired:

drush('en module_filter,admin_menu,admin_menu_toolbar,ctools,views,wysiwyg,libraries');
drush('sqlq "TRUNCATE TABLE `filter_format`;"');
// Note the escaped single quotes in SQL queries, otherwise no other special treatement is needed.
drush('sqlq "INSERT INTO `filter_format` (`format`, `name`, `cache`, `status`, `weight`) VALUES
  (\'filtered_html\', \'Filtered HTML\', 1, 1, 1),
  (\'full_html\', \'Full HTML\', 1, 1, 2),
  (\'markdown\', \'Markdown\', 1, 1, 0),
  (\'plain_text\', \'Plain text\', 1, 1, 10);"'
);
drush('vset error_level 2');
drush('pm-list --no-core --status=enabled');

---

Modification of drubs code to support this new file format etc, seems like it would be straightforward and fairly simple: just call drubs_run() or even drush_scr() to execute the setup file (new name for .py file?) with the correct action/function depending on what drubs action (install or update) is being performed. The only breaking change will be the switch from a node-specific parameter named "py_file" in project.yml to one named "setup_file". Accordingly, file templates will need updated/added.

We could even detect whether the setup file is a .py or a .drush file, thus supporting old .py file formats, and this might be nice, however, there are only 3-4 projects live "in the wild" that are known to presently use drubs, and at least a couple of those are likely to be good candidates to try to switch to the new .drush setup file format for testing. This may be unnecessary work to try to support both formats.