[4.0][GSoC 2018] Improve Override Management (joomla#21851)

* Load correct core files of override files (#2)

Start implements loadcorefile() in administrator/components/com_templates/Model/TemplateModel.php

* CS (#3) Coding Standards

* codingstandards

* codingstandards (#4)

* Test (#6)

Phase 2 (2 part) Mechanism to find correct core file and implementation.

* Remove Notice: Only available for html-folder

* Remove Warning if core file not found (#11)

Thanks.
So one part of the issue joomla-projects/gsoc18_override_management#12 is done.

* Implement the diff view in template manager 

Implement the diff view in template manager

* coding standard (joomla#17)

* fix diff (joomla#18) Fix bug in path in case of administrator template override.

Fix bug in path in case of administrator template override.

* Notification after update and TEST (joomla#16)

Find changed files of overridden files and show message.

* coding standard (joomla#21)

* correction

* correction (joomla#26)

* Correcthtmlpath (joomla#27)

* correction

* change oldhtml to newhtml

* List of updated override files. (joomla#30)

* addcss (joomla#34)

* Final Product  (joomla#39)

Core and Diff view
Updated override history list.
Quick icon notification plugin.
Override control plugin.

* save 3 lines :)

* New feature show status. (joomla#47)

show status in com_template view templates

* link

* corrected namespace

* Button to Switch (joomla#35)

* wip add Switcher

* wip style switcher

* wip style switch make inline and change on off text

* wip start with js

* wip js

* wip delete buttons and make js more robust

* wip save to storage

* wip delete old code

* wip

* wip lint

* wip css

* set default value for switcher

* wip make switcher blue

* wip

* wip

* build

* correct names

* create new functions

* fist test code

* use onchange

* undo installer.min.js

* add forgotten new line at the end of css file

* correct align

* correct compare.es6 - only deleted the toggle part

* correct compare.js - only deleted the toggle part

* wip

* reduce timeout

* wrap in funcitons

* wip

* add use strict to both js-files(compare and toggle)

* add the timeout value of 500 again, because 200 are not enought in my case

* use css class 'active' for toggle views

* add strict

* time out for editor

* wip

* improvments use newActive and switch

* correction

* width of switcher-spans

* correct align

* do not use global

* wip

* removed timeouts

* JTEXT to TEXT

* forgotton last line

* deleted duplicated comments

* css fix align

* use unnamed functions in es6

* Sql files for fix database (joomla#50)

* sql files for database fix

* delete space

* Suggestion for displaying Dates in view updates files (joomla#52)

Correct Dates and do not use date of file any more

* Store Date as UTC and show it in server time zone (joomla#57)

* modified and created date are created and stored in UTC

* convert dates for displaying in model

* spar a loop

* normalize timezone in view

* use language constants for dateformat

* JToolbarHelper to ToolbarHelper

* CS

* namespace

* plural

* name

* clean

* text

* fx

* sin

* files

* s

* Suggestion for language strings (joomla#60)

* language strings

* correct typo

* delete media folder plg_quickicon

* add folder plg_quickicon to build/media_src

* delete files in media folder

* Move media folder - System (joomla#66)

* multi

* cs

* delete files in media folder for joomla toolbar (joomla#67)

* Fix button switchers style. (joomla#70)

* button

* CS

* changed uitab.addTab for updated files

* Bring back core.js changes. (joomla#69)

* core.js

* const

* fix

* form

* core

* hound

* CS

* scopr

* grid

* alpha

* cs

* lang

* only override file

* lang

* override lang installer

* Cs

* sub

* Update list of core extensions (joomla#71)

* Language changes (joomla#76)

* update

* Update en-GB.com_templates.ini

* override JLIB_HTML_PUBLISH_ITEM

this is the hover text on the publish icon in the list of files

* Change icon (joomla#74)

change the icon to use an outline for more consistency

* lang

* not core (joomla#75)

* not core

* Update en-GB.plg_installer_override.ini

* namespace

* cs

* Updated files (joomla#82)

* Update default_updated_files.php

* Update en-GB.com_templates.ini

* Update en-GB.com_templates.ini (joomla#81)

* Update en-GB.plg_quickicon_overridecheck.ini (joomla#80)

* Update en-GB.plg_quickicon_overridecheck.ini (joomla#79)

* remove space (joomla#78)

* Update en-GB.plg_quickicon_overridecheck.ini

* Update en-GB.plg_quickicon_overridecheck.sys.ini

* remove hardcoded id

* null get function

* state

* clean

* More changes "core" to "original" (joomla#85)

* cs

* update

* plural

administrator/components/com_admin/sql/others/mysql/utf8mb4-conversion-02.sql

@@ -48,7 +48,7 @@ ALTER TABLE `#__users` MODIFY `name` varchar(400) NOT NULL DEFAULT '';
 --
 -- Step 2.2: Convert all tables to utf8mb4 chracter set with utf8mb4_unicode_ci collation
 -- except #__finder_xxx tables, those will have utf8mb4_general_ci collation.
--- Note: The database driver for mysql will change utf8mb4 to utf8 if utf8mb4 is not supported     
+-- Note: The database driver for mysql will change utf8mb4 to utf8 if utf8mb4 is not supported
 --
 
 ALTER TABLE `#__assets` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
@@ -108,6 +108,7 @@ ALTER TABLE `#__redirect_links` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4
 ALTER TABLE `#__schemas` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__session` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__tags` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
+ALTER TABLE `#__template_overrides` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__template_styles` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__ucm_base` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__ucm_content` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
@@ -200,6 +201,7 @@ ALTER TABLE `#__redirect_links` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_un
 ALTER TABLE `#__schemas` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__session` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__tags` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
+ALTER TABLE `#__template_overrides` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__template_styles` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__ucm_base` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
 ALTER TABLE `#__ucm_content` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

administrator/components/com_admin/sql/updates/mysql/4.0.0-2018-07-19.sql

@@ -0,0 +1,18 @@
+CREATE TABLE IF NOT EXISTS `#__template_overrides` (
+  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
+  `template` varchar(50) NOT NULL DEFAULT '',
+  `hash_id` varchar(255) NOT NULL DEFAULT '',
+  `extension_id` int(11) DEFAULT 0,
+  `state` tinyint(1) NOT NULL DEFAULT 0,
+  `action` varchar(50) NOT NULL DEFAULT '',
+  `client_id` tinyint(1) unsigned NOT NULL DEFAULT 0,
+  `created_date` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
+  `modified_date` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
+  PRIMARY KEY (`id`),
+  KEY `idx_template` (`template`),
+  KEY `idx_extension_id` (`extension_id`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 DEFAULT COLLATE=utf8mb4_unicode_ci;
+
+INSERT INTO `#__extensions` (`extension_id`, `package_id`, `name`, `type`, `element`, `folder`, `client_id`, `enabled`, `access`, `protected`, `manifest_cache`, `params`, `checked_out`, `checked_out_time`, `ordering`, `state`, `namespace`) VALUES
+(491, 0, 'plg_installer_override', 'plugin', 'override', 'installer', 0, 1, 1, 1, '', '', 0, '0000-00-00 00:00:00', 4, 0, ''),
+(492, 0, 'plg_quickicon_overridecheck', 'plugin', 'overridecheck', 'quickicon', 0, 1, 1, 1, '', '', 0, '0000-00-00 00:00:00', 0, 0, ''),

administrator/components/com_admin/sql/updates/postgresql/4.0.0-2018-07-19.sql

@@ -0,0 +1,18 @@
+CREATE TABLE IF NOT EXISTS "#__template_overrides" (
+  "id" serial NOT NULL,
+  "template" varchar(50) DEFAULT '' NOT NULL,
+  "hash_id" varchar(255) DEFAULT '' NOT NULL,
+  "extension_id" bigint DEFAULT 0,
+  "state" smallint DEFAULT 0 NOT NULL,
+  "action" varchar(50) DEFAULT '' NOT NULL,
+  "client_id" smallint DEFAULT 0 NOT NULL,
+  "created_date" timestamp without time zone DEFAULT '1970-01-01 00:00:00' NOT NULL,
+  "modified_date" timestamp without time zone DEFAULT '1970-01-01 00:00:00' NOT NULL,
+  PRIMARY KEY ("id")
+);
+CREATE INDEX "#__template_overrides_idx_template" ON "#__template_overrides" ("template");
+CREATE INDEX "#__template_overrides_idx_extension_id" ON "#__template_overrides" ("extension_id");
+
+INSERT INTO "#__extensions" ("extension_id", "package_id", "name", "type", "element", "folder", "client_id", "enabled", "access", "protected", "manifest_cache", "params", "checked_out", "checked_out_time", "ordering", "state", "namespace") VALUES
+(491, 0, 'plg_installer_override', 'plugin', 'override', 'installer', 0, 1, 1, 1, '', '', 0, '1970-01-01 00:00:00', 4, 0, ''),
+(492, 0, 'plg_quickicon_overridecheck', 'plugin', 'overridecheck', 'quickicon', 0, 1, 1, 1, '', '', 0, '1970-01-01 00:00:00', 0, 0, '');

administrator/components/com_installer/Model/UpdateModel.php

@@ -404,6 +404,9 @@ public function update($uids, $minimum_stability = Updater::STABILITY_STABLE)
 	 */
 	private function install($update)
 	{
+		// Load overrides plugin.
+		PluginHelper::importPlugin('installer');
+
 		$app = Factory::getApplication();
 
 		if (!isset($update->get('downloadurl')->_data))

administrator/components/com_joomlaupdate/Model/UpdateModel.php

@@ -27,6 +27,7 @@
 use Joomla\CMS\Log\Log;
 use Joomla\CMS\Http\HttpFactory;
 use Joomla\CMS\Installer\Installer;
+use Joomla\CMS\Plugin\PluginHelper;
 
 /**
  * Joomla! update overview Model
@@ -389,7 +390,8 @@ protected function downloadPackage($url, $target)
 	}
 
 	/**
-	 * Create restoration file.
+	 * Create restoration file and trigger onJoomlaBeforeUpdate event, which find the updated core files
+	 * which have changed during the update, where there are override for.
 	 *
 	 * @param   string  $basename  Optional base path to the file.
 	 *
@@ -399,11 +401,17 @@ protected function downloadPackage($url, $target)
 	 */
 	public function createRestorationFile($basename = null)
 	{
+		// Load overrides plugin.
+		PluginHelper::importPlugin('installer');
+
 		// Get a password
 		$password = UserHelper::genRandomPassword(32);
 		$app = Factory::getApplication();
 		$app->setUserState('com_joomlaupdate.password', $password);
 
+		// Trigger event before joomla update.
+		$app->triggerEvent('onJoomlaBeforeUpdate');
+
 		// Do we have to use FTP?
 		$method = Factory::getApplication()->getUserStateFromRequest('com_joomlaupdate.method', 'method', 'direct', 'cmd');
 
@@ -826,19 +834,28 @@ public function finaliseUpgrade()
 	}
 
 	/**
-	 * Removes the extracted package file.
+	 * Removes the extracted package file and trigger onJoomlaAfterUpdate event, which find the updated core files
+	 * which have changed during the update, where there are override for.
 	 *
 	 * @return  void
 	 *
 	 * @since   2.5.4
 	 */
 	public function cleanUp()
 	{
+		// Load overrides plugin.
+		PluginHelper::importPlugin('installer');
+
+		$app = Factory::getApplication();
+
+		// Trigger event after joomla update.
+		$app->triggerEvent('onJoomlaAfterUpdate');
+
 		// Remove the update package.
 		$config = Factory::getConfig();
 		$tempdir = $config->get('tmp_path');
 
-		$file = Factory::getApplication()->getUserState('com_joomlaupdate.file', null);
+		$file = $app->getUserState('com_joomlaupdate.file', null);
 		$target = $tempdir . '/' . $file;
 
 		if (!@unlink($target))

administrator/components/com_templates/Controller/TemplateController.php

@@ -11,14 +11,16 @@
 
 defined('_JEXEC') or die;
 
+use Joomla\CMS\Client\ClientHelper;
+use Joomla\CMS\Factory;
+use Joomla\CMS\Language\Text;
 use Joomla\CMS\MVC\Controller\BaseController;
 use Joomla\CMS\MVC\Factory\MVCFactoryInterface;
-use Joomla\Component\Installer\Administrator\Model\InstallModel;
-use Joomla\CMS\Language\Text;
+use Joomla\CMS\Plugin\PluginHelper;
 use Joomla\CMS\Router\Route;
 use Joomla\CMS\Session\Session;
-use Joomla\CMS\Client\ClientHelper;
-use Joomla\CMS\Factory;
+use Joomla\Component\Installer\Administrator\Model\InstallModel;
+use Joomla\Utilities\ArrayHelper;
 
 /**
  * Template style controller class.
@@ -44,6 +46,9 @@ public function __construct($config = array(), MVCFactoryInterface $factory = nu
 
 		// Apply, Save & New, and Save As copy should be standard on forms.
 		$this->registerTask('apply', 'save');
+		$this->registerTask('unpublish', 'publish');
+		$this->registerTask('publish',   'publish');
+		$this->registerTask('deleteOverrideHistory', 'publish');
 	}
 
 	/**
@@ -74,6 +79,64 @@ public function close()
 		$this->setRedirect(Route::_($url, false));
 	}
 
+	/**
+	 * Marked as Checked/Unchecked of override history.
+	 *
+	 * @return  void
+	 *
+	 * @since   __DEPLOY_VERSION__
+	 */
+	public function publish()
+	{
+		// Check for request forgeries.
+		Session::checkToken() or jexit(Text::_('JINVALID_TOKEN'));
+
+		$app  = Factory::getApplication();
+		$file = $this->input->get('file');
+		$id   = $this->input->get('id');
+
+		$ids    = $this->input->get('cid', array(), 'array');
+		$values = array('publish' => 1, 'unpublish' => 0, 'deleteOverrideHistory' => -3);
+		$task   = $this->getTask();
+		$value  = ArrayHelper::getValue($values, $task, 0, 'int');
+
+		if (empty($ids))
+		{
+			$this->setMessage(Text::_('COM_TEMPLATES_ERROR_NO_FILE_SELECTED'), 'warning');
+		}
+		else
+		{
+			/* @var \Joomla\Component\Templates\Administrator\Model\TemplateModel $model */
+			$model = $this->getModel();
+
+			// Change the state of the records.
+			if (!$model->publish($ids, $value, $id))
+			{
+				$this->setMessage(implode('<br>', $model->getErrors()), 'warning');
+			}
+			else
+			{
+				if ($value === 1)
+				{
+					$ntext = 'COM_TEMPLATES_N_OVERRIDE_CHECKED';
+				}
+				elseif ($value === 0)
+				{
+					$ntext = 'COM_TEMPLATES_N_OVERRIDE_UNCHECKED';
+				}
+				elseif ($value === -3)
+				{
+					$ntext = 'COM_TEMPLATES_N_OVERRIDE_DELETED';
+				}
+
+				$this->setMessage(Text::plural($ntext, count($ids)));
+			}
+		}
+
+		$url  = 'index.php?option=com_templates&view=template&id=' . $id . '&file=' . $file;
+		$this->setRedirect(Route::_($url, false));
+	}
+
 	/**
 	 * Method for copying the template.
 	 *
@@ -738,4 +801,43 @@ public function extractArchive()
 			$this->setRedirect(Route::_($url, false));
 		}
 	}
+
+	/**
+	 * Fetch and report updates in \JSON format, for AJAX requests
+	 *
+	 * @return void
+	 *
+	 * @since __DEPLOY_VERSION__
+	 */
+	public function ajax()
+	{
+		$app = $this->app;
+
+		if (!Session::checkToken('get'))
+		{
+			$app->setHeader('status', 403, true);
+			$app->sendHeaders();
+			echo Text::_('JINVALID_TOKEN');
+			$app->close();
+		}
+
+		// Checks status of installer override plugin.
+		if (!PluginHelper::isEnabled('installer', 'override'))
+		{
+			$error = array('installerOverride' => 'disabled');
+
+			echo json_encode($error);
+
+			$app->close();
+		}
+
+		/* @var \Joomla\Component\Templates\Administrator\Model\TemplateModel $model */
+		$model = $this->getModel();
+
+		$result = $model->getUpdatedList(true, true);
+
+		echo json_encode($result);
+
+		$app->close();
+	}
 }