diff --git a/docs/Manifest Parameters.md b/docs/Manifest Parameters.md index 266a689f1..31b9efe74 100644 --- a/docs/Manifest Parameters.md +++ b/docs/Manifest Parameters.md @@ -918,10 +918,10 @@ The query command can be used to gather information about the system and the Vis | `` | `source` | R | | | `` | `` | R | | | `` | `` | R | | -| `` | `` | R | | -| `` | `https://dl-ssl.google.com/linux/linux_signing_key.pub` | R | | -| `` | `google-chrome.gpg` | R | | -| `` | `` | R | | +| `` | `` | O | | +| `` | `https://dl-ssl.google.com/linux/linux_signing_key.pub` | O | | +| `` | `google-chrome.gpg` | O | | +| `` | `` | O | | | `` | `` | R | | | `` | `` | R | | | `` | `deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main` | R | | @@ -935,7 +935,7 @@ The query command can be used to gather information about the system and the Vis -#### Source application add Manifest Example +#### Source Application Add Manifest Example using remote GPG key ```xml @@ -958,6 +958,59 @@ The query command can be used to gather information about the system and the Vis ``` +#### Source Application Add Manifest Example (deb822 format) +```xml + + + source + + + + Enabled: yes + Types: deb + URIs: http://dl.google.com/linux/chrome/deb/ + Suites: stable + Components: main + Signed-By: + -----BEGIN PGP PUBLIC KEY BLOCK----- + Version: GnuPG v1.4.2.2 (GNU/Linux) + . + mQGiBEXwb0YRBADQva2NLpYXxgjNkbuP0LnPoEXruGmvi3XMIxjEUFuGNCP4Rj/a + kv2E5VixBP1vcQFDRJ+p1puh8NU0XERlhpyZrVMzzS/RdWdyXf7E5S8oqNXsoD1z + fvmI+i9b2EhHAA19Kgw7ifV8vMa4tkwslEmcTiwiw8lyUl28Wh4Et8SxzwCggDcA + feGqtn3PP5YAdD0km4S4XeMEAJjlrqPoPv2Gf//tfznY2UyS9PUqFCPLHgFLe80u + QhI2U5jt6jUKN4fHauvR6z3seSAsh1YyzyZCKxJFEKXCCqnrFSoh4WSJsbFNc4PN + b0V0SqiTCkWADZyLT5wll8sWuQ5ylTf3z1ENoHf+G3um3/wk/+xmEHvj9HCTBEXP + 78X0A/0Tqlhc2RBnEf+AqxWvM8sk8LzJI/XGjwBvKfXe+l3rnSR2kEAvGzj5Sg0X + 4XmfTg4Jl8BNjWyvm2Wmjfet41LPmYJKsux3g0b8yzQxeOA4pQKKAU3Z4+rgzGmf + HdwCG5MNT2A5XxD/eDd+L4fRx0HbFkIQoAi1J3YWQSiTk15fw7RMR29vZ2xlLCBJ + bmMuIExpbnV4IFBhY2thZ2UgU2lnbmluZyBLZXkgPGxpbnV4LXBhY2thZ2VzLWtl + eW1hc3RlckBnb29nbGUuY29tPohjBBMRAgAjAhsDBgsJCAcDAgQVAggDBBYCAwEC + HgECF4AFAkYVdn8CGQEACgkQoECDD3+sWZHKSgCfdq3HtNYJLv+XZleb6HN4zOcF + AJEAniSFbuv8V5FSHxeRimHx25671az+uQINBEXwb0sQCACuA8HT2nr+FM5y/kzI + A51ZcC46KFtIDgjQJ31Q3OrkYP8LbxOpKMRIzvOZrsjOlFmDVqitiVc7qj3lYp6U + rgNVaFv6Qu4bo2/ctjNHDDBdv6nufmusJUWq/9TwieepM/cwnXd+HMxu1XBKRVk9 + XyAZ9SvfcW4EtxVgysI+XlptKFa5JCqFM3qJllVohMmr7lMwO8+sxTWTXqxsptJo + pZeKz+UBEEqPyw7CUIVYGC9ENEtIMFvAvPqnhj1GS96REMpry+5s9WKuLEaclWpd + K3krttbDlY1NaeQUCRvBYZ8iAG9YSLHUHMTuI2oea07Rh4dtIAqPwAX8xn36JAYG + 2vgLAAMFB/wKqaycjWAZwIe98Yt0qHsdkpmIbarD9fGiA6kfkK/UxjL/k7tmS4Vm + CljrrDZkPSQ/19mpdRcGXtb0NI9+nyM5trweTvtPw+HPkDiJlTaiCcx+izg79Fj9 + KcofuNb3lPdXZb9tzf5oDnmm/B+4vkeTuEZJ//IFty8cmvCpzvY+DAz1Vo9rA+Zn + cpWY1n6z6oSS9AsyT/IFlWWBZZ17SpMHu+h4Bxy62+AbPHKGSujEGQhWq8ZRoJAT + G0KSObnmZ7FwFWu1e9XFoUCt0bSjiJWTIyaObMrWu/LvJ3e9I87HseSJStfw6fki + 5og9qFEkMrIrBCp3QGuQWBq/rTdMuwNFiEkEGBECAAkFAkXwb0sCGwwACgkQoECD + D3+sWZF/WACfeNAu1/1hwZtUo1bR+MWiCjpvHtwAnA1R3IHqFLQ2X3xJ40XPuAyY + /FJG + %20=Quqp + -----END PGP PUBLIC KEY BLOCK----- + + google-chrome.sources + + + + +``` + #### Source Application Update Manifest Parameters | Tag | Example | Required/Optional | Notes | |:-----------------------------------------|:-----------------------------------------------------------------------------------------------|:-----------------:|:------| @@ -966,7 +1019,6 @@ The query command can be used to gather information about the system and the Vis | `` | `source` | R | | | `` | `` | R | | | `` | `` | R | | -| `` | `` | R | | | `` | `` | R | | | `` | `` | R | | | `` | `deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main` | R | | @@ -999,27 +1051,27 @@ The query command can be used to gather information about the system and the Vis ``` #### Source Application Remove Manifest Parameters -| Tag | Example | Required/Optional | Notes | -|:-----------------------------------------|:-----------------------------------------|:-----------------:|:------| -| `` | `` | R | | -| `` | `` | R | | -| `` | `source` | R | | -| `` | `` | R | | -| `` | `` | R | | -| `` | `` | R | | -| `` | `google-chrome.gpg` | R | | -| `` | `` | R | | -| `` | `` | R | | -| `` | `google-chrom.list` | R | | -| `` | `` | R | | -| `` | `` | R | | -| `` | `` | R | | -| `` | `` | R | | - - - - -#### Source Application Remove Manifest Example +| Tag | Example | Required/Optional | Notes | +|:-----------------------------------------|:------------------------------------------|:-----------------:|:------| +| `` | `` | R | | +| `` | `` | R | | +| `` | `source` | R | | +| `` | `` | R | | +| `` | `` | R | | +| `` | `` | O | | +| `` | `google-chrome.gpg` | O | | +| `` | `` | O | | +| `` | `` | R | | +| `` | `google-chrome.list` | R | | +| `` | `` | R | | +| `` | `` | R | | +| `` | `` | R | | +| `` | `` | R | | + + + + +#### Source Application Remove Manifest Example (Including GPG key) ```xml @@ -1037,6 +1089,21 @@ The query command can be used to gather information about the system and the Vis ``` +#### Source Application Remove Manifest Example (Excluding GPG key) +```xml + + + source + + + + google-chrome.sources + + + + +``` + #### Source Application List Manifest Parameters | Tag | Example | Required/Optional | Notes | |:-----------------------------------------|:-----------------------------------------|:-----------------:|:------| diff --git a/inbc-program/README.md b/inbc-program/README.md index 4737645c6..4d36ab54d 100644 --- a/inbc-program/README.md +++ b/inbc-program/README.md @@ -414,20 +414,20 @@ inbc query --option sw ## SOURCE APPLICATION ADD ### Description -Downloads and encrypts GPG key and stores it on the system under /usr/share/keyrings. Creates a file under /etc/apt/sources.list.d to store the update source information. -This list file is used during 'sudo apt update' to update the application +Optionally Downloads and encrypts GPG key and stores it on the system under /usr/share/keyrings. Creates a file under /etc/apt/sources.list.d to store the update source information. +This list file is used during 'sudo apt update' to update the application. Deb882 format may be used instead of downloading a GPG key. ### Usage ``` -inbc source application add - {--gpgKeyUri, -gku=GPG_KEY_URI} - {--gpgKeyName, -gkn=GPG_KEY_NAME} +inbc source application add {--sources, -s=SOURCES} {--filename, -f=FILENAME} + [--gpgKeyUri, -gku=GPG_KEY_URI] + [--gpgKeyName, -gkn=GPG_KEY_NAME] ``` ### Example -#### Add an application source +#### Add an Application Source (non-deb822 format with remote GPG key) ``` inbc source application add --gpgKeyUri https://dl-ssl.google.com/linux/linux_signing_key.pub @@ -436,25 +436,81 @@ inbc source application add --filename google-chrome.list ``` +#### Add an Application Source (using deb822 format) + +**NOTE:** In the Signed-By: Section, use the following guidelines. + + - Each blank line has a period in it. -> " ." + - Each line after the Signed-By: starts with a space -> " gibberish" + +``` +inbc source application add + --sources + "Enabled: yes" + "Types: deb" + "URIs: http://dl.google.com/linux/chrome/deb/" + "Suites: stable" + "Components: main" + "Signed-By:" + " -----BEGIN PGP PUBLIC KEY BLOCK-----" + " Version: GnuPG v1.4.2.2 (GNU/Linux)" + " ." + " mQGiBEXwb0YRBADQva2NLpYXxgjNkbuP0LnPoEXruGmvi3XMIxjEUFuGNCP4Rj/a" + " kv2E5VixBP1vcQFDRJ+p1puh8NU0XERlhpyZrVMzzS/RdWdyXf7E5S8oqNXsoD1z" + " fvmI+i9b2EhHAA19Kgw7ifV8vMa4tkwslEmcTiwiw8lyUl28Wh4Et8SxzwCggDcA" + " feGqtn3PP5YAdD0km4S4XeMEAJjlrqPoPv2Gf//tfznY2UyS9PUqFCPLHgFLe80u" + " QhI2U5jt6jUKN4fHauvR6z3seSAsh1YyzyZCKxJFEKXCCqnrFSoh4WSJsbFNc4PN" + " b0V0SqiTCkWADZyLT5wll8sWuQ5ylTf3z1ENoHf+G3um3/wk/+xmEHvj9HCTBEXP" + " 78X0A/0Tqlhc2RBnEf+AqxWvM8sk8LzJI/XGjwBvKfXe+l3rnSR2kEAvGzj5Sg0X" + " 4XmfTg4Jl8BNjWyvm2Wmjfet41LPmYJKsux3g0b8yzQxeOA4pQKKAU3Z4+rgzGmf" + " HdwCG5MNT2A5XxD/eDd+L4fRx0HbFkIQoAi1J3YWQSiTk15fw7RMR29vZ2xlLCBJ" + " bmMuIExpbnV4IFBhY2thZ2UgU2lnbmluZyBLZXkgPGxpbnV4LXBhY2thZ2VzLWtl" + " eW1hc3RlckBnb29nbGUuY29tPohjBBMRAgAjAhsDBgsJCAcDAgQVAggDBBYCAwEC" + " HgECF4AFAkYVdn8CGQEACgkQoECDD3+sWZHKSgCfdq3HtNYJLv+XZleb6HN4zOcF" + " AJEAniSFbuv8V5FSHxeRimHx25671az+uQINBEXwb0sQCACuA8HT2nr+FM5y/kzI" + " A51ZcC46KFtIDgjQJ31Q3OrkYP8LbxOpKMRIzvOZrsjOlFmDVqitiVc7qj3lYp6U" + " rgNVaFv6Qu4bo2/ctjNHDDBdv6nufmusJUWq/9TwieepM/cwnXd+HMxu1XBKRVk9" + " XyAZ9SvfcW4EtxVgysI+XlptKFa5JCqFM3qJllVohMmr7lMwO8+sxTWTXqxsptJo" + " pZeKz+UBEEqPyw7CUIVYGC9ENEtIMFvAvPqnhj1GS96REMpry+5s9WKuLEaclWpd" + " K3krttbDlY1NaeQUCRvBYZ8iAG9YSLHUHMTuI2oea07Rh4dtIAqPwAX8xn36JAYG" + " 2vgLAAMFB/wKqaycjWAZwIe98Yt0qHsdkpmIbarD9fGiA6kfkK/UxjL/k7tmS4Vm" + " CljrrDZkPSQ/19mpdRcGXtb0NI9+nyM5trweTvtPw+HPkDiJlTaiCcx+izg79Fj9" + " KcofuNb3lPdXZb9tzf5oDnmm/B+4vkeTuEZJ//IFty8cmvCpzvY+DAz1Vo9rA+Zn" + " cpWY1n6z6oSS9AsyT/IFlWWBZZ17SpMHu+h4Bxy62+AbPHKGSujEGQhWq8ZRoJAT" + " G0KSObnmZ7FwFWu1e9XFoUCt0bSjiJWTIyaObMrWu/LvJ3e9I87HseSJStfw6fki" + " 5og9qFEkMrIrBCp3QGuQWBq/rTdMuwNFiEkEGBECAAkFAkXwb0sCGwwACgkQoECD" + " D3+sWZF/WACfeNAu1/1hwZtUo1bR+MWiCjpvHtwAnA1R3IHqFLQ2X3xJ40XPuAyY" + " /FJG" + " %20=Quqp" + " -----END PGP PUBLIC KEY BLOCK-----" + --filename google-chrome.sources +``` + ## SOURCE APPLICATION REMOVE ### Description -Removes the GPG key file from under /usr/share/keyrings. Removes the source file from under /etc/apt/sources.list.d/. +Removes the source file from under /etc/apt/sources.list.d/. Optionally removes the GPG key file from under /usr/share/keyrings. ### Usage ``` -inbc source application remove - {--gpgKeyName, -gkn=GPG_KEY_NAME} +inbc source application remove {--filename, -f=FILE_NAME} + [--gpgKeyName, -gkn=GPG_KEY_NAME] ``` ### Example -#### Remove an application source +#### Remove an application source (Both GPG key and Source File) ```commandline inbc source application remove --gpgKeyName google-chrome.gpg --filename google-chrome.list ``` +#### Remove an application source (deb822 format) +```commandline +inbc source application remove + --filename google-chrome.sources +``` + ## SOURCE APPLICATION UPDATE ### Description Updates Application sources that are used to update the system @@ -478,7 +534,6 @@ inbc source application update ## SOURCE APPLICATION LIST ### Description Lists Application sources -NOTE: Currently this only works on Ubuntu ### Usage ``` @@ -545,7 +600,6 @@ inbc source os update ## SOURCE OS LIST ### Description Lists OS sources -NOTE: Currently this only works on Ubuntu ### Usage ```commandline diff --git a/inbc-program/inbc/parser/parser.py b/inbc-program/inbc/parser/parser.py index 97201444b..3f3208d50 100644 --- a/inbc-program/inbc/parser/parser.py +++ b/inbc-program/inbc/parser/parser.py @@ -65,11 +65,11 @@ def parse_source_args(self) -> None: # Application Add Command app_add_parser = app_subparsers.add_parser('add') - app_add_parser.add_argument('--gpgKeyUri', '-gku', required=True, + app_add_parser.add_argument('--gpgKeyUri', '-gku', required=False, type=lambda x: validate_string_less_than_n_characters( x, 'str', 1500), help='Uri from which to download GPG key') - app_add_parser.add_argument('--gpgKeyName', '-gkn', required=True, + app_add_parser.add_argument('--gpgKeyName', '-gkn', required=False, type=lambda x: validate_string_less_than_n_characters( x, 'str', 200), help='Name to store the GPG key information') @@ -85,7 +85,7 @@ def parse_source_args(self) -> None: # Application Remove Command app_remove_parser = app_subparsers.add_parser('remove') - app_remove_parser.add_argument('--gpgKeyName', '-gkn', required=True, + app_remove_parser.add_argument('--gpgKeyName', '-gkn', required=False, type=lambda x: validate_string_less_than_n_characters( x, 'str', 50), help='GPG key name of the source to remove.') diff --git a/inbc-program/inbc/parser/source_app_parser.py b/inbc-program/inbc/parser/source_app_parser.py index 680dd4615..3702f3690 100644 --- a/inbc-program/inbc/parser/source_app_parser.py +++ b/inbc-program/inbc/parser/source_app_parser.py @@ -4,11 +4,19 @@ SPDX-License-Identifier: Apache-2.0 """ import argparse +import logging +from ..inbc_exception import InbcException from ..xml_tag import create_xml_tag +logger = logging.getLogger(__name__) + def application_add(args: argparse.Namespace) -> str: + if (args.gpgKeyUri and args.gpgKeyName is None) or (args.gpgKeyName and args.gpgKeyUri is None): + raise InbcException( + "Source requires either both gpgKeyUri and gpgKeyName to be provided, or neither of them.") + arguments = { 'uri': args.gpgKeyUri, 'keyname': args.gpgKeyName, @@ -19,15 +27,16 @@ def application_add(args: argparse.Namespace) -> str: manifest = ('' + '' + 'source' + - '' + - '' - '{0}' + - '{1}' - '').format(create_xml_tag(arguments, "uri"), - create_xml_tag(arguments, "keyname")) + '' + '') + + if args.gpgKeyUri and args.gpgKeyName: + manifest += ('' + '{0}' + '{1}' + '').format(create_xml_tag(arguments, "uri"), + create_xml_tag(arguments, "keyname")) + + manifest += '' for source in args.sources: - manifest += '' + source.strip() + '' + manifest += '' + source + '' manifest += ('' f'{create_xml_tag(arguments, "filename")}' @@ -47,13 +56,16 @@ def application_remove(args: argparse.Namespace) -> str: manifest = ('' + 'source' + '' + - '' - f'{create_xml_tag(arguments, "keyname")}' + - '' + - f'{create_xml_tag(arguments, "filename")}' - '' - '' + - '') + '') + + if args.gpgKeyName: + manifest += f'{create_xml_tag(arguments, "keyname")}' + + manifest += ('' + + f'{create_xml_tag(arguments, "filename")}' + '' + '' + + '') print(f"manifest {manifest}") return manifest diff --git a/inbc-program/tests/unit/test_source_app_parser.py b/inbc-program/tests/unit/test_source_app_parser.py index 43cfbaeb8..c16d98595 100644 --- a/inbc-program/tests/unit/test_source_app_parser.py +++ b/inbc-program/tests/unit/test_source_app_parser.py @@ -1,8 +1,10 @@ +import pytest from unittest import TestCase from unittest.mock import patch from inbc.inbc import Inbc from inbc.parser.parser import ArgsParser +from inbc.inbc_exception import InbcException class TestSourceApplicationParser(TestCase): @@ -10,7 +12,7 @@ def setUp(self): self.arg_parser = ArgsParser() self.maxDiff = None - def test_parse_add_arguments_successfully(self): + def test_parse_add_all_arguments_successfully(self): f = self.arg_parser.parse_args( ['source', 'application', 'add', '--gpgKeyUri', 'https://repositories.intel.com/gpu/intel-graphics.key', @@ -24,8 +26,76 @@ def test_parse_add_arguments_successfully(self): 'deb-src http://example.com/ focal-security main']) self.assertEqual(f.filename, 'intel-gpu-jammy.list') + def test_parse_add_arguments_deb822_format_separate_lines_successfully(self): + f = self.arg_parser.parse_args( + ['source', 'application', 'add', + '--sources', 'X-Repolib-Name: Google Chrome', + 'Enabled: yes', + 'Types: deb', + 'URIs: https://dl-ssl.google.com/linux/linux_signing_key.pub', + 'Suites: stable', + 'Components: main', + '--filename', 'intel-gpu-jammy.list']) + self.assertEqual(f.gpgKeyUri, None) + self.assertEqual(f.gpgKeyName, None) + self.assertEqual(f.sources, ['X-Repolib-Name: Google Chrome', + 'Enabled: yes', + 'Types: deb', + 'URIs: https://dl-ssl.google.com/linux/linux_signing_key.pub', + 'Suites: stable', + 'Components: main']) + self.assertEqual(f.filename, 'intel-gpu-jammy.list') + + @patch('inbm_lib.mqttclient.mqtt.mqtt.Client.connect') + def test_raise_application_add_with_only_one_gpgKeyUri_param(self, m_connect): + p = self.arg_parser.parse_args( + ['source', 'application', 'add', + '--gpgKeyUri', 'https://repositories.intel.com/gpu/intel-graphics.key', + '--sources', 'deb http://example.com/ focal main restricted universe', + 'deb-src http://example.com/ focal-security main', + '--filename', 'intel-gpu-jammy.list']) + with pytest.raises(InbcException, + match="Source requires either both gpgKeyUri and gpgKeyName " + "to be provided, or neither of them."): + Inbc(p, 'source', False) + + @patch('inbm_lib.mqttclient.mqtt.mqtt.Client.connect') + def test_raise_application_add_with_only_one_gpgKeyName_param(self, m_connect): + p = self.arg_parser.parse_args( + ['source', 'application', 'add', + '--gpgKeyName', 'intel-graphics.gpg', + '--sources', 'deb http://example.com/ focal main restricted universe', + 'deb-src http://example.com/ focal-security main', + '--filename', 'intel-gpu-jammy.list']) + with pytest.raises(InbcException, + match="Source requires either both gpgKeyUri and gpgKeyName " + "to be provided, or neither of them."): + Inbc(p, 'source', False) + + @patch('inbm_lib.mqttclient.mqtt.mqtt.Client.connect') + def test_create_add_deb_822_format_manifest_successfully(self, m_connect): + p = self.arg_parser.parse_args( + ['source', 'application', 'add', + '--sources', 'X-Repolib-Name: Google Chrome', + 'Enabled: yes', + 'Types: deb', + 'URIs: https://dl-ssl.google.com/linux/linux_signing_key.pub', + 'Suites: stable', + 'Components: main', + '--filename', 'google-chrome.sources']) + Inbc(p, 'source', False) + expected = 'source' \ + 'X-Repolib-Name: Google Chrome' \ + 'Enabled: yes' \ + 'Types: deb'\ + 'URIs: https://dl-ssl.google.com/linux/linux_signing_key.pub' \ + 'Suites: stable' \ + 'Components: main' \ + 'google-chrome.sources' + self.assertEqual(p.func(p), expected) + @patch('inbm_lib.mqttclient.mqtt.mqtt.Client.connect') - def test_create_add_manifest_successfully(self, m_connect): + def test_create_add_all_param_manifest_successfully(self, m_connect): p = self.arg_parser.parse_args( ['source', 'application', 'add', '--gpgKeyUri', 'https://repositories.intel.com/gpu/intel-graphics.key', @@ -42,7 +112,22 @@ def test_create_add_manifest_successfully(self, m_connect): 'intel-gpu-jammy.list' self.assertEqual(p.func(p), expected) - def test_parse_remove_arguments_successfully(self): + @patch('inbm_lib.mqttclient.mqtt.mqtt.Client.connect') + def test_create_add_minus_gpg_manifest_successfully(self, m_connect): + p = self.arg_parser.parse_args( + ['source', 'application', 'add', + '--sources', 'deb http://example.com/ focal main restricted universe', + 'deb-src http://example.com/ focal-security main', + '--filename', 'intel-gpu-jammy.list']) + Inbc(p, 'source', False) + expected = 'source' \ + '' \ + 'deb http://example.com/ focal main restricted universe' \ + 'deb-src http://example.com/ focal-security main' \ + 'intel-gpu-jammy.list' + self.assertEqual(p.func(p), expected) + + def test_parse_all_remove_arguments_successfully(self): f = self.arg_parser.parse_args( ['source', 'application', 'remove', '--gpgKeyName', 'intel-gpu-jammy.gpg', @@ -50,8 +135,15 @@ def test_parse_remove_arguments_successfully(self): self.assertEqual(f.gpgKeyName, 'intel-gpu-jammy.gpg') self.assertEqual(f.filename, 'intel-gpu-jammy.list') + def test_parse_minus_gpg_remove_arguments_successfully(self): + f = self.arg_parser.parse_args( + ['source', 'application', 'remove', + '--filename', 'intel-gpu-jammy.list']) + self.assertEqual(f.gpgKeyName, None) + self.assertEqual(f.filename, 'intel-gpu-jammy.list') + @patch('inbm_lib.mqttclient.mqtt.mqtt.Client.connect') - def test_create_remove_manifest_successfully(self, m_connect): + def test_create_remove_manifest_all_params_successfully(self, m_connect): p = self.arg_parser.parse_args( ['source', 'application', 'remove', '--gpgKeyName', 'intel-gpu-jammy.gpg', @@ -62,6 +154,16 @@ def test_create_remove_manifest_successfully(self, m_connect): 'intel-gpu-jammy.list' self.assertEqual(p.func(p), expected) + @patch('inbm_lib.mqttclient.mqtt.mqtt.Client.connect') + def test_create_remove_manifest_minus_gpg_successfully(self, m_connect): + p = self.arg_parser.parse_args( + ['source', 'application', 'remove', + '--filename', 'intel-gpu-jammy.list']) + Inbc(p, 'source', False) + expected = 'source' \ + 'intel-gpu-jammy.list' + self.assertEqual(p.func(p), expected) + def test_parse_update_arguments_successfully(self): f = self.arg_parser.parse_args( ['source', 'application', 'update', diff --git a/inbm/dispatcher-agent/dispatcher/source/constants.py b/inbm/dispatcher-agent/dispatcher/source/constants.py index 3d7d3f94c..908aab9cb 100644 --- a/inbm/dispatcher-agent/dispatcher/source/constants.py +++ b/inbm/dispatcher-agent/dispatcher/source/constants.py @@ -6,6 +6,7 @@ from enum import Enum, unique from dataclasses import dataclass, field +from typing import Optional UBUNTU_APT_SOURCES_LIST = "/etc/apt/sources.list" UBUNTU_APT_SOURCES_LIST_D = "/etc/apt/sources.list.d" @@ -25,16 +26,16 @@ class SourceParameters: @dataclass(kw_only=True, frozen=True) class ApplicationAddSourceParameters: - gpg_key_uri: str - gpg_key_name: str file_name: str sources: list[str] = field(default_factory=lambda: []) + gpg_key_uri: Optional[str] = field(default=None) + gpg_key_name: Optional[str] = field(default=None) @dataclass(kw_only=True, frozen=True) class ApplicationRemoveSourceParameters: - gpg_key_name: str file_name: str + gpg_key_name: Optional[str] = field(default=None) @dataclass(kw_only=True, frozen=True) diff --git a/inbm/dispatcher-agent/dispatcher/source/source_command.py b/inbm/dispatcher-agent/dispatcher/source/source_command.py index 2f90e0e7e..4b30b5cc8 100644 --- a/inbm/dispatcher-agent/dispatcher/source/source_command.py +++ b/inbm/dispatcher-agent/dispatcher/source/source_command.py @@ -54,7 +54,7 @@ def _handle_os_source_command(parsed_head: XmlHandler, os_type: OsType, os_actio Handle the os source commands. @param parsed_head: XmlHandler with command information - @param os_type: os type + @param os_type: OS type @param os_action: The action to be performed @return Result """ @@ -94,8 +94,7 @@ def _handle_os_source_command(parsed_head: XmlHandler, os_type: OsType, os_actio def _handle_app_source_command( - parsed_head: XmlHandler, os_type: OsType, app_action: dict -) -> Result: + parsed_head: XmlHandler, os_type: OsType, app_action: dict) -> Result: """ Handle the application source commands. @@ -113,7 +112,12 @@ def _handle_app_source_command( return Result(status=200, message=serialized_list) if "remove" in app_action: - keyname = parsed_head.get_children("applicationSource/remove/gpg")["keyname"] + keyname = None + try: + keyname = parsed_head.get_children("applicationSource/remove/gpg")["keyname"] + except XmlException: + # These children may not be present + logger.info(f"Optional GPG key parameters not present in manifest") filename = parsed_head.get_children("applicationSource/remove/repo")["filename"] application_source_manager.remove( ApplicationRemoveSourceParameters(file_name=filename, gpg_key_name=keyname) @@ -121,8 +125,16 @@ def _handle_app_source_command( return Result(status=200, message="SUCCESS") if "add" in app_action: - gpg_key_uri = parsed_head.get_children("applicationSource/add/gpg")["uri"] - gpg_key_name = parsed_head.get_children("applicationSource/add/gpg")["keyname"] + gpg_key_uri = None + gpg_key_name = None + + try: + gpg_key_uri = parsed_head.get_children("applicationSource/add/gpg")["uri"] + gpg_key_name = parsed_head.get_children("applicationSource/add/gpg")["keyname"] + except XmlException: + # These children may not be present + logger.info(f"Optional GPG key parameters not present in manifest") + repo_filename = parsed_head.get_children("applicationSource/add/repo")["filename"] add_source_pkgs: list[str] = [] diff --git a/inbm/dispatcher-agent/dispatcher/source/ubuntu_source_manager.py b/inbm/dispatcher-agent/dispatcher/source/ubuntu_source_manager.py index 220b8be3f..8838ff58a 100644 --- a/inbm/dispatcher-agent/dispatcher/source/ubuntu_source_manager.py +++ b/inbm/dispatcher-agent/dispatcher/source/ubuntu_source_manager.py @@ -23,7 +23,6 @@ from inbm_common_lib.utility import ( get_canonical_representation_of_path, remove_file, - move_file, create_file_with_contents, ) @@ -98,8 +97,10 @@ def __init__(self) -> None: pass def add(self, parameters: ApplicationAddSourceParameters) -> None: - # Step 1: Add key - add_gpg_key(parameters.gpg_key_uri, parameters.gpg_key_name) + """Adds a source file and optional GPG key to be used during Ubuntu application updates.""" + # Step 1: Add key (Optional) + if parameters.gpg_key_name and parameters.gpg_key_uri: + add_gpg_key(parameters.gpg_key_uri, parameters.gpg_key_name) # Step 2: Add the source try: @@ -134,11 +135,13 @@ def list(self) -> list[ApplicationSourceList]: raise SourceError(f"Error listing application sources: {e}") from e def remove(self, parameters: ApplicationRemoveSourceParameters) -> None: - """Removes a source file from the Ubuntu source file list under /etc/apt/sources.list.d + """Removes a source file from the Ubuntu source file list under /etc/apt/sources.list.d. Optionally + removes the gpg key from /usr/share/keyrings @parameters: dataclass parameters for ApplicationRemoveSourceParameters """ - # Remove the GPG key - remove_gpg_key_if_exists(parameters.gpg_key_name) + if parameters.gpg_key_name: + # Remove the GPG key (Optional) + remove_gpg_key_if_exists(parameters.gpg_key_name) # Remove the file under /etc/apt/sources.list.d try: diff --git a/inbm/dispatcher-agent/fpm-template/usr/share/dispatcher-agent/manifest_schema.xsd b/inbm/dispatcher-agent/fpm-template/usr/share/dispatcher-agent/manifest_schema.xsd index 6a8fad59b..36ce3c38a 100644 --- a/inbm/dispatcher-agent/fpm-template/usr/share/dispatcher-agent/manifest_schema.xsd +++ b/inbm/dispatcher-agent/fpm-template/usr/share/dispatcher-agent/manifest_schema.xsd @@ -319,11 +319,11 @@ - + - - + + @@ -333,7 +333,7 @@ - + @@ -353,7 +353,7 @@ - + @@ -367,10 +367,10 @@ - + - + diff --git a/inbm/dispatcher-agent/tests/unit/source/test_ubuntu_source_cmd.py b/inbm/dispatcher-agent/tests/unit/source/test_ubuntu_source_cmd.py index 9206154f9..933e37379 100644 --- a/inbm/dispatcher-agent/tests/unit/source/test_ubuntu_source_cmd.py +++ b/inbm/dispatcher-agent/tests/unit/source/test_ubuntu_source_cmd.py @@ -8,6 +8,7 @@ ApplicationRemoveSourceParameters, SourceParameters, ApplicationUpdateSourceParameters, + ApplicationAddSourceParameters ) from dispatcher.source.ubuntu_source_manager import ( UbuntuApplicationSourceManager, @@ -192,14 +193,44 @@ def test_update_sources_os_error(self): class TestUbuntuApplicationSourceManager: - @patch("dispatcher.source.ubuntu_source_manager.move_file") - def test_update_app_source_successfully(self, mock_move): + def test_add_app_with_gpg_key_successfully(self): + try: + params = ApplicationAddSourceParameters( + file_name="intel-gpu-jammy.list", + sources="deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main", + gpg_key_uri="https://dl-ssl.google.com/linux/linux_signing_key.pub", + gpg_key_name="google-chrome.gpg" + ) + command = UbuntuApplicationSourceManager() + with patch("builtins.open", new_callable=mock_open()): + command.add(params) + except SourceError as err: + assert False, f"'UbuntuApplicationSourceManager.add' raised an exception {err}" + + def test_add_app_deb_822_format_successfully(self): + try: + params = ApplicationAddSourceParameters( + file_name="google-chrome.sources", + sources="X-Repolib-Name: Google Chrome" + "Enabled: yes" + "Types: deb" + "URIs: https://dl-ssl.google.com/linux/linux_signing_key.pub" + "Suites: stable" + "Components: main", + ) + command = UbuntuApplicationSourceManager() + with patch("builtins.open", new_callable=mock_open()): + command.add(params) + except SourceError as err: + assert False, f"'UbuntuApplicationSourceManager.add' raised an exception {err}" + + def test_update_app_source_successfully(self): try: params = ApplicationUpdateSourceParameters( file_name="intel-gpu-jammy.list", sources=APP_SOURCE ) command = UbuntuApplicationSourceManager() - with patch("builtins.open", new_callable=mock_open()) as m: + with patch("builtins.open", new_callable=mock_open()): command.update(params) except SourceError as err: assert False, f"'UbuntuApplicationSourceManager.update' raised an exception {err}" diff --git a/inbm/integration-reloaded/README.md b/inbm/integration-reloaded/README.md index 1cb2c9ed5..3a96a3a1b 100644 --- a/inbm/integration-reloaded/README.md +++ b/inbm/integration-reloaded/README.md @@ -1,20 +1,20 @@ # Introduction -integration-reloaded is sub-project within 'inb' which helps create a Ubuntu 20.04 or 22.04 server environment +integration-reloaded is subproject within 'inb' which helps create a Ubuntu 20.04 or 22.04 server environment with a BTRFS file system (mounted as / ) # Uses 1. It can be used as a test box for testing 'inb' artifacts 2. Demo machine -3. Currently being used as a part of iotg-inb's CI system. +3. It is currently being used as a part of iotg-inb's CI system. # Local setup -If you want to setup a system ready to go with a click of a button...you will have to perform a one-time setup +If you want to set up a system ready to go with a click of a button...you will have to perform a one-time setup on your local dev system -## Pre-requistes: +## Pre-requisites: Installing Vagrant, Vagrant-libvirt, Qmenu-KVM and the plugins etc 1. Download vagrant from vagrant https://www.vagrantup.com/downloads.html and @@ -22,12 +22,12 @@ Installing Vagrant, Vagrant-libvirt, Qmenu-KVM and the plugins etc 2. Edit the /etc/apt/sources.list and uncomment all the deb-src lines -3. Run the below commands in sequence (preferrably as root) +3. Run the below commands in sequence (preferably as root) ``` apt update - apt install qemu qemu-kvm libvirt-bin + apt install qemu qemu-kvm libvirt-daemon-system libvirt-clients apt-get build-dep vagrant ruby-libvirt - apt-get install qemu libvirt-bin ebtables dnsmasq + apt-get install qemu libvirt-daemon-system libvirt-clients ebtables dnsmasq apt-get install libxslt-dev libvirt-dev zlib1g-dev ruby-dev ``` @@ -38,8 +38,8 @@ vagrant plugin install vagrant-libvirt ``` ## How to use it -1. Clone the iotg-inb repo -2. `cd ~/iotg-inb/integration-reloaded` +1. Clone the `intel-inb-manageability` repo +2. `cd ~/intel-inb-manageability/inbm/integration-reloaded` 3. Create a folder in integration-reloaded called `input` 4. Download all the iotg-inb artifacts in that `input` folder and unzip the artifacts 5. export your docker username as DOCKER_USERNAME and your docker password as DOCKER_PASSWORD diff --git a/inbm/integration-reloaded/test/source/SOURCE.sh b/inbm/integration-reloaded/test/source/SOURCE.sh index c19cef04d..e8c69ef13 100755 --- a/inbm/integration-reloaded/test/source/SOURCE.sh +++ b/inbm/integration-reloaded/test/source/SOURCE.sh @@ -16,6 +16,7 @@ OPERA_KEY_NAME="opera.gpg" OPERA_SOURCES="deb [arch=amd64 signed-by=/usr/share/keyrings/$OPERA_KEY_NAME] https://deb.opera.com/opera-stable/ stable non-free" OPERA_LIST="opera.list" NEW_APP_SOURCE="deb newsource" +CHROME_SOURCES_FILE="google-chrome.sources" cp "$APT_SOURCES" "$BAK_APT_SOURCES" @@ -56,12 +57,26 @@ fi inbc source application list 2>&1 | grep "$OPERA_KEY_NAME" inbc source application remove --gpgKeyName "$OPERA_KEY_NAME" --filename "$OPERA_LIST" +inbc source application add --filename $CHROME_SOURCES_FILE --sources \"Enabled: yes\" \"Types: deb\" \"URIs: http://dl.google.com/linux/chrome/deb/\" \"Suites: stable\" \"Components: main\" + +if [ ! -e "/etc/apt/sources.list.d/$CHROME_SOURCES_FILE" ]; then + echo "Error: The file '/etc/apt/sources.list.d/$CHROME_SOURCES_FILE' does not exist!" + exit 1 +fi +inbc source application remove --filename "$CHROME_SOURCES_FILE" + if inbc source application list 2>&1 | grep -q "$OPERA_KEY_NAME"; then echo "Error: $OPERA_KEY_NAME should not be present in the application list after removal" exit 1 fi +if inbc source application list 2>&1 | grep -q "$CHROME_SOURCES_FILE"; then + echo "Error: $CHROME_SOURCES_FILE should not be present in the application list after removal" + exit 1 +fi + inbc source application add --gpgKeyUri "$OPERA_KEY_URI" --gpgKeyName "$OPERA_KEY_NAME" --sources "$OPERA_SOURCES" --filename "$OPERA_LIST" + inbc source application update --sources "$NEW_APP_SOURCE" --filename "$OPERA_LIST" inbc source application list 2>&1 | grep "$NEW_APP_SOURCE"