Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for lazy on-demand syncing for RPM. #117

Closed
wants to merge 1 commit into from

Conversation

jeremycline
Copy link
Contributor

This closes issue #73


:param server_config: A :class:`pulp_smash.config.ServerConfig` object.
:param importer: A dict-like object with importer configuration.
:returns: A tuple of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is returned?

@Ichimonji10
Copy link
Contributor

Please write a proper commit message. For example commit messages, browse git log or the online commit log. (Even better, read up on the subject. I still fail to do things like hard-wrap commit message bodies at 72 chars, and could probably focus more heavily on explaining the why in my commit messages.) For some context on why I'm so nuts about good commit messages, see #111. I really do depend on them.


for service in services:
service.start()


def reset_squid(server_config):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding this function in a separate commit. It is easier to revert commits if each contains one logical change, among other benefits.

@dkliban
Copy link
Member

dkliban commented Feb 19, 2016

I got the following error when I tried running this PR:

`[jenkins@rhel7-vanilla-np-qeos-73966 pulp-smash]$ python -m unittest2 pulp_smash.tests.rpm.api_v2.test_sync_publish

.................sE....

ERROR: setUpClass (pulp_smash.tests.rpm.api_v2.test_sync_publish.SyncOnDemandTestCase)

Traceback (most recent call last):
File "pulp_smash/tests/rpm/api_v2/test_sync_publish.py", line 503, in setUpClass
importer=importer_config
File "pulp_smash/tests/rpm/api_v2/test_sync_publish.py", line 110, in _sync_and_publish_repo
tasks = tuple(utils.poll_spawned_tasks(server_config, sync))
AttributeError: 'module' object has no attribute 'poll_spawned_tasks'

`

@jeremycline
Copy link
Contributor Author

@dkliban ahh, I rebased and I guess failed to resolve the merge conflicts correctly.

@Ichimonji10
Copy link
Contributor

FYI, as of c6308d4, pulp_smash.api.safe_handler and pulp_smash.api.json_handler check the final state of asynchronous tasks.

@Ichimonji10
Copy link
Contributor

The specific error you're seeing may be a result of 45cadf4, wherein several functions were moved to pulp_smash.api.

@jeremycline
Copy link
Contributor Author

@Ichimonji10 It's going to be a while before I have time for this. It will probably be much more efficient for everyone involved if you simply take what's here and modify it to fit your needs.

Alternatively you can re-review it and it'll be a few days before I address the issues.

@Ichimonji10
Copy link
Contributor

@jeremycline OK, I may do that. Thanks for the status update. 😄

@Ichimonji10 Ichimonji10 self-assigned this Feb 19, 2016
@Ichimonji10
Copy link
Contributor

I took what is in this pull request and modified it in https://github.com/Ichimonji10/pulp-smash/tree/117. Unfortunately, most of the tests pass fail. It should be revisited for sanity.

@Ichimonji10 Ichimonji10 removed their assignment Feb 19, 2016
@jeremycline
Copy link
Contributor Author

@Ichimonji10 Is there something else you're expecting from me? That link is no longer around and I'm not really sure what you mean by 'Unfortunately, most of the tests pass'.

@Ichimonji10
Copy link
Contributor

Last I checked, most of the tests fail. Can you take a look at them and see if they still fail or not? Sorry about the broken link. That was a snafu on my part. The link should be valid again.

Fix pulp#73. This adds tests for consuming content from a repository that
has been configured with the `on_demand` download policy. This also adds
a method to reset Squid and its cache, and also fixes the `reset_pulp`
method.

Test results prior to this commit:

Test results after this commit:
@Ichimonji10
Copy link
Contributor

This test fails for me with the following error:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='dev.example.com', port=443): Max retries exceeded with url: /streamer/var/lib/pulp/content/units/rpm/76/78177c241777af22235092f21c3932dd4f0664e1624e5a2c77a201ec70f930/bear-4.1-1.noarch.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTkyLjE2OC4xMjEuMSJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS83Ni83ODE3N2MyNDE3NzdhZjIyMjM1MDkyZjIxYzM5MzJkZDRmMDY2NGUxNjI0ZTVhMmM3N2EyMDFlYzcwZjkzMC9iZWFyLTQuMS0xLm5vYXJjaC5ycG0iLCAiZXhwaXJhdGlvbiI6IDE0NTYxNjY2NzR9;signature=UpAVdrBnJib6aVTJPuOSKPgRjXu7UbfGNdhNF8vmX_qZryUzXeiMuhrFdSEwknMVJDUBHmmo9Hf8DNJG9_FUhVCY_a4nq1l18bv14Q9sMDV2g1sP5tJr5F0VzIAX68raCvtrkCeiUeW3UrBRy29z43rhuJve9goeRQZd3tXnaaiJ-cNxfIQ-4ldri1XLNLITcokeLjgVcr8wKQUVg6zx5O4uVKY4pvrmZ1zHu5rpX8X5xsaYGew0gXpm0CRpwYWgH3-uVkAnMbucGBBOkBPDAGeL2acabyT0H6p6Yzb7zMvJLlDjTows_z5Qj8fIdg41WLwFm_IOySPwMp5UYtXHlQ%3D%3D (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7face40d5210>: Failed to establish a new connection: [Errno -2] Name or service not known',))

The interesting part there is the mention of "dev.example.com." That makes me think that this test will succeed when run from within a development Vagrant VM, but not in other contexts, and it would explain the "name or service not known" error I'm receiving. Any insight here?

@jeremycline
Copy link
Contributor Author

@Ichimonji10 Is the Pulp server's hostname dev.example.com?

I'm guessing some setup automation needs to change to set up the reverse proxy, caching proxy, and streamer for lazy content syncing. The Ansible role we use in the development environment[0] is probably a good starting point, but the Squid configuration file needs to be different for RHEL6[1].

[0] https://github.com/pulp/pulp/tree/master/playpen/ansible/roles/lazy
[1] https://github.com/pulp/pulp/blob/master/docs/user-guide/deferred-download.rst#squid

@Ichimonji10
Copy link
Contributor

@Ichimonji10 Is the Pulp server's hostname dev.example.com?

Yes. For what it's worth, I'm running the test suite outside of the VM itself.

@jeremycline
Copy link
Contributor Author

Yes, that's expected behavior. Lazy redirects (by default) the the FQDN of the host Pulp is running on.

@Ichimonji10
Copy link
Contributor

@dkliban, do the Pulp VMs used by Jenkins return correct hostnames?

@jeremycline, is this feature new in 2.8?

@jeremycline
Copy link
Contributor Author

@Ichimonji10 yes, it's new in 2.8

@Ichimonji10
Copy link
Contributor

OK. I'll add in appropriate test skipping logic when I return to this tomorrow.

@Ichimonji10
Copy link
Contributor

Merged as cd92cd1 and 036f67d. Thank you, @jeremycline!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants