-
Notifications
You must be signed in to change notification settings - Fork 198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: search current working directory for config file #1464
base: main
Are you sure you want to change the base?
Changes from 4 commits
5e47e97
3d85bf0
3ebbb8c
2bd6913
fe029fe
a838a9c
6d14ece
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. there are a couple other places where pyiceberg.yaml is referenced in the docs There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nice finds! That grep tool is pretty neat. I just made some corrections to these in 3ebbb8c. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -28,7 +28,7 @@ hide: | |
|
||
There are three ways to pass in configuration: | ||
|
||
- Using the `~/.pyiceberg.yaml` configuration file | ||
- Using the `.pyiceberg.yaml` configuration file stored in either the directory specified by the `PYICEBERG_HOME` environment variable, the home directory, or current working directory. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: move the extra info about where the file is located down to L37. |
||
- Through environment variables | ||
- By passing in credentials through the CLI or the Python API | ||
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried adding a test here, but I wonder if there are opportunities to clean it up. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,6 +15,7 @@ | |
# specific language governing permissions and limitations | ||
# under the License. | ||
import os | ||
from typing import Any, Dict, Optional | ||
from unittest import mock | ||
|
||
import pytest | ||
|
@@ -93,3 +94,61 @@ def test_from_configuration_files_get_typed_value(tmp_path_factory: pytest.TempP | |
|
||
assert Config().get_bool("legacy-current-snapshot-id") | ||
assert Config().get_int("max-workers") == 4 | ||
|
||
|
||
@pytest.mark.parametrize( | ||
"config_location, config_content, expected_result", | ||
Fokko marked this conversation as resolved.
Show resolved
Hide resolved
|
||
[ | ||
( | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: for test readability, use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. looks like the parameterize test is testing
i'd add a test for all 3 |
||
"config", | ||
{"catalog": {"default": {"uri": "https://service.io/api"}}}, | ||
{"catalog": {"default": {"uri": "https://service.io/api"}}}, | ||
), | ||
( | ||
"home", | ||
{"catalog": {"default": {"uri": "https://service.io/api"}}}, | ||
{"catalog": {"default": {"uri": "https://service.io/api"}}}, | ||
), | ||
( | ||
"current", | ||
{"catalog": {"default": {"uri": "https://service.io/api"}}}, | ||
{"catalog": {"default": {"uri": "https://service.io/api"}}}, | ||
), | ||
("none", None, None), | ||
], | ||
) | ||
def test_from_multiple_configuration_files( | ||
IndexSeek marked this conversation as resolved.
Show resolved
Hide resolved
|
||
monkeypatch: pytest.MonkeyPatch, | ||
tmp_path_factory: pytest.TempPathFactory, | ||
config_location: str, | ||
config_content: Optional[Dict[str, Any]], | ||
expected_result: Optional[Dict[str, Any]], | ||
) -> None: | ||
def create_config_file(directory: str, content: Optional[Dict[str, Any]]) -> None: | ||
config_file_path = os.path.join(directory, ".pyiceberg.yaml") | ||
with open(config_file_path, "w", encoding="utf-8") as file: | ||
IndexSeek marked this conversation as resolved.
Show resolved
Hide resolved
|
||
yaml_str = as_document(content).as_yaml() if content else "" | ||
file.write(yaml_str) | ||
|
||
config_path = str(tmp_path_factory.mktemp("config")) | ||
home_path = str(tmp_path_factory.mktemp("home")) | ||
current_path = str(tmp_path_factory.mktemp("current")) | ||
|
||
location_to_path = { | ||
"config": config_path, | ||
"home": home_path, | ||
"current": current_path, | ||
} | ||
|
||
if config_location in location_to_path and config_content: | ||
create_config_file(location_to_path[config_location], config_content) | ||
|
||
monkeypatch.setenv("PYICEBERG_HOME", config_path) | ||
monkeypatch.setattr(os.path, "expanduser", lambda _: home_path) | ||
|
||
if config_location == "current": | ||
monkeypatch.chdir(current_path) | ||
|
||
assert Config()._from_configuration_files() == expected_result, ( | ||
f"Unexpected configuration result for content: {config_content}" | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: include warning about accidentally checking in secrets with git when using the current working directory