Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancements to the costpuller tool #1

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

webbnh
Copy link

@webbnh webbnh commented Aug 5, 2024

This PR represents a number of enhancements to the tool. While it may not be apparent from the diffs, I've tried to retain the original capabilities of the tool while adding new ones and preparing the code for further enhancement.

The major enhancement is integration with Google Sheets to allow the downloaded data to be uploaded to a new sheet in the existing Developer Cloud Expenses spreadsheet, including handling the OAuth 2.0 authentication flow required for access. With this change, the previous manual steps of producing a CSV file, creating a new sheet, pasting and formatting the data from the file into the sheet, and updating the main sheet are no longer required.

The other significant change is a reorganization of the accounts YAML file. Previously, there were only two levels of keys: groups and account IDs. This PR introduces two more levels. Groups are now placed under keys for cloud providers under the root-level cloud_providers key. Also at the root, there is a configuration key which holds information about access to OAuth authorization, individual cloud providers, and Google Sheets.

I added a command line option, -output to allow control over whether the tool produces a local CSV file or uploads directly into GSheets. I also modified the defaults for the options to allow them to be omitted in the typical usage case: by default, the tool pulls data from the previous month and loads it into the configured GSheets spreadsheet.

In addition to the above, I've made a number of smaller changes, including:

  • picked lint and wrapped long lines
  • upgraded to Go 1.22
  • corrected accesses to the kooky package
  • reworked the AWS NormalizeResponse() to produce typed output
  • put the command line options into a single structure to make them easier to pass around
  • refactored the main function, moving self-contained code blobs to their own functions and better isolating independent functions.
  • enhanced the creation of the AwsPuller to allow it to access an explicit AWS profile in the default credentials file
  • relocated the creation of the CSV file so that it isn't created when we're not going to write to it
  • made the output less verbose
  • alphabetized the .gitignore file and added entries

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant