Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server-utils): add logging configuration to server_utils to be used by servers. #13787

Closed
wants to merge 1 commit into from

Conversation

vegano1
Copy link
Contributor

@vegano1 vegano1 commented Oct 13, 2023

Overview

We want to split up the hardware_controller from the opentrons package, we are configuring the logs for each server independently. We would like to have that configuration in a common location, so let's throw it into server-utils and some other useful variables.

Closes: RET-1381

Test Plan

  • Make sure buildroot builds succeed with this change
  • Make sure oe-core builds succeed with this change
  • Make sure the OT-2 and the Flex boot and all servers are started correctly
  • Make sure we are correctly logging for each server

Changelog

  • Move common logging config code out of each server and keep it in server-utils
  • Import server-utils to update/robot/system/ot3usb servers to configure logging
  • Adds pydantic to update-server Pipfile to support ServerSettings

Review requests

Risk assessment

TODO

  • Fix all the tests

@vegano1 vegano1 requested review from a team as code owners October 13, 2023 21:29
@vegano1 vegano1 requested review from sfoster1 and fsinapi October 13, 2023 21:30
@codecov
Copy link

codecov bot commented Oct 13, 2023

Codecov Report

Merging #13787 (8ee97dd) into edge (981b3cb) will increase coverage by 0.00%.
Report is 12 commits behind head on edge.
The diff coverage is 47.05%.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             edge   #13787   +/-   ##
=======================================
  Coverage   70.95%   70.96%           
=======================================
  Files        2453     2453           
  Lines       69023    69009   -14     
  Branches     8257     8257           
=======================================
- Hits        48977    48971    -6     
+ Misses      18094    18086    -8     
  Partials     1952     1952           
Flag Coverage Δ
notify-server 89.13% <ø> (ø)
system-server 96.43% <100.00%> (+0.40%) ⬆️
update-server 64.00% <0.00%> (+0.29%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
api/src/opentrons/__init__.py 51.16% <ø> (ø)
robot-server/robot_server/app_setup.py 95.65% <ø> (ø)
robot-server/robot_server/settings.py 100.00% <ø> (ø)
robot-server/robot_server/system/time_utils.py 83.33% <ø> (-0.39%) ⬇️
system-server/system_server/__main__.py 100.00% <100.00%> (ø)
system-server/system_server/app_setup.py 95.83% <100.00%> (+0.59%) ⬆️
system-server/system_server/settings/settings.py 95.65% <100.00%> (+0.19%) ⬆️
system-server/system_server/systemd.py 60.00% <100.00%> (-10.00%) ⬇️
update-server/otupdate/common/systemd.py 0.00% <0.00%> (ø)
update-server/otupdate/buildroot/__main__.py 0.00% <0.00%> (ø)
... and 1 more

Copy link
Contributor

@fsinapi fsinapi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My initial thought is that, if we're just going to have hard-coded configurations in here, there's little benefit over just keeping everything hard-coded in each package. I think we should either

  1. Set up a single, common config that applies to everything (including the full breadth of loggers/handlers)
  2. Have a function that can be called with a set of configurable handlers & loggers to be supplied by the caller

...or option 3 is that this isn't a useful refactor because it's a minimal amount of duplicated code to set this up independently in each server. I'll leave that up to your judgement 😄

from .constants import *
from .logging import log_init

print("Im importing server_utils")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debugging line?

Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inline comments, and I think I'm on Frank's side that this isn't quite a useful enough refactor yet. Specifically, it's weird to me that we have these fully qualified package names in server utils, you know? The configurations are so incredibly specific to these other servers that it's strange to have them in server-utils which is in theory a common-use library.

The thing is though that if we make this general enough that server-utils doesn't contain fully qualified package names (which I think is a good bar for "general enough"), then what does it contain on the logging front? It's already pretty terse.

I like getting config out of the opentrons package but as discussed in person I think that's only useful if we can keep opentrons from having a dependency on server_utils, which I think needs a little more refactoring before it can happen.

from .constants import *
from .logging import log_init

print("Im importing server_utils")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug print

@@ -140,6 +140,7 @@ async def initialize() -> ThreadManagedHardware:
"""
Initialize the Opentrons hardware returning a hardware instance.
"""
# TODO (ba, 2023-10-12): Remove log init once the opentrons_hardware process is up and running
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could remove it now if we put the logging config for the hardware controller in server utils



if __name__ == "__main__":
args = build_root_parser().parse_args()
systemd.configure_logging(level=args.log_level.upper())
LOG.info(f"Starting system server on {args.host}:{args.port}")
log.info(f"Starting system server on {args.host}:{args.port}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this now won't be logged, right? since the logging is set up when we invoke uvicorn now

@@ -6,6 +6,8 @@ name = "pypi"
[packages]
aiohttp = "==3.4.4"
typing-extensions = "==3.10.0.0"
pydantic = "==1.8.2"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this necessary? on the flex, that's a full second copy of pydantic and all its transitive deps in update-server

@vegano1 vegano1 closed this Oct 18, 2023
@vegano1
Copy link
Contributor Author

vegano1 commented Oct 18, 2023

Agreed,
The only required change to move the logging config will happen once the hardware controller is in its own package.
So for now I think ill skip this refactor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants