Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: strict tsconfig and use threadedClass #60

Merged
merged 70 commits into from
Jan 17, 2020
Merged

Conversation

Julusian
Copy link
Member

@Julusian Julusian commented Oct 8, 2019

  • Remaining
  • Parse/remove the newly added properties to the AtemCapabilites object.
  • How should we handle applyToState failing? That be ignored or logged in a way that doesnt trigger an error in cases that are expected (ie, no dve and jibberish data)
  • Use release version of threadedClass

Some non-blocking tasks moved to #73

  • What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

This enables some stricter typescript rules. Primarily to enable strictPropertyInitialization to ensure that all class properties are initialised before they can be used.
Additionally, fork has been replaced with threadedClass which is more durable and allows for using workers instead of fork when using recent enough node.

The connection handling logic has been thoroughly reviewed and many bugs have been fixed. There were some problems with retransmits when the packet id would wrap, which was causing instability with media upload, and other possible connection deadlocks if the connection is unstable and started to hit some timeouts.

Data parsing has been loosened, as we cannot guarantee data ranges, and that they wont change. We now parse the command, and then decide whether to apply it to the state based on whether the ids are within known good ranges for the current device. We assume that any non-id value received is valid, and we assume all values in outbound commands are within range (including ids). This should fix #24 #15 #61 and prevent similar issues cropping up in future.

  • What is the current behavior? (You can also link to an open issue here)

Typescript is a bit loose in its checks, so some fields can be undefined even though their typings do not indicate that.

Using fork to open the child process has caused some stability problems, with it getting orphaned under certain circumstances. #30 #40 #68 #65 (possibly more)

  • Other information:

This is a massive overhaul of a majority of the library.
Every command class had to be refactored to allow the typings to work. AbstractCommand has been replaced with a couple of separate base classes & interfaces. Different for each of serializable and deserializable commands.
Other benefits of this is that various properties are now readonly when possible, and there is less dependencies on ensuring 'random' properties are set at the right points.

Threadedclass does come with 4-5% more cpu usage (19% vs 14%) than the fork implementation while doing still media uploads. But with that we get a safer wrapper around the child process, both in typings and in durability. The performance drop hasnt been investigated fully yet, but that could be done if this is found to be a problem.

Additionally, media is now a lot quicker to upload than previously.
in node 8 with fork I was getting ~1400ms per frame, and that is down to ~1000ms when running this in node 12. node8 still performs around ~1400ms, the benefit here comes only when running newer node.

@Julusian Julusian requested a review from mint-dewit October 8, 2019 10:52
src/enums/index.ts Outdated Show resolved Hide resolved
@Julusian Julusian changed the base branch from master to develop November 29, 2019 12:24
@codecov-io
Copy link

codecov-io commented Dec 1, 2019

Codecov Report

Merging #60 into develop will increase coverage by <.01%.
The diff coverage is 87.5%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop      #60      +/-   ##
===========================================
+ Coverage       87%   87.01%   +<.01%     
===========================================
  Files          112      113       +1     
  Lines         2701     2733      +32     
  Branches       380      409      +29     
===========================================
+ Hits          2350     2378      +28     
- Misses         345      348       +3     
- Partials         6        7       +1
Impacted Files Coverage Δ
src/atem.ts 29.06% <0%> (-0.41%) ⬇️
src/index.ts 100% <100%> (ø) ⬆️
src/commands/TimeCommand.ts 100% <100%> (ø)
src/enums/index.ts 100% <100%> (ø) ⬆️
src/commands/CommandBase.ts 100% <100%> (ø) ⬆️
src/commands/index.ts 100% <100%> (ø) ⬆️
src/lib/tally.ts 93.47% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77ef0d1...cdad157. Read the comment docs.

@Julusian Julusian force-pushed the feat/strict-tsconfig branch from c9bac04 to 3b79a9e Compare December 4, 2019 23:09
… mask). Returns true if anything was changed
@mint-dewit
Copy link
Contributor

How should we handle applyToState failing?

I think we should follow the idea that we should only error for a case that we did not expect to find. So in the case for a device not having a DVE that should be handled / ignored, possibly applyToState shouldn't even throw. In any other case it's unexpected and should be logged as an error.

I am wondering if AtemState and its children should be all interfaces, this way we can allow/support the state to be cloned by whatever method desired without the methods disappearing off it

We could do that, but at the same time it's not a requirement for any of the current users. We could also wait until someone needs to do that. I imagine it can be done in a non-breaking way

@Julusian
Copy link
Member Author

How should we handle applyToState failing?

I think we should follow the idea that we should only error for a case that we did not expect to find. So in the case for a device not having a DVE that should be handled / ignored, possibly applyToState shouldn't even throw. In any other case it's unexpected and should be logged as an error.

yeah, so should that be a warn/info/log/debug/trace level message or not at all logged?
At the very least I need to create a new error type for it so that we can easily filter it out and not ignore real errors.

I am wondering if AtemState and its children should be all interfaces, this way we can allow/support the state to be cloned by whatever method desired without the methods disappearing off it

We could do that, but at the same time it's not a requirement for any of the current users. We could also wait until someone needs to do that. I imagine it can be done in a non-breaking way

I would like to throw in the unit-tests as a 'user' of this requirement.
There is currently a bunch of use of class-transformer to convert states stored as json files for tests into the real classes.
https://github.com/TipoftheHats/atem-compositor is another example. In that the state is sent over IPC into the ui. I dont expect this to be an uncommon use case.

@mint-dewit
Copy link
Contributor

yeah, so should DVE related failures be a warn/info/log/debug/trace level message or not at all logged?

It makes sense to add debug logging for it, I imagine a scenario where someone wants to make sure the handling of those failures is or is not the root cause of some bug they are investigating and in that case it would be nice to have that logged.

If there are current usecases for the serializing/deserializing of state object I guess it's a good idea to make that possible. I think we could either define the interfaces, have classes with initializers and move out the helper methods to be utility functions or define everything as interfaces and have a utility function for generating a state with initialized values.

@Julusian
Copy link
Member Author

@baltedewit sorry for the large touching every command changes happening again.

This now has the state purely as interfaces, with AtemStateUtil containing all the get* methods we had scattered on the state before. I am not 100% sure this is the best right now, I shall update atem-state to this to check it feels ok from another library (it may not).

Then I have made a custom Error type for the expected invalid id errors thrown in applyToState. And changed how logs are passed out of the library. It now has 3 levels of emit for logs ('error', 'info' and 'debug')

@Julusian
Copy link
Member Author

Change pushed to atem-state to match this state refactor.
I think it is ok actually. It was pretty painless to convert to.

@mint-dewit
Copy link
Contributor

Both changes look good to me!

@Julusian Julusian merged commit a830261 into develop Jan 17, 2020
@Julusian Julusian deleted the feat/strict-tsconfig branch January 17, 2020 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants