-
-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add or document a way to write unit tests for Godot C# projects (e.g. with nunit3) #432
Comments
Normally tests are executed directly in TestRunner process. This causes any unmanaged calls to Godot api-s to fail. So if you want to test a piece of logic without calling unmanaged Godot api-s it would work. Basic example van800/SkyOfSteel@6515c3f I have seen 2 different implementations, which overcome this limitation:
Both approaches are not easy/fast to be implemented. |
This seems like it should be a pretty important issue. Most serious projects will probably need to have unit tests, and having the standard unit testing tools available seems essential. Even for my little hobby project in Godot, I feel quite hindered by not being able to easily use standard style C# tests. It's been making me reconsider if Godot is really the engine I should be using, or if I should be switching to something that better enables regular coding practices. I can only imagine how much this deters more serious game developers from using Godot. This is probably too extreme of a blanket statement, but typically more experienced developers are more likely to rely on unit tests. And having experienced developers choose Godot as their standard tool will certainly help Godot move forward. |
This comment has been minimized.
This comment has been minimized.
@martinruefenacht Please don't bump issues without contributing significant new information. Use the 👍 reaction button on the first post instead. |
So, I'm sorry, I don't understand the release plan about this feature ?
|
So this is self-promotion but it is also very relevant to the issue at hand. I ran into this issue recently myself when I wanted to change from GDScript to C#. I decided to wrap WAT (The Godot GDScript Plugin) in C#. You can find the reddit thread with more detailed information and links here. This has a GUI, CLI, Parameterized Testing & a number of other features available. Since it is built mainly using GDScript (with C# on user-interfacing scripts) it doesn't run into the same errors as the more established C# Frameworks meaning you can test any Godot Class without worry. With that said I do think it is probably a good idea to implement some form of communication to the godot api if 1) it is relatively painless to implement 2) is generic so this fix can allow Godot to work with any standard c# test framework. |
There were a couple comments that involved something to do with the UI. I just wanted to clarify that (I think) the main part of this issue requires no sort of UI changes. It's just that a test runner currently can't call any Godot related code, and for the tests to be run, the runner needs to be able to make those calls. I suspect a majority of the people who would be writing tests would be using an external IDE anyway (though I certainly may be wrong). |
I managed to make a proof-of-concept integration Rider UnitTestRunner with Godot. With the following changes:
Still some work ahead.
If I make a PR which solves (3), would it be welcomed? @neikeq |
Yes, that would be very welcome! |
For Rider 2021.1EAP feature was merged in Rider Godot support plugin, it still requires a small preparation step - see instructions. |
@van800 It works fine for 3.2.3, but fails for 3.2.4.beta6 (using the same project), the test result is 'Inconclusive: Test not run', with GodotSharp assembly not found. |
This comment has been minimized.
This comment has been minimized.
@BenMcLean Please don't bump issues without contributing significant new information. Use the 👍 reaction button on the first post instead. |
Hi, what's up about the prob in C# unit test. |
Have you never use Unity 3D ? There is truely way, as any C# developper can expect, to unit tests : using unit tests in VS 2019. That what I need in Godot mono. :/ So for me the proposal here is not good. |
This is an issue about Assembly cross-talk. The JetBrains Rider plugin has a basic system working to handle this. Alternatively you can use WAT Mono which is a C# Unit Testing Plugin I made from the ground up to work with Godot without issue. |
Alternative based on xunit. Promising! |
One option that would go a long way for me would even to just be able to unit test a library with no Godot references in it but I'm struggling to get that setup in a way that feels natural to me because of the insistence from the editor about where the
It seemed like godot didn't mind too much where the solution file was, but the csproj must be at the same level as the godot project file. Without wanting to enforce my layout on others, it would be nice if we could at least tell Godot where to look for the csproject it needs to build. The build command itself only seems to rely on the solution file but I keep getting this error (
|
@mscharley Issues of existing c# project managed by Godot Mono Tools
I do agree, we need to continue to discuss the most optimal approach for unit testing for Godot c# developers. |
@GeorgeS2019` yes there also exists a scene runner
With next major release v2.3.0 the c# API will be feature complete and update the documentation |
For those coming here via google in 2023 because you want to run NUnit unit tests in Rider on Godot projects, this is the solution:
Thanks @van800. This is super clean. |
Is there an equivalent option for those of that use Visual Studio 2022? |
I experimented a bit and was able to create a TestAdapter based on Tests are listed based on special attribute in the main project assembly. The project also contains a .cs, that inherits a custom TestRunner class, that uses the same attribute to run tests. This is non-avoidable, I think, because we have to use Godot to actually execute stuff, so we need a script. The executor simply runs Godot without window (version does not matter), but only once. It passes list of chosen tests as argument and also the Runner script as "-s" Godot parameter. Then the Runner does its thing and, for each test, outputs some special stuff on console, which is in turn understood by the executor as the specific test result. This is a very simple solution that seems to work pretty well. No debugging, though, I'm not sure it's possible with this approach, but I've only spent two hours on this for now. Not sure if someone is interested in this.. ? I will try making a NuGet package in a few days (so there's no need for a Godot plugin), since I need a test solution myself. Also, I've no idea why, but |
@dioptryk do you have your test-framework code up anywhere? I'd love to take a look at it. |
@zorbathut I worked a bit more on it, but I ultimately understood that Godot is not really something that should be used in unit test scenario. Try to have as many types as possible not inherit from GodotObject (implement composition in your class hierarchy) and unit test that. I realized that what I really wanted are pipeline test results. And this is actually trivial, because you can generate test result xml in JUnit format with anything, be it GDScript or C#. You don't need to use XUnit, NUnit or whatever, you only need result files. So what I'm doing now is running godot --headless on the pipeline, starting some "scenario" scenes and verifying if they complete as expected. I also change the engine time multiplier for stuff to happen faster :) Then I am generating XML result file for my "scenarios" (see https://github.com/testmoapp/junitxml/tree/main) and publish it on Azure DevOps using |
Hey! Any progress on this issue? I'm currently exploring ways to run isolated tests on my scenes using xunit and the dotnet test command. Any solution that lets me test just the scene logic without the engine would be fantastic. Alternatively, knowing how to run all the necessary Godot dependencies before executing dotnet test would work too. Currently, I'm using Godot 3.5. Cheers! |
using the latest gdUnit4API TestAdaptor on VS2022 from @MikeSchulze |
C# (Scene) Test-Driven-Development (TDD) using Godot4 in VS2022 using the latest gdUnit4API TestAdaptor from @MikeSchulze Test.Driven.Development.in.Godot4.mp4 |
gdUnit4 VS integration is cool and it also turned out that it is easy to use in the JetBrains Rider. |
Now all we need is a stable version of GDUnit built into Godot. :P |
OMG, this requires a complete rewrite in c++ :) |
Can't they like publicize some of the existing test runner stuff? They already have pretty good tests for the GDScript implementation itself. https://github.com/godotengine/godot/tree/master/modules/gdscript/tests |
What do you mean publicize? |
Make it usable outside of the source code for the Godot Game engine. |
Gotcha, never heard that term used like that (was confused as the code is public) |
Describe the project you are working on:
A 3D spaceship/spacebase building game (think Space Engineers)
Describe the problem or limitation you are having in your project:
The more code you write to do custom calculations the more cumbersome it becomes to keep looking back to see if previously written code still functions as expected. Adding more code or refactoring the current code is also much easier if you can build on properly (unit)tested foundations.
As an actual example from my current project, I have to calculate how fast my spaceship can accelerate/decelerate given:
Manually testing all of these again and again (and again) is possible, however it's time consuming and inefficient.
Describe how this feature / enhancement will help you overcome this problem or limitation:
Proper unittests can save a lot of time by quickly telling me if my (new or old) code still functions as expected, thus saving me time that can be instead spent on new features.
Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:
While having a GUI is nice and all, I think CLI support is the more critical for this particular feature. Running the tests locally and in CI can eliminate further bugs while also allowing for better cooperation on the project with other contibutors.
Workflow in case it is possible to run tests directly using
nunit3-console.exe
:./path/to/nunit3-console.exe /path/to/game.dll --where "<filter for testcases>" --pause
where
option allows to filter for testcases (for when you only want to run specific ones instead of all of them)pause
option allows to attach external debuggers to debug test casesnunit3-console.exe
outputs the tests results and returns0
if all tests passed (for CI support)Workflow in case the tests need to be ran through the Godot editor's CLI:
./path/to/Godot_v3.2-stable_yourplatform --path "/path/to/game" --run-nunit3-console --where "<filter for testcases>" --pause
where
option allows to filter for testcases (for when you only want to run specific ones instead of all of them)pause
option allows to attach external debuggers to debug test casesGodot_v3.2-stable_yourplatform
outputs the tests results provided bynunit3-console.exe
and returns0
if all tests passed (for CI support)Describe implementation detail for your proposal (in code), if possible:
Unfortunately I only have limited understanding on Godot's source code or how c# (mono) support is implemented, so I do not know what needs to be implemented or changed to allow this (if anything). If using nunit3 is already possible, then this feature request is actually a request for documentation/examples.
What I do know however is that right now if you try to run your game dll with
nunit3-console.exe
your testcases will actually work until your testcase reaches the first call to Godot (more specifically to anything insideGodotSharp.dll
). You will be bombarded with error messages like this:What this error message actually means (despite what it says) is that the c# function called
Godot.NativeCalls::godot_icall_Camera_Ctor
is defined as anextern
function inGodotSharp.dll
but no actual implementation is found anywhere. The same code (same project) does work as expected when exporting/running it through Godot Editor.If this enhancement will not be used often, can it be worked around with a few lines of script?:
Not that I am aware of.
Is there a reason why this should be core and not an add-on in the asset library?:
I think at least documentation on this should be included within Godot's official documentation, even if any implementation is outside of the main source code. The reason for this is to avoid confusion by providing a singular official way for running c# unittests.
The text was updated successfully, but these errors were encountered: