Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a windows_dev.md document for buildkit developers on Windows #5291

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

danielGithinji
Copy link

This PR adds a windows_dev.md file to the docs folder. This file contains documentation to help buildkit developers navigate the development of buildkit on Windows. This 1st version adds documentation on how to run integration tests.

docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
@danielGithinji danielGithinji marked this pull request as ready for review September 5, 2024 12:30
Copy link
Collaborator

@profnandaa profnandaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just those last comments and LGTM.

docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Outdated Show resolved Hide resolved
docs/windows_dev.md Show resolved Hide resolved
Copy link
Collaborator

@profnandaa profnandaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please squash the commits into one.

@danielGithinji danielGithinji force-pushed the create-windows-dev-doc branch 3 times, most recently from abff257 to 4f81f3c Compare October 2, 2024 07:13
@profnandaa
Copy link
Collaborator

/cc. @crazy-max

@castrombithisamm
Copy link
Contributor

castrombithisamm commented Oct 3, 2024

I am following these steps to setup on my local machine, and there is a place where you said, "Remove any other buildkit binary available in $env:PATH". When I cd $env:GOPATH\bin and dir into $env:GOPATH\bin directory, I get some files there:

- -a----         10/1/2024   5:23 PM       28587520 buildctl.exe
- -a----         10/1/2024   5:23 PM       56246272 buildkitd.exe
- -a----         5/14/2024   5:31 PM       19312128 dlv.exe
- -a----         4/19/2024   8:30 AM       31768576 gopls.exe
- -a----         9/30/2024   4:30 PM       52032512 registry.exe
- -a----         4/19/2024   8:32 AM       13856256 staticcheck.exe

my question is which ones should I remove?

@danielGithinji
Copy link
Author

danielGithinji commented Oct 8, 2024

Hi @castrombithisamm . The files you should remove or rename is both buildkitctl.exe and buildkitd.exe

@castrombithisamm
Copy link
Contributor

Hi @danielGithinji thank you for getting back, so that means I should delete them from the $env:GOPATH\bin directory, and only be left with these? Then after that, I can run the test. right?

- -a----         5/14/2024   5:31 PM       19312128 dlv.exe
- -a----         4/19/2024   8:30 AM       31768576 gopls.exe
- -a----         9/30/2024   4:30 PM       52032512 registry.exe
- -a----         4/19/2024   8:32 AM       13856256 staticcheck.exe

@danielGithinji
Copy link
Author

danielGithinji commented Oct 9, 2024

@castrombithisamm, when you install buildkit for the 1st time (using these instructions) its found in $env:PATH. It appears there just like any other binary you've installed in your machine.

$env:GOPATH\bin - stores binaries of go projects that you've installed from your local machine. This means that when you git clone the buildkit source code, and install it, its going to be available in $env:GOPATH\bin .

When running buildkit and in order for there to be no conflicts between the buildkit binary in your PATH i.e $env:PATH, and the one installed in $env:GOPATH\bin, you need to remove or rename the binary in your PATH. (The binary here includes both buildctl.exe and buildkitd.exe)

@castrombithisamm
Copy link
Contributor

I hear you @danielGithinji and thanks for clarifying this. I forgot to mention that my PATH $env:PATH doesn't have any buildkit binary files installed. The ones shown here were installed in $env:GOPATH\bin

PS C:\Users\awond\buildkit> cd "$env:GOPATH\bin"
PS C:\Users\awond\go\bin> dir
    Directory: C:\Users\awond\go\bin
Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         10/1/2024   5:23 PM       28587520 buildctl.exe
-a----         10/1/2024   5:23 PM       56246272 buildkitd.exe
-a----         5/14/2024   5:31 PM       19312128 dlv.exe
-a----         4/19/2024   8:30 AM       31768576 gopls.exe
-a----         9/30/2024   4:30 PM       52032512 registry.exe
-a----         4/19/2024   8:32 AM       13856256 staticcheck.exe

Maybe it would have been easier to understand if we listed all locations where the buildkit binary is available using Get-Command buildkitd Like in my case here when I checked using the command it gave me this:

PS C:\Users\awond\buildkit> Get-Command buildkitd
CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     buildkitd.exe                                      0.0.0.0    C:\Users\awond\go\bin\buildkitd.exe

I don't know if I am correct, but this means my buildkit binary is installed in $env:GOPATH\bin .

To conclude maybe we can say that we need to make sure that Get-Command buildkitd returns an equivalent of $env:GOPATH\bin\buildkit.exe for the Source field. If not, delete any other buildkitd being returned or remove them from $env:PATH

@danielGithinji
Copy link
Author

@castrombithisamm - I have updated the document based on your sentiments so that it becomes clear how someone would need to proceed in this specific scenario.

@castrombithisamm
Copy link
Contributor

@danielGithinji many thanks for incorporating my sentiments into the document

@profnandaa
Copy link
Collaborator

@crazy-max -- can take a look at this?

Comment on lines +2 to +14
**Prerequisites**:
- You've setup Go and the buildkit repo and you can build from source.
- Optional: You have installed the [Delve debugger](https://github.com/go-delve/delve)

### Main Steps
- Install `registry.exe` by running the following command:

```go install -v github.com/distribution/distribution/v3/cmd/registry@latest```
- Install buildkit with the following command:
```
cd to repo directory
go install -v .\cmd\...
```
Copy link
Member

@crazy-max crazy-max Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is necessary, you can just run:

docker buildx bake binaries-for-test --set *.platform=windows/amd64

To output dev binaries:

$ tree -nh ./bin/build
[4.0K]  ./bin/build
├── [ 29M]  buildctl.exe
├── [ 56M]  buildkitd.exe
├── [ 17M]  containerd-shim-runhcs-v1.exe
├── [ 37M]  containerd.exe
├── [2.2M]  gocovmerge.exe
├── [7.5M]  gotestsum.exe
├── [ 573]  gotestsumandcover
└── [ 23M]  registry.exe

0 directories, 8 files

Will open a PR to also include delve

So just cloning repo, installing go and command above should be enough to get started.

docs/windows.md Outdated
@@ -243,7 +243,7 @@ $networkName = 'nat'
# Get-HnsNetwork is available once you have enabled the 'Hyper-V Host Compute Service' feature
# which must have been done at the Quick setup above
# Enable-WindowsOptionalFeature -Online -FeatureName containers -All
# Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All -All
# Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not seem related to this PR, can you open a new one for this fix?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed it. I'll open a new one to fix it.

### Running the whole test-suite
Running the whole test suite can take a long time, and it's not advisable to do it on your local machine. However, if you'd like to run it, you can do so with the following command:

> Make sure to install gotestsum by running: `go install gotest.tools/gotestsum@latest`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a windows_dev.md file to the docs folder. This file is supposed
to help Windows devs navigate the development of buildkit on Windows.
This 1st version adds documentation on how to run and debug integration tests.

Signed-off-by: Daniel Githinji <[email protected]>
Signed-off-by: Daniel Githinji <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants