feat: Emit management canister idl during build when imported by Motoko canister

CHANGELOG.md

@@ -2,6 +2,12 @@
 
 # UNRELEASED
 
+### feat: emit management canister idl when imported by Motoko canister
+
+`import management "ic:aaaaa-aa;`
+
+This will automatically produce the idl in the `.dfx` folder.
+
 ### fix: Include remote canisters in canisters_to_generate
 
 Generate frontend declarations for remote canisters too because frontend JS code may want to call them.
@@ -326,7 +332,7 @@ Previously, it would only retry when waiting for the request to complete.
 
 ### fix: now considers fewer error types to be retryable
 
-Previously, errors were assumed to be retryable, except for a few specific error messages and 403/unauthorized responses.  This could cause deployment to appear to hang until timeout.  
+Previously, errors were assumed to be retryable, except for a few specific error messages and 403/unauthorized responses.  This could cause deployment to appear to hang until timeout.
 
 Now, only transport errors and timeout errors are considered retryable.
 
@@ -381,7 +387,7 @@ When running `dfx canister delete` on a canister that has not been stopped, dfx
 
 ### feat: gzip option in dfx.json
 
-`dfx` can gzip wasm module as the final step in building canisters. 
+`dfx` can gzip wasm module as the final step in building canisters.
 
 This behavior is disabled by default.
 
@@ -471,7 +477,7 @@ Previously, it would only retry when waiting for the request to complete.
 
 ### fix: now considers fewer error types to be retryable
 
-Previously, errors were assumed to be retryable, except for a few specific error messages and 403/unauthorized responses.  This could cause deployment to appear to hang until timeout.  
+Previously, errors were assumed to be retryable, except for a few specific error messages and 403/unauthorized responses.  This could cause deployment to appear to hang until timeout.
 
 Now, only transport errors and timeout errors are considered retryable.
 
@@ -570,20 +576,20 @@ When creating a canister on non-mainnet replica, you can now specify the caniste
 `dfx deploy <CANISTER_NAME> --specified-id <PRINCIPAL>`
 
 You can specify the ID in the range of `[0, u64::MAX / 2]`.
-If not specify the ID, the canister will be created in the range of `[u64::MAX / 2 + 1, u64::MAX]`. 
+If not specify the ID, the canister will be created in the range of `[u64::MAX / 2 + 1, u64::MAX]`.
 This canister ID allocation behavior only applies to the replica, not the emulator (ic-ref).
 
 ### feat: dfx nns install --ledger-accounts
 
-`dfx nns install` now takes an option `--ledger-accounts` to initialize the ledger canister with these accounts. 
+`dfx nns install` now takes an option `--ledger-accounts` to initialize the ledger canister with these accounts.
 
 ### fix: update Rust canister template.
 
 `ic-cdk-timers` is included in the dependencies.
 
 ### chore: change the default Internet Computer gateway domain to `icp0.io`
 
-By default, DFX now uses the `icp0.io` domain to connect to Internet Computer as opposed to using `ic0.app`. 
+By default, DFX now uses the `icp0.io` domain to connect to Internet Computer as opposed to using `ic0.app`.
 Canisters communicating with `ic0.app` will continue to function nominally.
 
 ### feat: --no-asset-upgrade
@@ -762,7 +768,7 @@ This is no longer the case.  See rules above for grant_permission and revoke_per
 
 ### feat(frontend-canister)!: default secure configuration for assets in frontend project template
 
-- Secure HTTP headers, preventing several typical security vulnerabilities (e.g. XSS, clickjacking, and many more). For more details, see comments in `headers` section in [default `.ic-assets.json5`](https://raw.githubusercontent.com/dfinity/sdk/master/src/dfx/assets/new_project_node_files/src/__project_name___frontend/src/.ic-assets.json5). 
+- Secure HTTP headers, preventing several typical security vulnerabilities (e.g. XSS, clickjacking, and many more). For more details, see comments in `headers` section in [default `.ic-assets.json5`](https://raw.githubusercontent.com/dfinity/sdk/master/src/dfx/assets/new_project_node_files/src/__project_name___frontend/src/.ic-assets.json5).
 - Configures `allow_raw_access` option in starter `.ic-assets.json5` config files, with the value set to its default value (which is `false`). We are showing that configuration in the default starter projects for the sake of easier discoverability, even though its value is set to the default.
 
 ### feat(frontend-canister)!: add `allow_raw_access` config option
@@ -780,12 +786,12 @@ By default, the frontend canister will now restrict the access of traffic to the
 **Important**: Note that any assets already uploaded to an asset canister will be protected by this redirection, because at present the asset synchronization process does not update the `allow_raw_access` property, or any other properties, after creating an asset.  This also applies to assets that are deployed without any configuration, and later configured to allow raw access.
 At the present time, there are two ways to reconfigure an existing asset:
 1. re-create the asset
-    1. delete the asset in your project's directory 
+    1. delete the asset in your project's directory
     1. execute `dfx deploy`
     1. re-create the asset in your project's directory
-    1. modify `.ic-assets.json` acordingly 
+    1. modify `.ic-assets.json` acordingly
     1. execute `dfx deploy`
-2. via manual candid call 
+2. via manual candid call
     ```
     dfx canister call PROJECT_NAME_frontend set_asset_properties '( record { key="/robots.txt"; allow_raw_access=opt(opt(true)) })'
     ```
@@ -801,7 +807,7 @@ Callable only by a controller.  Clears list of authorized principals and adds th
 
 ### feat(frontend-canister): add `get_asset_properties` and `set_asset_properties` to frontend canister
 
-As part of creating the support for future work, it's now possible to get and set AssetProperties for assets in frontend canister. 
+As part of creating the support for future work, it's now possible to get and set AssetProperties for assets in frontend canister.
 
 ### feat: add `--argument-file` argument to the `dfx canister sign` command
 
@@ -912,7 +918,7 @@ This incorporates the following executed proposals:
 
 ### feat(frontend-canister): add warning if config is provided in `.ic-assets.json` but not used
 
-### fix(frontend-canister): Allow overwriting default HTTP Headers for assets in frontend canister 
+### fix(frontend-canister): Allow overwriting default HTTP Headers for assets in frontend canister
 
 Allows to overwrite `Content-Type`, `Content-Encoding`, and `Cache-Control` HTTP headers with custom values via `.ic-assets.json5` config file. Example `.ic-assets.json5` file:
 ```json5
@@ -926,7 +932,7 @@ Allows to overwrite `Content-Type`, `Content-Encoding`, and `Cache-Control` HTTP
     }
 ]
 ```
-This change will trigger the update process for frontend canister (new module hash: `2ff0513123f11c57716d889ca487083fac7d94a4c9434d5879f8d0342ad9d759`). 
+This change will trigger the update process for frontend canister (new module hash: `2ff0513123f11c57716d889ca487083fac7d94a4c9434d5879f8d0342ad9d759`).
 
 ### feat: warn if an unencrypted identity is used on mainnet
 
@@ -1151,12 +1157,12 @@ The running replica port and url are generally useful information. Previously th
 
 Instead, the build process relies on `ic-wasm` to provide candid metadata for the canister, and
 shrinking the canister size by stripping debug symbols and unused fuctions.
-Additionally, after build step, the `.wasm` file is archived with `gzip`. 
+Additionally, after build step, the `.wasm` file is archived with `gzip`.
 
 ### chore: Move all `frontend canister`-related code into the SDK repo
 
 | from (`repository` `path`)                  | to (path in `dfinity/sdk` repository)          | summary                                                                                     |
-|:--------------------------------------------|:-----------------------------------------------|:--------------------------------------------------------------------------------------------|
+| :------------------------------------------ | :--------------------------------------------- | :------------------------------------------------------------------------------------------ |
 | `dfinity/cdk-rs` `/src/ic-certified-assets` | `/src/canisters/frontend/ic-certified-asset`   | the core of the frontend canister                                                           |
 | `dfinity/certified-assets` `/`              | `/src/canisters/frontend/ic-frontend-canister` | wraps `ic-certified-assets` to build the canister wasm                                      |
 | `dfinity/agent-rs` `/ic-asset`              | `/src/canisters/frontend/ic-asset`             | library facilitating interactions with frontend canister (e.g. uploading or listing assets) |
@@ -1173,7 +1179,7 @@ as if they were [JSON5](https://json5.org/) format. Example content of the `.ic-
     "match": "*", // comment
     /*
     keys below not wrapped in quotes
-*/  cache: { max_age: 999 }, // trailing comma 
+*/  cache: { max_age: 999 }, // trailing comma
   },
 ]
 ```

docs/cli-reference/dfx-build.md

@@ -16,16 +16,16 @@ dfx build [flag] [option] [--all | canister_name]
 
 You can use the following optional flags with the `dfx build` command.
 
-| Flag              | Description                                                                                                                                                      |
-|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `--check`         | Builds canisters using a temporary, hard-coded, locally-defined canister identifier for testing that your program compiles without connecting to the IC. |
+| Flag      | Description                                                                                                                                              |
+| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `--check` | Builds canisters using a temporary, hard-coded, locally-defined canister identifier for testing that your program compiles without connecting to the IC. |
 
 ## Options
 
 You can specify the following options for the `dfx build` command.
 
 | Option                | Description                                                                                                                                                |
-|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `--network <network>` | Specifies the network alias or URL you want to connect to. You can use this option to override the network specified in the `dfx.json` configuration file. |
 | `--output-env-file`   | Writes dfx environment variables to a provided path. Overrides the `output_env_file` configuration from `dfx.json` if passed.                              |
 
@@ -34,7 +34,7 @@ You can specify the following options for the `dfx build` command.
 You can specify the following arguments for the `dfx build` command.
 
 | Argument        | Description                                                                                                                                                                                                                                                                                                                              |
-|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `--all`         | Builds all of the canisters configured in the project’s `dfx.json` file.                                                                                                                                                                                                                                                                 |
 | `canister_name` | Specifies the name of the canister you want to build. If you are not using the `--all` option, you can continue to use `dfx build` or provide a canister name as an argument (the canister name must match at least one name that you have configured in the `canisters` section of the `dfx.json` configuration file for your project.) |
 
@@ -57,3 +57,7 @@ To test whether a canister compiles without connecting to the IC or the local ca
 ``` bash
 dfx build --check
 ```
+
+## Management Canister
+
+If `dfx` detects that your Motoko project is importing the Management Canister (e.g. `import Management "ic:aaaaa-aa";`) it will automatically provide the Candid interface for the Management Canister during the build.

e2e/assets/motoko_management/main.mo

@@ -0,0 +1,7 @@
+import Management "ic:aaaaa-aa";
+
+actor {
+  public func rand() : async Blob {
+    await Management.raw_rand();
+  };
+};

e2e/assets/motoko_management/patch.bash

@@ -0,0 +1 @@
+jq '.canisters.e2e_project_backend.main="main.mo"' dfx.json | sponge dfx.json

e2e/assets/motoko_management_recursive/dependency.mo

@@ -0,0 +1,7 @@
+import Management "ic:aaaaa-aa";
+
+module Rand {
+  public func rand() : async Blob {
+    await Management.raw_rand();
+  };
+};

e2e/assets/motoko_management_recursive/main.mo

@@ -0,0 +1,7 @@
+import Rand "dependency";
+
+actor {
+  public func rand() : async Blob {
+    await Rand.rand();
+  };
+};

e2e/assets/motoko_management_recursive/patch.bash

@@ -0,0 +1 @@
+jq '.canisters.e2e_project_backend.main="main.mo"' dfx.json | sponge dfx.json

e2e/tests-dfx/build.bash

@@ -106,6 +106,24 @@ teardown() {
   assert_match "10World"
 }
 
+@test "build supports auto-generated idl for management canister imports in motoko" {
+    install_asset motoko_management
+    dfx_start
+    dfx canister create --all
+    assert_command dfx build
+    dfx deploy
+    assert_command dfx canister call e2e_project_backend rand
+}
+
+@test "build supports auto-generated idl for recursive management canister imports in motoko" {
+    install_asset motoko_management_recursive
+    dfx_start
+    dfx canister create --all
+    assert_command dfx build
+    dfx deploy
+    assert_command dfx canister call e2e_project_backend rand
+}
+
 @test "build succeeds on default project" {
   dfx_start
   dfx canister create --all

src/dfx/src/lib/builders/motoko.rs

@@ -8,6 +8,7 @@ use crate::lib::error::{BuildError, DfxError, DfxResult};
 use crate::lib::metadata::names::{CANDID_ARGS, CANDID_SERVICE};
 use crate::lib::models::canister::CanisterPool;
 use crate::lib::package_arguments::{self, PackageArguments};
+use crate::util::assets::management_idl;
 use anyhow::Context;
 use candid::Principal as CanisterId;
 use dfx_core::config::cache::Cache;
@@ -38,59 +39,59 @@ impl MotokoBuilder {
     }
 }
 
+#[context("Failed to find imports for canister at '{}'.", file.to_string_lossy())]
+fn get_imports(cache: &dyn Cache, file: &Path) -> DfxResult<BTreeSet<MotokoImport>> {
+    #[context("Failed recursive dependency detection at {}.", file.to_string_lossy())]
+    fn get_imports_recursive(
+        cache: &dyn Cache,
+        file: &Path,
+        result: &mut BTreeSet<MotokoImport>,
+    ) -> DfxResult {
+        if result.contains(&MotokoImport::Relative(file.to_path_buf())) {
+            return Ok(());
+        }
+
+        result.insert(MotokoImport::Relative(file.to_path_buf()));
+
+        let mut command = cache.get_binary_command("moc")?;
+        let command = command.arg("--print-deps").arg(file);
+        let output = command
+            .output()
+            .with_context(|| format!("Error executing {:#?}", command))?;
+        let output = String::from_utf8_lossy(&output.stdout);
+
+        for line in output.lines() {
+            let import = MotokoImport::try_from(line).context("Failed to create MotokoImport.")?;
+            match import {
+                MotokoImport::Relative(path) => {
+                    get_imports_recursive(cache, path.as_path(), result)?;
+                }
+                _ => {
+                    result.insert(import);
+                }
+            }
+        }
+
+        Ok(())
+    }
+
+    let mut result = BTreeSet::new();
+    get_imports_recursive(cache, file, &mut result)?;
+
+    Ok(result)
+}
+
 impl CanisterBuilder for MotokoBuilder {
     #[context("Failed to get dependencies for canister '{}'.", info.get_name())]
     fn get_dependencies(
         &self,
         pool: &CanisterPool,
         info: &CanisterInfo,
     ) -> DfxResult<Vec<CanisterId>> {
-        let mut result = BTreeSet::new();
         let motoko_info = info.as_info::<MotokoCanisterInfo>()?;
+        let imports = get_imports(self.cache.as_ref(), motoko_info.get_main_path())?;
 
-        #[context("Failed recursive dependency detection at {}.", file.to_string_lossy())]
-        fn find_deps_recursive(
-            cache: &dyn Cache,
-            file: &Path,
-            result: &mut BTreeSet<MotokoImport>,
-        ) -> DfxResult {
-            if result.contains(&MotokoImport::Relative(file.to_path_buf())) {
-                return Ok(());
-            }
-
-            result.insert(MotokoImport::Relative(file.to_path_buf()));
-
-            let mut command = cache.get_binary_command("moc")?;
-            let command = command.arg("--print-deps").arg(file);
-            let output = command
-                .output()
-                .with_context(|| format!("Error executing {:#?}", command))?;
-
-            let output = String::from_utf8_lossy(&output.stdout);
-            for line in output.lines() {
-                let import =
-                    MotokoImport::try_from(line).context("Failed to create MotokoImport.")?;
-                match import {
-                    MotokoImport::Canister(_) => {
-                        result.insert(import);
-                    }
-                    MotokoImport::Relative(path) => {
-                        find_deps_recursive(cache, path.as_path(), result)?;
-                    }
-                    MotokoImport::Lib(_) => (),
-                    MotokoImport::Ic(_) => (),
-                }
-            }
-
-            Ok(())
-        }
-        find_deps_recursive(
-            self.cache.as_ref(),
-            motoko_info.get_main_path(),
-            &mut result,
-        )?;
-
-        Ok(result
+        Ok(imports
             .iter()
             .filter_map(|import| {
                 if let MotokoImport::Canister(name) = import {
@@ -132,6 +133,14 @@ impl CanisterBuilder for MotokoBuilder {
         std::fs::create_dir_all(idl_dir_path)
             .with_context(|| format!("Failed to create {}.", idl_dir_path.to_string_lossy()))?;
 
+        // If the management canister is being imported, emit the candid file.
+        if get_imports(cache.as_ref(), input_path)?
+            .contains(&MotokoImport::Ic("aaaaa-aa".to_string()))
+        {
+            let management_idl_path = idl_dir_path.join("aaaaa-aa.did");
+            dfx_core::fs::write(management_idl_path, management_idl()?)?;
+        }
+
         let package_arguments =
             package_arguments::load(cache.as_ref(), motoko_info.get_packtool())?;