diff --git a/docs/developer-docs/getting-started/install.mdx b/docs/developer-docs/getting-started/install.mdx index 8469a2110b..ce5bc0a57f 100644 --- a/docs/developer-docs/getting-started/install.mdx +++ b/docs/developer-docs/getting-started/install.mdx @@ -136,6 +136,8 @@ It also has dependencies of a few additional, important packages: - [**Candid**](/docs/current/developer-docs/smart-contracts/candid/candid-concepts): An interface description language (IDL) used to interact with a canister's methods. A **method** is a function exposed by the canister that can be called by a user, another canister, or the application's frontend. +For troubleshooting common `dfx` errors, see [IC SDK troubleshooting](troubleshooting.mdx). + ## Next step Next, you must create a developer identity. This identity will be used to control and manage your project's canisters. diff --git a/docs/developer-docs/getting-started/troubleshooting.mdx b/docs/developer-docs/getting-started/troubleshooting.mdx new file mode 100644 index 0000000000..06197041f5 --- /dev/null +++ b/docs/developer-docs/getting-started/troubleshooting.mdx @@ -0,0 +1,113 @@ +--- +keywords: [intermediate, test, tutorial, troubleshooting, dfx troubleshooting, ic sdk, dfx, ic sdk troubleshooting] +--- + +import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; + +# IC SDK troubleshooting + + + +## Overview + +This section provides information to help you troubleshoot and resolve or work around common issues that are related to the following tasks: + +- Downloading and installing the IC SDK. + +- Creating, building, or deploying canisters. + +- Using the [IC SDK](/docs/current/developer-docs/getting-started/install). + +- Running the local replica. + +## Migrating an existing project + +Currently, there is no automatic migration or backward compatibility for any projects that you might have created using previous versions of the IC SDK. After upgrading to the latest version, you might see error or failure messages if you attempt to build or install a project created with a previous version of the IC SDK. + +In many cases, however, you can continue to work with projects from a previous release by manually changing the `dfx` setting in the `dfx.json` configuration file, then rebuilding the project to be compatible with the version of the IC SDK you have currently installed. + +For example, if you have a project that was created with IC SDK version `0.8.0`, open the `dfx.json` file in a text editor and change the `dfx` setting to the latest version or remove the section entirely. + +## Restarting the local replica + +In some cases, starting the local replica fails due to stale state. If you encounter issues when running `dfx start` to start the local replica: + +- #### Step 1: Interrupt the local replica process through Ctrl+C if necessary, then stop the local replica: + +```bash +dfx stop +``` + +- #### Step 2: If not all `dfx` processes can be stopped, forcibly end them: + +``` +dfx killall +``` + +- #### Step 3: Restart the local replica in a clean state by running the following command: + +```bash +dfx start --clean --background +``` + +The `--clean` option removes checkpoints and stale state information from your project’s cache so that you can restart the local replica and web server processes in a clean state. + +:::danger +Resetting the state information by running `dfx start --clean` removes your existing canisters and new canister IDs may differ from old ones. +::: + +- #### Step 4: Recreate your canisters: + +```bash +dfx canister create --all +dfx build +dfx canister install --all +``` + +## Removing the canisters directory + +If you run into problems building or deploying canisters after successfully connecting to ICP and registering canister identifiers, you should remove the `canisters` directory before attempting to rebuild or redeploy the canisters. + +You can remove the `canisters` directory for a project by running the following command in the project’s root directory: + +```bash +rm -rf ./.dfx/* canisters/* +``` + +## Reinstalling `dfx` + +Some bugs can be addressed by uninstalling and reinstalling the IC SDK: + +```bash +~/.cache/dfinity/uninstall.sh && sh -ci "$(curl -sSL https://internetcomputer.org/install.sh)" +``` + +If you have modified the location of the IC SDK binary (the binary is titled `dfx`), you might want run the following command to uninstall the version of the IC SDK that is in your PATH, then reinstall the latest version of the IC SDK: + +```bash +rm -rf ~/.cache/dfinity && rm $(which dfx) && sh -ci "$(curl -sSL https://internetcomputer.org/install.sh)" +``` + +## Xcode prerequisite + +For macOS environments, you should have **Developer Command Line Tools** installed if you want to create a Git repository for your project. + +You can check whether you have the developer tools installed by running `xcode-select -p`. You can install the developer tools by running `xcode-select --install`. + +## Memory leak + +Fixing memory leaks is an ongoing process. If you encounter any error messages related to memory leaks, you should do the following: + +- #### Step 1: Run `dfx stop` to stop currently running processes. + +- #### Step 2: Uninstall the IC SDK to prevent further degradation. + +- #### Step 3: Re-install the IC SDK + +- #### Step 4: Run `dfx start` to restart replica processes. + +Alternatively, you can remove the `.cache/dfinity` directory and re-install the latest IC SDK `dfx` binary: + +```bash +rm -rf ~/.cache/dfinity && sh -ci "$(curl -sSL https://internetcomputer.org/install.sh)" +``` diff --git a/docs/developer-docs/smart-contracts/advanced-features/async-code.mdx b/docs/developer-docs/smart-contracts/advanced-features/async-code.mdx index 1bfdbf6f3b..ae18e42072 100644 --- a/docs/developer-docs/smart-contracts/advanced-features/async-code.mdx +++ b/docs/developer-docs/smart-contracts/advanced-features/async-code.mdx @@ -1,3 +1,13 @@ +--- +keywords: [advanced, concept, async code, inter-canister calls, async inter-canister, async] +--- + +import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; + +# Composite queries + + + # Async code and inter-canister calls ## Overview diff --git a/docs/developer-docs/smart-contracts/advanced-features/composite-query.mdx b/docs/developer-docs/smart-contracts/advanced-features/composite-query.mdx index 5c058ff694..7c2663b012 100644 --- a/docs/developer-docs/smart-contracts/advanced-features/composite-query.mdx +++ b/docs/developer-docs/smart-contracts/advanced-features/composite-query.mdx @@ -2,8 +2,11 @@ keywords: [advanced, tutorial, composite queries, queries] --- - +import TabItem from "@theme/TabItem"; +import { AdornedTabs } from "/src/components/Tabs/AdornedTabs"; +import { AdornedTab } from "/src/components/Tabs/AdornedTab"; import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; +import { GlossaryTooltip } from "/src/components/Tooltip/GlossaryTooltip"; # Composite queries @@ -12,23 +15,24 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; ## Overview The Internet Computer Protocol supports two types of messages: updates and queries. An update message is executed on all nodes and persists canister state changes. A query message discards state changes and typically executes on a single node. It is possible to execute a query message as an update. In such a case, the query still discards the state changes, but the execution happens on all nodes and the result of execution goes through consensus. This “query-as-update” execution mode is also known as replicated query. -An update can call other updates and queries. However a query cannot make any calls, which can hinder development of scalable decentralized applications (dapps), especially those that shard data across multiple canisters. +An update can call other updates and queries. However a query cannot make any calls, which can hinder development of scalable decentralized applications, especially those that shard data across multiple canisters. -Composite queries solve this problem. A composite query is a new type of query that you can add to your canister using the following annotations: +Composite queries solve this problem. You can add composite queries to your canister using the following annotations: * Candid: `composite_query` - * Azle: `$query`; in combination with `async` * Motoko: `composite query` * Rust: `#[query(composite = true)]` -Users and the client-side JavaScript code can invoke a composite query endpoint of a canister using the same query URL as for existing regular queries. In contrast to regular queries, a composite query can call other composite and regular queries. Due to limitations of the current implementation, composite queries have two restrictions: +Users and the client-side JavaScript code can invoke a composite query endpoint of a canister using the same query URL for existing regular queries. In contrast to regular queries, a composite query can call other composite and regular queries. Due to limitations of the current implementation, composite queries have two restrictions: - * A composite query cannot call canisters on another subnet. - * A composite query cannot be executed as an update. As a result, updates cannot call composite queries. +| Query | Update | Composite query | +|--------|--------|----------------| +| Cannot call other queries or composite queries| Can call other updates and queries ; Cannot call composite queries| Can call other queries and composite queries | +| Can be called as an update | Cannot be called as a query | Cannot be called as an update | +| Can call canisters on another subnet | Can call canisters on another subnet | Cannot call canisters on another subnet | -These restrictions will be hopefully lifted in future implementations. -Composite queries are enabled in the following releases: +Composite queries were enabled in the following releases: | Platform / Language | Version | | -------------------------- | ------- | @@ -36,7 +40,6 @@ Composite queries are enabled in the following releases: | Candid | [2023-06-30 (Rust 0.9.0)](https://github.com/dfinity/candid/blob/master/Changelog.md#2023-06-30-rust-090) | | Motoko | [0.9.4](https://github.com/dfinity/motoko/releases/tag/0.9.4), revision: [2d9902f](https://github.com/dfinity/motoko/commit/2d9902fb75bb04e377c28913c311aa2be373e159) | | Rust | [0.6.8](https://github.com/dfinity/cdk-rs/blob/219ae179b9c5ef0ebfff20b926a90f6624ebe704/src/ic-cdk/CHANGELOG.md#068---2022-11-28) | -| Azle | [0.11.0](https://github.com/demergent-labs/azle/releases/tag/0.11.0) | ## Sample code @@ -45,104 +48,35 @@ As an example, consider a partitioned key-value store, where a single frontend d - First, determines the ID of the data partition canister that holds the value with the given key. - Then, makes a call into the `get` or `put` function of that canister and parses the result. -Below is a simplified example of the frontend code. Take note of the line `#[query(composite = true)]` which is used to leverage the new composite query feature: - -```rust -#[query(composite = true)] -async fn frontend_get(key: u128) -> Option { - let canister_id = get_partition_for_key(key); - match call(canister_id, "get", (key, ), ).await { - Ok(r) => { - let (res,): (Option,) = r; - res - }, - Err(_) => None, - } -} -``` - -The backend simply stores the key value pairs in a `BTreeMap` in stable memory: + + -```rust -#[query] -fn get(key: u128) -> Option { - STORE.with(|store| store.borrow().get(&key)) -} +```motoko no-repl file=../../../references/samples/motoko/composite_query/src/map/Map.mo ``` -## Using composite queries -### Prerequisites - - [x] [Download and install the IC SDK.](/docs/current/developer-docs/getting-started/install) - - [x] [Download and install git.](https://git-scm.com/downloads) + + -### Setting up the example -- #### Step 1: Open a terminal window and clone the DFINITY examples repo with the command: - -```bash -git clone https://github.com/dfinity/examples.git +```rust file=../../../references/samples/rust/composite_query/src/kv_frontend/lib.rs ``` + -- #### Step 2: Navigate into the `rust/composite_query/src` directory, start a local replica, and build the frontend canister with the commands: - -```bash -cd rust/composite_query/src -dfx start -dfx canister create kv_frontend -dfx build kv_frontend -During compilation of the fronted canister, the backend canister's wasm code will be compiled and inlined in the frontend canister's wasm code. -``` +}> -- #### Step 3: Then, install the frontend canister with the command: +[Learn more about Azle](https://demergent-labs.github.io/azle/the_azle_book.html). -``` -dfx canister install kv_frontend -``` - -### Interacting with the canisters -- #### Step 1: To add a key-value pair via the frontend canister, run the following command: - -```bash -dfx canister call kv_frontend put '(1, 1337)' -``` - -:::note -The first call to put might be slow to respond because the data partition canisters have to be created first. -::: - -The output should resemble the following indicating that no value has previously been registered for this key: -```(null)``` - -- #### Step 2: Retrieve the value associated with a key using composite queries with the command: - -```bash -dfx canister call kv_frontend get '(1)' -``` - -The output should resemble the following: -```(opt (1337 : nat))``` - -This workflow displays the ability to fetch the value using composite queries with very low latency. - -## Comparing composite queries to calls from update functions -Let’s now compare the performance of composite query calls with those of an equivalent implementation that leverages calls from update functions. To do this, you will use the `get_update` method, which contains the exact same logic, but is implemented based on update calls. Run the following command in your terminal window: - -```bash -dfx canister call kv_frontend get_update '(1)' -``` + -The output will resemble the following: -```(opt (1_337 : nat))``` +}> -You can observe that with update calls you receive the very same result, but the call is at least one order of magnitude slower compared to composite query calls. +[Learn more about Kybra](https://demergent-labs.github.io/kybra/). -:::note -The examples repository also contains an equivalent [Motoko example](https://github.com/dfinity/examples/tree/master/motoko/composite_query). -::: + + ## Resources The following example canisters demonstrate how to use composite queries: - * [Azle example](https://github.com/demergent-labs/azle/tree/main/examples/composite_queries) * [Motoko example](https://github.com/dfinity/examples/tree/master/motoko/composite_query) * [Rust example](https://github.com/dfinity/examples/tree/master/rust/composite_query) diff --git a/docs/developer-docs/smart-contracts/advanced-features/handling-get-post-requests.mdx b/docs/developer-docs/smart-contracts/advanced-features/handling-get-post-requests.mdx index 8c49d27288..1d1f801617 100644 --- a/docs/developer-docs/smart-contracts/advanced-features/handling-get-post-requests.mdx +++ b/docs/developer-docs/smart-contracts/advanced-features/handling-get-post-requests.mdx @@ -8,7 +8,7 @@ import { AdornedTab } from "/src/components/Tabs/AdornedTab"; import { BetaChip } from "/src/components/Chip/BetaChip"; import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -# Handling GET/POST requests +# HTTPS gateways and incoming requests @@ -16,382 +16,138 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; Canisters running on ICP can use HTTP requests in two ways: incoming and outgoing. Incoming HTTP requests refer to HTTP requests that are sent to a canister and can be used to retrieve data from a canister or send new data to the canister. Outgoing HTTP requests refer to HTTP requests that the canister sends to other canisters or external services to retrieve data or send new data. -### Outgoing HTTP requests +## HTTP gateways -For outgoing HTTP requests, the [HTTPS outcalls](./https-outcalls/https-outcalls-how-to-use.mdx) feature should be used. +The ICP HTTP gateway protocol enables conventional HTTP clients to interact with the ICP network. HTTP gateways run adjacent to ICP and provide a connection used by software such as web browsers to send and receive standard HTTP requests and responses, including static assets, such as HTML, JavaScript, images, or videos. The HTTP gateway makes this workflow possible by translating standard HTTP requests into API canister calls that ICP canisters can understand and vice versa for HTTP responses. -- [How to use HTTPS outcalls](./https-outcalls/https-outcalls-how-to-use.mdx) +There are several HTTP gateway instances maintained for ICP, but it should be noted that an HTTP gateway is a centralized component that could become compromised, creating a threat for users receiving HTTP content. -- [HTTPS outcalls: `GET`](./https-outcalls/https-outcalls-get.mdx) +DFINITY exclusively controls the HTTP gateways that serve canisters on `ic0.app` and `icp0.io`. Developers can then also configure their own [custom domains](https://internetcomputer.org/docs/current/developer-docs/web-apps/custom-domains/using-custom-domains) to point at the DFINITY controlled HTTP gateways. -- [HTTPS outcalls: `POST`](./https-outcalls/https-outcalls-post.mdx) +Alternatively, developers can run their own HTTP gateways on their custom domains instead of using the DFINITY controlled gateways, but this is not well supported yet. -- [HTTPS outcalls: technology overview](/docs/current/references/https-outcalls-how-it-works) +For a more secure solution, it is possible to run your own HTTP gateway instance locally. -### Incoming HTTP requests +### Terminology -To handle incoming HTTP requests, canisters must define methods for `http_requests` and `http_requests_update` for `GET` and `POST` requests respectively. +- **Request**: A message sent to a server or endpoint. -All HTTP requests are handled by the ICP HTTP Gateway, therefore you cannot make direct `POST` calls to a canister's `http_request_update` method with HTTP clients such as curl. Instead, you can make a `POST` call to a canister's HTTP endpoint, then configure the canister's `http_request` method to [upgrade the call to `http_request_update` if necessary](/docs/current/references/http-gateway-protocol-spec#upgrade-to-update-calls). Below is an example `POST` call to a canister's endpoint: +- **Client**: A service that is able to send an HTTP formatted request and receive a response from a server. -``` -curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://.raw.ic0.app/ -``` +- **Gateway**: Enables the transfer of inbound and outbound messages. -## `GET` requests +## HTTP request lifecycle -HTTP `GET` requests are used to retrieve and return existing data from an endpoint. To handle a canister's incoming `GET` requests, the `http_request` method can be exposed. Users and other services can call this method using a `query` call. To return HTTP response data, the following examples display how to configure the `http_request` to return an HTTP `GET` request. +On ICP, an HTTP request goes through the following lifecycle: - - +1. An HTTP client makes an outbound request. -In Motoko, a `case` configuration can be used to return different `GET` responses based on the endpoint: - -```motoko no-repl -import FHM "mo:StableHashMap/FunctionalStableHashMap"; -import SHA256 "mo:motoko-sha/SHA256"; -import CertTree "mo:ic-certification/CertTree"; -import CanisterSigs "mo:ic-certification/CanisterSigs"; -import CertifiedData "mo:base/CertifiedData"; -import HTTP "./Http"; -import Iter "mo:base/Iter"; -import Blob "mo:base/Blob"; -import Option "mo:base/Option"; -import Time "mo:base/Time"; -import Text "mo:base/Text"; -import Debug "mo:base/Debug"; -import Prelude "mo:base/Prelude"; -import Principal "mo:base/Principal"; -import Buffer "mo:base/Buffer"; -import Nat8 "mo:base/Nat8"; -import CertifiedCache "lib"; -import Int "mo:base/Int"; - -actor Self { - type HttpRequest = HTTP.HttpRequest; - type HttpResponse = HTTP.HttpResponse; - - var two_days_in_nanos = 2 * 24 * 60 * 60 * 1000 * 1000 * 1000; - - stable var entries : [(Text, (Blob, Nat))] = []; - var cache = CertifiedCache.fromEntries( - entries, - Text.equal, - Text.hash, - Text.encodeUtf8, - func(b : Blob) : Blob { b }, - two_days_in_nanos + Int.abs(Time.now()), - ); - - public query func http_request(req : HttpRequest) : async HttpResponse { - switch (req.method, not Option.isNull(Array.find(req.headers, isGzip)), req.url) { - case ("GET", false, "/stream") {{ - status_code = 200; - headers = [ ("content-type", "text/plain") ]; - body = Text.encodeUtf8("Counter"); - streaming_strategy = ?#Callback({ - callback = http_streaming; - token = { - arbitrary_data = "start"; - } - }); - upgrade = ?false; - }}; - case ("GET", false, _) {{ - status_code = 200; - headers = [ ("content-type", "text/plain") ]; - body = Text.encodeUtf8("Counter is " # Nat.toText(counter) # "\n" # req.url # "\n"); - streaming_strategy = null; - upgrade = null; - }}; - case ("GET", true, _) {{ - status_code = 200; - headers = [ ("content-type", "text/plain"), ("content-encoding", "gzip") ]; - body = "\1f\8b\08\00\98\02\1b\62\00\03\2b\2c\4d\2d\aa\e4\02\00\d6\80\2b\05\06\00\00\00"; - streaming_strategy = null; - upgrade = null; - }}; - }; - } -} -``` +2. The HTTP gateway intercepts the request and resolves the canister ID of the request's destination. -Check out the [certified cache](https://github.com/krpeacock/certified-cache/blob/8657652c4062ef0e91ebe269843ccef1bb4796ae/src/demo.mo#L40) example project to see this code in use. +3. The request is encoded with Candid and sent in a query call to the destination canister's `http_request` method. - - +4. This request is sent to an ICP API boundary node, which will then forward the request to a replica node on the relevant subnet. -```rust -use http::{Request, Response, StatusCode}; +5. The canister receives and processes the request, then returns the response. -#[query] -fn respond_to(req: Request<()>) -> http::Result> { - if req.uri() != "/hello" { - return Response::builder() - .status(StatusCode::NOT_FOUND) - .body(()) - } +6. The HTTP gateway decodes the Candid encoding on the response. - let response_header = req.headers().contains_key("Hello!"); - let body = req.body(); +7. If the canister requests it, the gateway sends the request again via an update call to the canister's `http_request_update` method. The update call goes through consensus on the subnet. -} -``` +8. If the response size [exceeds the maximum response size](/docs/current/developer-docs/smart-contracts/maintain/resource-limits), the HTTP gateway fetches additional response body data through query calls. -Check out the [Rust documentation](https://docs.rs/ic-cdk/latest/ic_cdk/attr.query.html) for more info on query calls. +9. The HTTP gateway validates the response's certificate, if applicable. - -}> - -```typescript -import { - call, - candidEncode, - id, - IDL, - Principal, - query, - reply, - update -} from 'azle'; -import { - HttpRequestArgs, - HttpResponse, - HttpTransformArgs -} from 'azle/canisters/management'; - -export default class { - @update([], IDL.Text) - async xkcd(): Promise { - const httpResponse = await call('aaaaa-aa', 'http_request', { - paramIdlTypes: [HttpRequestArgs], - returnIdlType: HttpResponse, - args: [ - { - url: `https://xkcd.com/642/info.0.json`, - max_response_bytes: [2_000n], - method: { - get: null - }, - headers: [], - body: [], - transform: [ - { - function: [id(), 'xkcdTransform'], - context: Uint8Array.from([]) - } - ] - } - ], - payment: 50_000_000n - }); - - return Buffer.from(httpResponse.body).toString(); - } +10. The HTTP gateway returns the decoded response to the HTTP client. - @update([], HttpResponse, { manual: true }) - async xkcdRaw(): Promise { - const httpResponse = await call( - Principal.fromText('aaaaa-aa'), - 'http_request', - { - raw: candidEncode(` - ( - record { - url = "https://xkcd.com/642/info.0.json"; - max_response_bytes = 2_000 : nat64; - method = variant { get }; - headers = vec {}; - body = null; - transform = record { function = func "${id().toString()}".xkcdTransform; context = vec {} }; - } - ) - `), - payment: 50_000_000n - } - ); - - reply({ raw: httpResponse }); - } +Additional details can be found in the [HTTP gateway specification](https://internetcomputer.org/docs/current/references/http-gateway-protocol-spec). - @query([HttpTransformArgs], HttpResponse) - xkcdTransform(args: HttpTransformArgs): HttpResponse { - return { - ...args.response, - headers: [] - }; - } -} +## Running a local HTTP gateway -``` +To run your own local HTTP gateway, a proof-of-concept implementation can be used that enables a secure end-to-end connection with canisters deployed on ICP. -Check out the [Azle documentation](https://github.com/demergent-labs/azle/tree/main/tests/end_to_end/candid_rpc/class_syntax/outgoing_http_requests) for more info on HTTP requests. +This implementation features: - - +- Translation between HTTP asset requests and ICP API calls. -```python -from kybra import ( - Alias, - blob, - Func, - ic, - nat, - nat16, - Opt, - Query, - query, - Record, - StableBTreeMap, - Tuple, - Variant, - Vec, -) +- Detects ICP domains from principals and custom domain records. +- Terminates TLS connections locally. -class Token(Record): - arbitrary_data: str +- Bypasses remote gateway denylists. +- Resolves crypto domains. -class StreamingCallbackHttpResponse(Record): - body: blob - token: Opt[Token] +### Installation +You can download the pre-built installation package for your [operating system](https://github.com/dfinity/http-proxy?tab=readme-ov-file#installation). -Callback = Func(Query[[Token], StreamingCallbackHttpResponse]) +Once installed, you will have the option to start or stop the IC HTTP proxy service. Start the service to begin using it. Once the proxy is running, it will handle all traffic on your computer. Any traffic that's not meant for the ICP mainnet will pass through the gateway, and traffic that is meant for ICP will be intercepted by the proxy. The proxy will log traffic in the file `$HOME/Library/Preferences/dfinity/ichttpproxy/ic-http-proxy-proxy.log` on macOS machines, or in the `tmp` directory on Windows machines. +For example, make the following `curl` request to the NNS: -class CallbackStrategy(Record): - callback: Callback - token: Token +``` +curl -v https://nns.ic0.app +``` +This will result in a log entry of: + +``` +{"level":30, "time": "2024-06-17T13:46:40.947Z","pid":43956,"hostname":"JessieMgeonsMBP.attlocal.net","name":"IC HTTP Proxy Server","msg":"Proxying web3 request for nns.ic0.app:443"} +``` -class StreamingStrategy(Variant, total=False): - Callback: CallbackStrategy +## Outgoing HTTP requests +For outgoing HTTP requests, the [HTTPS outcalls](./https-outcalls/https-outcalls-how-to-use.mdx) feature should be used. -HeaderField = Alias[Tuple[str, str]] +- [How to use HTTPS outcalls](./https-outcalls/https-outcalls-how-to-use.mdx) +- [HTTPS outcalls: `GET`](./https-outcalls/https-outcalls-get.mdx) -class HttpResponse(Record): - status_code: nat16 - headers: Vec[HeaderField] - body: blob - streaming_strategy: Opt[StreamingStrategy] - upgrade: Opt[bool] +- [HTTPS outcalls: `POST`](./https-outcalls/https-outcalls-post.mdx) +- [HTTPS outcalls: technology overview](/docs/current/references/https-outcalls-how-it-works) -class HttpRequest(Record): - method: str - url: str - headers: Vec[HeaderField] - body: blob - certificate_version: Opt[nat16] +## Incoming HTTP requests +To handle incoming HTTP requests, canisters must define methods for `http_requests` and `http_requests_update` for `GET` and `POST` requests respectively. -stable_storage = StableBTreeMap[str, nat]( - memory_id=0, max_key_size=15, max_value_size=1_000 -) +All HTTP requests are handled by the ICP HTTP Gateway, therefore you cannot make direct `POST` calls to a canister's `http_request_update` method with HTTP clients such as curl. Instead, you can make a `POST` call to a canister's HTTP endpoint, then configure the canister's `http_request` method to [upgrade the call to `http_request_update` if necessary](/docs/current/references/http-gateway-protocol-spec#upgrade-to-update-calls). Below is an example `POST` call to a canister's endpoint: -stable_storage.insert("counter", 0) +``` +curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://.raw.ic0.app/ +``` +## `GET` requests -def isGzip(x: HeaderField) -> bool: - return x[0].lower() == "accept-encoding" and "gzip" in x[1].lower() +HTTP `GET` requests are used to retrieve and return existing data from an endpoint. To handle a canister's incoming `GET` requests, the `http_request` method can be exposed. Users and other services can call this method using a `query` call. To return HTTP response data, the following examples display how to configure the `http_request` to return an HTTP `GET` request. + + -@query -def http_request(req: HttpRequest) -> HttpResponse: - ic.print("Hello from http_request") +In Motoko, a `case` configuration can be used to return different `GET` responses based on the endpoint. - if req["method"] == "GET": - if next(filter(isGzip, req["headers"]), None) is None: - if req["url"] == "/stream": - return { - "status_code": 200, - "headers": [("content-type", "text/plain")], - "body": "Counter".encode("utf-8"), - "streaming_strategy": { - "Callback": { - "callback": (ic.id(), "http_streaming"), - "token": {"arbitrary_data": "start"}, - } - }, - "upgrade": False, - } - return { - "status_code": 200, - "headers": [("content-type", "text/plain")], - "body": f"Counter is {stable_storage.get('counter')}\n{req['url']}".encode( - "utf-8" - ), - "streaming_strategy": None, - "upgrade": None, - } - return { - "status_code": 200, - "headers": [("content-type", "text/plain"), ("content-encoding", "gzip")], - "body": bytes( - [ - 31, - 139, - 8, - 0, - 152, - 2, - 27, - 98, - 0, - 3, - 43, - 44, - 77, - 45, - 170, - 228, - 2, - 0, - 214, - 128, - 43, - 5, - 6, - 0, - 0, - 0, - ] - ), - "streaming_strategy": None, - "upgrade": None, - } +Check out the [certified cache](https://github.com/krpeacock/certified-cache/blob/8657652c4062ef0e91ebe269843ccef1bb4796ae/src/demo.mo#L40) example project to see an example of this implementation. - if req["method"] == "POST": - return { - "status_code": 204, - "headers": [], - "body": "".encode("utf-8"), - "streaming_strategy": None, - "upgrade": True, - } + + - return { - "status_code": 400, - "headers": [], - "body": "Invalid request".encode("utf-8"), - "streaming_strategy": None, - "upgrade": None, - } -``` +Rust canisters can use the `query` attribute. -:::caution +Check out the [Rust documentation](https://docs.rs/ic-cdk/latest/ic_cdk/attr.query.html) for more info on query calls. -Kybra canisters must be deployed from a Python virtual environment. [Learn more in the Kybra docs](/docs/current/developer-docs/backend/python/). + +}> -::: +[Learn more about Azle](https://demergent-labs.github.io/azle/the_azle_book.html). -Check out the [Kybra documentation](https://demergent-labs.github.io/kybra/http.html) for more info on HTTP requests. + - +}> + +[Learn more about Kybra](https://demergent-labs.github.io/kybra/). + + ## `POST` requests @@ -401,352 +157,160 @@ HTTP `POST` requests are used to send data to an endpoint with the intention of -In Motoko, a `case` configuration can be used to return different `POST` responses based on the endpoint: - -```motoko no-repl -import FHM "mo:StableHashMap/FunctionalStableHashMap"; -import SHA256 "mo:motoko-sha/SHA256"; -import CertTree "mo:ic-certification/CertTree"; -import CanisterSigs "mo:ic-certification/CanisterSigs"; -import CertifiedData "mo:base/CertifiedData"; -import HTTP "./Http"; -import Iter "mo:base/Iter"; -import Blob "mo:base/Blob"; -import Option "mo:base/Option"; -import Time "mo:base/Time"; -import Text "mo:base/Text"; -import Debug "mo:base/Debug"; -import Prelude "mo:base/Prelude"; -import Principal "mo:base/Principal"; -import Buffer "mo:base/Buffer"; -import Nat8 "mo:base/Nat8"; -import CertifiedCache "lib"; -import Int "mo:base/Int"; - -actor Self { - type HttpRequest = HTTP.HttpRequest; - type HttpResponse = HTTP.HttpResponse; - - var two_days_in_nanos = 2 * 24 * 60 * 60 * 1000 * 1000 * 1000; - - stable var entries : [(Text, (Blob, Nat))] = []; - var cache = CertifiedCache.fromEntries( - entries, - Text.equal, - Text.hash, - Text.encodeUtf8, - func(b : Blob) : Blob { b }, - two_days_in_nanos + Int.abs(Time.now()), - ); - - public func http_request_update(req : HttpRequest) : async HttpResponse { - switch (req.method, not Option.isNull(Array.find(req.headers, isGzip))) { - case ("POST", false) { - counter += 1; - { - status_code = 201; - headers = [ ("content-type", "text/plain") ]; - body = Text.encodeUtf8("Counter updated to " # Nat.toText(counter) # "\n"); - streaming_strategy = null; - upgrade = null; - } - }; - case ("POST", true) { - counter += 1; - { - status_code = 201; - headers = [ ("content-type", "text/plain"), ("content-encoding", "gzip") ]; - body = "\1f\8b\08\00\37\02\1b\62\00\03\2b\2d\48\49\2c\49\e5\02\00\a8\da\91\6c\07\00\00\00"; - - streaming_strategy = null; - upgrade = null; - } - }; - }; - }; -} - -``` +In Motoko, a `case` configuration can be used to return different `POST` responses based on the endpoint. -Check out the [certified cache](https://github.com/krpeacock/certified-cache/blob/8657652c4062ef0e91ebe269843ccef1bb4796ae/src/demo.mo#L76) example project to see this code in use. +Check out the [certified cache](https://github.com/krpeacock/certified-cache/blob/8657652c4062ef0e91ebe269843ccef1bb4796ae/src/demo.mo#L40) example project to see an example of this implementation. -```rust -use http::{Request, Response, StatusCode}; - -#[update] -fn respond_to(req: Request<()>) -> http::Result> { - let mut builder = Response::builder() - .header("Hello!") - .status(StatusCode::OK); - - if req.headers().contains_key("Another-Header") { - builder = builder.header("Another-Header", "Ack"); - } - - builder.body(()) -} -``` +Rust canisters can use the `update` attribute. Check out the [Rust documentation](https://docs.rs/ic-cdk/latest/ic_cdk/attr.update.html) for more info on update calls. -}> - -```typescript -import { - call, - candidEncode, - id, - IDL, - Principal, - query, - reply, - update -} from 'azle'; -import { - HttpRequestArgs, - HttpResponse, - HttpTransformArgs -} from 'azle/canisters/management'; - -export default class { - @update([], IDL.Text) - async xkcd(): Promise { - const httpResponse = await call('aaaaa-aa', 'http_request', { - paramIdlTypes: [HttpRequestArgs], - returnIdlType: HttpResponse, - args: [ - { - url: `https://xkcd.com/642/info.0.json`, - max_response_bytes: [2_000n], - method: { - get: null - }, - headers: [], - body: [], - transform: [ - { - function: [id(), 'xkcdTransform'] as [ - Principal, - string - ], - context: Uint8Array.from([]) - } - ] - } - ], - payment: 50_000_000n - }); - - return Buffer.from(httpResponse.body).toString(); - } - - @update([], HttpResponse, { manual: true }) - async xkcdRaw(): Promise { - const httpResponse = await call( - Principal.fromText('aaaaa-aa'), - 'http_request', - { - raw: candidEncode(` - ( - record { - url = "https://xkcd.com/642/info.0.json"; - max_response_bytes = 2_000 : nat64; - method = variant { post }; - headers = vec {}; - body = null; - transform = record { function = func "${id().toString()}".xkcdTransform; context = vec {} }; - } - ) - `), - payment: 50_000_000n - } - ); - - reply({ raw: httpResponse }); - } - - @query([HttpTransformArgs], HttpResponse) - xkcdTransform(args: HttpTransformArgs): HttpResponse { - return { - ...args.response, - headers: [] - }; - } -} -``` - -Check out the [Azle documentation](https://github.com/demergent-labs/azle/tree/main/tests/end_to_end/candid_rpc/class_syntax/outgoing_http_requests) for more info on HTTP requests. - - - - -```python -from kybra import ( - Alias, - blob, - Func, - ic, - nat, - nat16, - Opt, - Query, - query, - Record, - StableBTreeMap, - Tuple, - update, - Variant, - Vec, -) +}> +[Learn more about Azle](https://demergent-labs.github.io/azle/the_azle_book.html). -class Token(Record): - arbitrary_data: str + +}> -class StreamingCallbackHttpResponse(Record): - body: blob - token: Opt[Token] +[Learn more about Kybra](https://demergent-labs.github.io/kybra/). + + -Callback = Func(Query[[Token], StreamingCallbackHttpResponse]) +## Serving HTTP requests +Canisters can serve or handle an incoming HTTP request using the +[HTTP Gateway Protocol](/docs/current/references/http-gateway-protocol-spec). +This allows developers to host web applications and APIs from a canister. -class CallbackStrategy(Record): - callback: Callback - token: Token +The following example returns 'Hello, World!' in the body at the `/hello` +endpoint. + + -class StreamingStrategy(Variant, total=False): - Callback: CallbackStrategy +```motoko +import HashMap = "mo:base/HashMap"; +import Blob = "mo:base/Blob"; +import Text "mo:base/Text"; +import Option "mo:base/Option"; +actor { -HeaderField = Alias[Tuple[str, str]] + public type HttpRequest = { + body: Blob; + headers: [HeaderField]; + method: Text; + url: Text; + }; + public type ChunkId = Nat; + public type SetAssetContentArguments = { + chunk_ids: [ChunkId]; + content_encoding: Text; + key: Key; + sha256: ?Blob; + }; + public type Path = Text; + public type Key = Text; -class HttpResponse(Record): - status_code: nat16 - headers: Vec[HeaderField] - body: blob - streaming_strategy: Opt[StreamingStrategy] - upgrade: Opt[bool] + public type HttpResponse = { + body: Blob; + headers: [HeaderField]; + status_code: Nat16; + }; + public type HeaderField = (Text, Text); -class HttpRequest(Record): - method: str - url: str - headers: Vec[HeaderField] - body: blob + private func removeQuery(str: Text): Text { + return Option.unwrap(Text.split(str, #char '?').next()); + }; + public query func http_request(req: HttpRequest): async (HttpResponse) { + let path = removeQuery(req.url); + if(path == "/hello") { + return { + body = Text.encodeUtf8("root page :" # path); + headers = []; + status_code = 200; + }; + }; -stable_storage = StableBTreeMap[str, nat]( - memory_id=0, max_key_size=15, max_value_size=1_000 -) + return { + body = Text.encodeUtf8("404 Not found :" # path); + headers = []; + status_code = 404; + }; + }; +} +``` -stable_storage.insert("counter", 0) + + -def isGzip(x: HeaderField) -> bool: - return x[0].lower() == "accept-encoding" and "gzip" in x[1].lower() +```rust +type HeaderField = (String, String); -@update -def http_request_update(req: HttpRequest) -> HttpResponse: - ic.print("Hello from update") - global stable_storage +struct HttpResponse { + status_code: u16, + headers: Vec, + body: Cow<'static, Bytes>, +} - if req["method"] == "POST": - counter = stable_storage.get("counter") or 0 - stable_storage.insert("counter", counter + 1) +struct HttpRequest { + method: String, + url: String, + headers: Vec<(String, String)>, + body: ByteBuf, +} - if next(filter(isGzip, req["headers"]), None) is None: - return { - "status_code": 201, - "headers": [("content-type", "text/plain")], - "body": f"Counter updated to {stable_storage.get('counter')}".encode( - "utf-8" - ), - "streaming_strategy": None, - "upgrade": None, - } - return { - "status_code": 201, - "headers": [("content-type", "text/plain"), ("content-encoding", "gzip")], - "body": bytes( - [ - 31, - 139, - 8, - 0, - 55, - 2, - 27, - 98, - 0, - 3, - 43, - 45, - 72, - 73, - 44, - 73, - 229, - 2, - 0, - 168, - 218, - 145, - 108, - 7, - 0, - 0, - 0, - ] - ), - "streaming_strategy": None, - "upgrade": None, +#[query] +fn http_request(req: HttpRequest) -> HttpResponse { + let path = req.url.path(); + if path == "/hello" { + HttpResponse { + status_code: 200, + headers: Vec::new(), + body: b"hello, world!".to_vec(), + streaming_strategy: None, + upgrade: None, } - - return { - "status_code": 400, - "headers": [], - "body": "Invalid request".encode("utf-8"), - "streaming_strategy": None, - "upgrade": None, - } - - -@query -def http_streaming(token: Token) -> StreamingCallbackHttpResponse: - ic.print("Hello from http_streaming") - if token["arbitrary_data"] == "start": - return {"body": " is ".encode("utf-8"), "token": {"arbitrary_data": "next"}} - if token["arbitrary_data"] == "next": - return { - "body": f"{stable_storage.get('counter')}".encode("utf-8"), - "token": {"arbitrary_data": "last"}, + } else { + HttpResponse { + status_code: 404, + headers: Vec::new(), + body: b"404 Not found :".to_vec(), + streaming_strategy: None, + upgrade: None, } - if token["arbitrary_data"] == "last": - return {"body": " streaming\n".encode("utf-8"), "token": None} - ic.trap("unreachable") + } +} ``` -:::caution + +}> -Kybra canisters must be deployed from a Python virtual environment. [Learn more in the Kybra docs](/docs/current/developer-docs/backend/python/). +[Learn more about Azle](https://demergent-labs.github.io/azle/the_azle_book.html). -::: + -Check out the [Kybra documentation](https://demergent-labs.github.io/kybra/http.html) for more info on HTTP requests. +}> - +[Learn more about Kybra](https://demergent-labs.github.io/kybra/). + + ## Resources +- [HTTP gateway specification](https://internetcomputer.org/docs/current/references/http-gateway-protocol-spec) + - [How to use HTTPS outcalls](./https-outcalls/https-outcalls-how-to-use.mdx) - [HTTPS outcalls: `GET`](./https-outcalls/https-outcalls-get.mdx) diff --git a/docs/developer-docs/smart-contracts/advanced-features/http-gateways.mdx b/docs/developer-docs/smart-contracts/advanced-features/http-gateways.mdx deleted file mode 100644 index d8ad3d5506..0000000000 --- a/docs/developer-docs/smart-contracts/advanced-features/http-gateways.mdx +++ /dev/null @@ -1,93 +0,0 @@ ---- -keywords: [advanced, tutorial, http, http gateways, gateways] ---- - -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; - -# HTTP gateways - - - -## Overview - -The ICP HTTP gateway protocol enables conventional HTTP clients to interact with the ICP network. HTTP gateways run adjacent to ICP and provide a connection used by software such as web browsers to send and receive standard HTTP requests and responses, including static assets, such as HTML, JavaScript, images, or videos. The HTTP gateway makes this workflow possible by translating standard HTTP requests into API canister calls that ICP canisters can understand and vice versa for HTTP responses. - -There are several HTTP gateway instances maintained for ICP, but it should be noted that an HTTP gateway is a centralized component that could become compromised, creating a threat for users receiving HTTP content. - -DFINITY exclusively controls the HTTP gateways that serve canisters on `ic0.app` and `icp0.io`. Developers can then also configure their own [custom domains](https://internetcomputer.org/docs/current/developer-docs/web-apps/custom-domains/using-custom-domains) to point at the DFINITY controlled HTTP gateways. - -Alternatively, developers can run their own HTTP gateways on their custom domains instead of using the DFINITY controlled gateways, but this is not well supported yet. - -For a more secure solution, it is possible to run your own HTTP gateway instance locally. - -### Terminology - -- **Request**: A message sent to a server or endpoint. - -- **Client**: A service that is able to send an HTTP formatted request and receive a response from a server. - -- **Gateway**: Enables the transfer of inbound and outbound messages. - -## HTTP request lifecycle - -On ICP, an HTTP request goes through the following lifecycle: - -1. An HTTP client makes an outbound request. - -2. The HTTP gateway intercepts the request and resolves the canister ID of the request's destination. - -3. The request is encoded with Candid and sent in a query call to the destination canister's `http_request` method. - -4. This request is sent to an ICP API boundary node, which will then forward the request to a replica node on the relevant subnet. - -5. The canister receives and processes the request, then returns the response. - -6. The HTTP gateway decodes the Candid encoding on the response. - -7. If the canister requests it, the gateway sends the request again via an update call to the canister's `http_request_update` method. The update call goes through consensus on the subnet. - -8. If the response size [exceeds the maximum response size](/docs/current/developer-docs/smart-contracts/maintain/resource-limits), the HTTP gateway fetches additional response body data through query calls. - -9. The HTTP gateway validates the response's certificate, if applicable. - -10. The HTTP gateway returns the decoded response to the HTTP client. - -Additional details can be found in the [HTTP gateway specification](https://internetcomputer.org/docs/current/references/http-gateway-protocol-spec). - -## Running a local HTTP gateway - -To run your own local HTTP gateway, a proof-of-concept implementation can be used that enables a secure end-to-end connection with canisters deployed on ICP. - -This implementation features: - -- Translation between HTTP asset requests and ICP API calls. - -- Detects ICP domains from principals and custom domain records. - -- Terminates TLS connections locally. - -- Bypasses remote gateway denylists. - -- Resolves crypto domains. - -### Installation - -You can download the pre-built installation package for your [operating system](https://github.com/dfinity/http-proxy?tab=readme-ov-file#installation). - -Once installed, you will have the option to start or stop the IC HTTP proxy service. Start the service to begin using it. Once the proxy is running, it will handle all traffic on your computer. Any traffic that's not meant for the ICP mainnet will pass through the gateway, and traffic that is meant for ICP will be intercepted by the proxy. The proxy will log traffic in the file `$HOME/Library/Preferences/dfinity/ichttpproxy/ic-http-proxy-proxy.log` on macOS machines, or in the `tmp` directory on Windows machines. - -For example, make the following `curl` request to the NNS: - -``` -curl -v https://nns.ic0.app -``` - -This will result in a log entry of: - -``` -{"level":30, "time": "2024-06-17T13:46:40.947Z","pid":43956,"hostname":"JessieMgeonsMBP.attlocal.net","name":"IC HTTP Proxy Server","msg":"Proxying web3 request for nns.ic0.app:443"} -``` - -## Resources - -- [HTTP gateway specification](https://internetcomputer.org/docs/current/references/http-gateway-protocol-spec). diff --git a/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-get.mdx b/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-get.mdx index 35877a0f31..2dfc04365c 100644 --- a/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-get.mdx +++ b/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-get.mdx @@ -15,319 +15,14 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; ## Overview -A minimal example to make a `GET` HTTP request. The purpose of this dapp is only to show how to make HTTP requests from a canister. +A minimal example to make a `GET` HTTP request. The purpose of this dapp is only to show how to make HTTP requests from a canister. It sends a `GET` request to the Coinbase API and retrieves some historical data about the ICP token. -The sample code is in both Motoko and Rust. This sample canister sends a `GET` request to the Coinbase API and retrieves some historical data about the ICP token. - - -**The main intent of this canister is to show developers how to make idempotent `GET` requests.** - -This example takes less than 5 minutes to complete. - -### Sample dapp - -The canister in this tutorial will have only **one public method** named `get_icp_usd_exchange()` which, when called, will trigger an HTTP `GET` request to an external service. The canister will not have a frontend (only a backend), but like all canisters, you can interact with its public methods via the Candid web UI, which will look like this: - - -![Candid web UI](../_attachments/https-get.webp) - -The `get_icp_usd_exchange()` method returns Coinbase data on the exchange rate between USD and ICP for a certain day. The data will look like this: - -The API response looks like this: -``` - [ - [ - 1682978460, <-- start timestamp - 5.714, <-- lowest price during time range - 5.718, <-- highest price during range - 5.714, <-- price at open - 5.714, <-- price at close - 243.5678 <-- volume of traded - ], -] -``` - -## Code structure - -Before you dive in, here is the structure of the code you will touch: - - - - -```motoko no-repl - -//Import some custom types from `src/backend_canister/Types.mo` file -import Types "Types"; - -actor { - - //0. method that uses the HTTPS outcalls feature and returns a string - public func foo() : async Text { - - //1. DECLARE MANAGEMENT CANISTER - let ic : Types.IC = actor ("aaaaa-aa"); - - //2. SETUP ARGUMENTS FOR HTTP GET request - let request : Types.HttpRequestArgs = { - //construct the request - }; - - //3. ADD CYCLES TO PAY FOR HTTP REQUEST - //code to add cycles - - //4. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - let response : Types.HttpResponsePayload = await ic.http_request(request); - - //5. DECODE THE RESPONSE - //code to decode response - - //6. RETURN RESPONSE OF THE BODY - response - }; - - //7. CREATE TRANSFORM FUNCTION - public query func transform(raw : Types.TransformArgs) : async Types.CanisterHttpResponsePayload { - ////code for the transform function - } - -}; -``` - -You will also create some custom types in `Types.mo`. It will look like this: - -```motoko no-repl -module Types { - - //type declarations for s, HTTP responses, management canister, etc... - -} -``` - - - - -Here is how the management canister is declared in a Rust canister (e.g. `lib.rs`): - -```rust -//1. DECLARE MANAGEMENT CANISTER -use ic_cdk::api::management_canister::http_request::{ - http_request, CanisterHttpRequestArgument, HttpHeader, HttpMethod, HttpResponse, TransformArgs, - TransformContext, -}; - -//Update method using the HTTPS outcalls feature -#[ic_cdk::update] -async fn foo() { - //2. SETUP ARGUMENTS FOR HTTP GET request - let request = CanisterHttpRequestArgument { - //instantiate the request - }; - - //3. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - //Note: in Rust, `http_request()` already sends the cycles needed - //so no need for explicit Cycles.add() as in Motoko - match http_request(request).await { - - //4. DECODE AND RETURN THE RESPONSE - Ok((response,)) => { - //Ok case - } - Err((r, m)) => { - //error case - } - } -} - -// 4. CREATE TRANSFORM FUNCTION -#[ic_cdk::query] -fn transform(raw: TransformArgs) -> HttpResponse { } -``` - - - - - -- #### Step 1: Create a new project by running the following command: +## `GET` example -```bash -dfx new send_http_get_motoko -cd send_http_get_motoko -npm install -``` - - - - -```bash -dfx new send_http_get_rust --type=rust -cd send_http_get_rust -npm install -rustup target add wasm32-unknown-unknown -``` - - - - - - -- #### Step 2: Edit the backend canister's code. - - - - -Open the `src/send_http_get_motoko_backend/main.mo` file in a text editor and replace content with: - -```motoko no-repl title="src/send_http_get_motoko_backend/main.mo" -import Debug "mo:base/Debug"; -import Blob "mo:base/Blob"; -import Cycles "mo:base/ExperimentalCycles"; -import Error "mo:base/Error"; -import Array "mo:base/Array"; -import Nat8 "mo:base/Nat8"; -import Nat64 "mo:base/Nat64"; -import Text "mo:base/Text"; - -//import the custom types you have in Types.mo -import Types "Types"; - - -//Actor -actor { - -//This method sends a GET request to a URL with a free API you can test. -//This method returns Coinbase data on the exchange rate between USD and ICP -//for a certain day. -//The API response looks like this: -// [ -// [ -// 1682978460, <-- start timestamp -// 5.714, <-- lowest price during time range -// 5.718, <-- highest price during range -// 5.714, <-- price at open -// 5.714, <-- price at close -// 243.5678 <-- volume of ICP traded -// ], -// ] - - public func get_icp_usd_exchange() : async Text { - - //1. DECLARE MANAGEMENT CANISTER - //You need this so you can use it to make the HTTP request - let ic : Types.IC = actor ("aaaaa-aa"); - - //2. SETUP ARGUMENTS FOR HTTP GET request - - // 2.1 Setup the URL and its query parameters - let ONE_MINUTE : Nat64 = 60; - let start_timestamp : Types.Timestamp = 1682978460; //May 1, 2023 22:01:00 GMT - let end_timestamp : Types.Timestamp = 1682978520;//May 1, 2023 22:02:00 GMT - let host : Text = "api.pro.coinbase.com"; - let url = "https://" # host # "/products/ICP-USD/candles?start=" # Nat64.toText(start_timestamp) # "&end=" # Nat64.toText(start_timestamp) # "&granularity=" # Nat64.toText(ONE_MINUTE); - - // 2.2 prepare headers for the system http_request call - let request_headers = [ - { name = "Host"; value = host # ":443" }, - { name = "User-Agent"; value = "exchange_rate_canister" }, - ]; - - // 2.2.1 Transform context - let transform_context : Types.TransformContext = { - function = transform; - context = Blob.fromArray([]); - }; - - // 2.3 The HTTP request - let http_request : Types.HttpRequestArgs = { - url = url; - max_response_bytes = null; //optional for request - headers = request_headers; - body = null; //optional for request - method = #get; - transform = ?transform_context; - }; - - //3. ADD CYCLES TO PAY FOR HTTP REQUEST - - //The IC specification spec says, "Cycles to pay for the call must be explicitly transferred with the call" - //The management canister will make the HTTP request so it needs cycles - //See: /docs/current/motoko/main/canister-maintenance/cycles - - //The way Cycles.add() works is that it adds those cycles to the next asynchronous call - //"Function add(amount) indicates the additional amount of cycles to be transferred in the next remote call" - //See: /docs/current/references/ic-interface-spec#ic-http_request - Cycles.add(20_949_972_000); - - //4. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - //Since the cycles were added above, you can just call the management canister with HTTPS outcalls below - let http_response : Types.HttpResponsePayload = await ic.http_request(http_request); - - //5. DECODE THE RESPONSE - - //As per the type declarations in `src/Types.mo`, the BODY in the HTTP response - //comes back as [Nat8s] (e.g. [2, 5, 12, 11, 23]). Type signature: - - //public type HttpResponsePayload = { - // status : Nat; - // headers : [HttpHeader]; - // body : [Nat8]; - // }; - - //You need to decode that [Nat8] array that is the body into readable text. - //To do this, you: - // 1. Convert the [Nat8] into a Blob - // 2. Use Blob.decodeUtf8() method to convert the Blob to a ?Text optional - // 3. You use a switch to explicitly call out both cases of decoding the Blob into ?Text - let response_body: Blob = Blob.fromArray(http_response.body); - let decoded_text: Text = switch (Text.decodeUtf8(response_body)) { - case (null) { "No value returned" }; - case (?y) { y }; - }; - - //6. RETURN RESPONSE OF THE BODY - //The API response will looks like this: - - // ("[[1682978460,5.714,5.718,5.714,5.714,243.5678]]") - - //Which can be formatted as this - // [ - // [ - // 1682978460, <-- start/timestamp - // 5.714, <-- low - // 5.718, <-- high - // 5.714, <-- open - // 5.714, <-- close - // 243.5678 <-- volume - // ], - // ] - decoded_text - }; - - //7. CREATE TRANSFORM FUNCTION - public query func transform(raw : Types.TransformArgs) : async Types.CanisterHttpResponsePayload { - let transformed : Types.CanisterHttpResponsePayload = { - status = raw.response.status; - body = raw.response.body; - headers = [ - { - name = "Content-Security-Policy"; - value = "default-src 'self'"; - }, - { name = "Referrer-Policy"; value = "strict-origin" }, - { name = "Permissions-Policy"; value = "geolocation=(self)" }, - { - name = "Strict-Transport-Security"; - value = "max-age=63072000"; - }, - { name = "X-Frame-Options"; value = "DENY" }, - { name = "X-Content-Type-Options"; value = "nosniff" }, - ]; - }; - transformed; - }; -}; +```motoko no-repl file=../../../../references/samples/motoko/send_http_get/src/send_http_get_backend/main.mo ``` - `get_icp_usd_exchange()` is an update call. All methods that make HTTPS outcalls must be update calls because they go through consensus, even if the HTTPS outcall is a `GET`. @@ -337,418 +32,35 @@ actor { -Open the `src/send_http_get_rust_backend/src/lib.rs` file in a text editor and replace the content with: - -```rust title="src/send_http_get_rust_backend/src/lib.rs" -//1. IMPORT MANAGEMENT CANISTER -//This includes all methods and types needed -use ic_cdk::api::management_canister::http_request::{ - http_request, CanisterHttpRequestArgument, HttpHeader, HttpMethod, HttpResponse, TransformArgs, - TransformContext, TransformFunc, -}; - - -//Update method using the HTTPS outcalls feature -#[ic_cdk::update] -async fn get_icp_usd_exchange() -> String { - //2. SETUP ARGUMENTS FOR HTTP GET request - - // 2.1 Setup the URL and its query parameters - type Timestamp = u64; - let start_timestamp: Timestamp = 1682978460; //May 1, 2023 22:01:00 GMT - let seconds_of_time: u64 = 60; //start with 60 seconds - let host = "api.pro.coinbase.com"; - let url = format!( - "https://{}/products/ICP-USD/candles?start={}&end={}&granularity={}", - host, start_timestamp, start_timestamp, seconds_of_time - ); - - // 2.2 Prepare headers for the system http_request call - //Note that `HttpHeader` is declared in line 4 - let request_headers = vec![ - HttpHeader { - name: "Host".to_string(), - value: format!("{host}:443"), - }, - HttpHeader { - name: "User-Agent".to_string(), - value: "exchange_rate_canister".to_string(), - }, - ]; - - //note "CanisterHttpRequestArgument" and "HttpMethod" are declared in line 4 - let request = CanisterHttpRequestArgument { - url: url.to_string(), - method: HttpMethod::GET, - body: None, //optional for request - max_response_bytes: None, //optional for request - transform: Some(TransformContext { - // The "method" parameter needs to have the same name as the function name of your transform function - function: TransformFunc(candid::Func { - principal: ic_cdk::api::id(), - method: "transform".to_string(), - }), - // The "TransformContext" function does need a context parameter, it can be empty - context: vec![], - }), - headers: request_headers, - }; - - //3. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - - //Note: in Rust, `http_request()` needs to pass cycles if you are using ic_cdk: ^0.9.0 - let cycles = 230_949_972_000; - - match http_request(request, cycles).await { - //4. DECODE AND RETURN THE RESPONSE - - //See:https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/struct.HttpResponse.html - Ok((response,)) => { - //if successful, `HttpResponse` has this structure: - // pub struct HttpResponse { - // pub status: Nat, - // pub headers: Vec, - // pub body: Vec, - // } - - //You need to decode that Vec that is the body into readable text. - //To do this: - // 1. Call `String::from_utf8()` on response.body - // 3. You use a switch to explicitly call out both cases of decoding the Blob into ?Text - - //The API response will look like this: - - // ("[[1682978460,5.714,5.718,5.714,5.714,243.5678]]") - - //Which can be formatted as this - // [ - // [ - // 1682978460, <-- start/timestamp - // 5.714, <-- low - // 5.718, <-- high - // 5.714, <-- open - // 5.714, <-- close - // 243.5678 <-- volume - // ], - // ] - - //Return the body as a string and end the method - String::from_utf8(response.body).expect("Transformed response is not UTF-8 encoded.") - } - Err((r, m)) => { - let message = - format!("The http_request resulted into error. RejectionCode: {r:?}, Error: {m}"); - - //Return the error as a string and end the method - message - } - } -} - -// Strips all data that is not needed from the original response. -#[ic_cdk::query] -fn transform(raw: TransformArgs) -> HttpResponse { - let headers = vec![ - HttpHeader { - name: "Content-Security-Policy".to_string(), - value: "default-src 'self'".to_string(), - }, - HttpHeader { - name: "Referrer-Policy".to_string(), - value: "strict-origin".to_string(), - }, - HttpHeader { - name: "Permissions-Policy".to_string(), - value: "geolocation=(self)".to_string(), - }, - HttpHeader { - name: "Strict-Transport-Security".to_string(), - value: "max-age=63072000".to_string(), - }, - HttpHeader { - name: "X-Frame-Options".to_string(), - value: "DENY".to_string(), - }, - HttpHeader { - name: "X-Content-Type-Options".to_string(), - value: "nosniff".to_string(), - }, - ]; - - let mut res = HttpResponse { - status: raw.response.status.clone(), - body: raw.response.body.clone(), - headers, - }; - - if res.status == 200u64 { - res.body = raw.response.body; - } else { - ic_cdk::api::print(format!("Received an error from coinbase: err = {:?}", raw)); - } - res -} +```rust file=../../../../references/samples/rust/send_http_get/src/send_http_get_backend/src/lib.rs ``` - `get_icp_usd_exchange() -> String` returns a `String`, but this is not necessary. In this tutorial, this is done for easier testing. - The `lib.rs` file uses [http_request](https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/fn.http_request.html) which is a convenient Rust CDK method that already sends cycles to the management canister under the hood. It knows how many cycles to send for a 13-node subnet in most cases. If your HTTPS outcall needs more cycles, you should use the [http_request_with_cycles()](https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/fn.http_request_with_cycles.html) method and explicitly call the cycles needed. - The Rust CDK method `http_request` used above wraps the management canister method [`http_request`](/docs/current/references/ic-interface-spec#ic-http_request), but it is not strictly the same. - - - -:::caution - -Headers in the response may not always be identical across all nodes that process the request for consensus, causing the result of the call to be "No consensus could be reached." This particular error message can be hard to debug, but one method to resolve this error is to edit the response using the transform function. The transform function is run before consensus, and can be used to remove some headers from the response. For example, the following Rust variation removes all headers aside from the body and status code of the call: - -```rust title="src/send_http_get_rust_backend/src/lib.rs" -use ic_cdk::{ - api::management_canister::http_request::{HttpResponse, TransformArgs}, - query, -}; - -#[query] -fn transform(raw: TransformArgs) -> HttpResponse { - let mut res = HttpResponse { - status: raw.response.status.clone(), - body: raw.response.body.clone(), - ..Default::default() - }; - - if i32::try_from(res.status.clone().0).unwrap() == 200 { - res.body = raw.response.body; - } else { - ic_cdk::api::print(format!("Received an error from proxy: err = {:?}", raw)); - } - - res -} -``` -::: - - -- #### Step 3: Edit the Type or Candid files. - - - - -Open the `src/send_http_get_motoko_backend/Types.mo` file in a text editor and replace content with: - -```motoko no-repl title="src/send_http_get_motoko_backend/Types.mo" -module Types { - - public type Timestamp = Nat64; - - //1. Type that describes the Request arguments for an HTTPS outcall - //See: /docs/current/references/ic-interface-spec#ic-http_request - public type HttpRequestArgs = { - url : Text; - max_response_bytes : ?Nat64; - headers : [HttpHeader]; - body : ?[Nat8]; - method : HttpMethod; - transform : ?TransformRawResponseFunction; - }; - - public type HttpHeader = { - name : Text; - value : Text; - }; - - public type HttpMethod = { - #get; - #post; - #head; - }; - - public type HttpResponsePayload = { - status : Nat; - headers : [HttpHeader]; - body : [Nat8]; - }; - - //2. HTTPS outcalls have an optional "transform" key. These two types help describe it. - //"The transform function may, for example, transform the body in any way, add or remove headers, - //modify headers, etc. " - //See: /docs/current/references/ic-interface-spec#ic-http_request - - - //2.1 This type describes a function called "TransformRawResponse" used in line 14 above - //"If provided, the calling canister itself must export this function." - //In this minimal example for a `GET` request, you declare the type for completeness, but - //you do not use this function. You will pass "null" to the HTTP request. - public type TransformRawResponseFunction = { - function : shared query TransformArgs -> async HttpResponsePayload; - context : Blob; - }; - - //2.2 These types describes the arguments the transform function needs - public type TransformArgs = { - response : HttpResponsePayload; - context : Blob; - }; - - public type CanisterHttpResponsePayload = { - status : Nat; - headers : [HttpHeader]; - body : [Nat8]; - }; - - public type TransformContext = { - function : shared query TransformArgs -> async HttpResponsePayload; - context : Blob; - }; - - - //3. Declaring the management canister which you use to make the HTTPS outcall - public type IC = actor { - http_request : HttpRequestArgs -> async HttpResponsePayload; - }; - -} -``` - - - - -Open the `src/send_http_get_rust_backend/send_http_get_rust_backend.did` file in a text editor and replace the content with: - -You update the Candid interface file so it matches the method `get_icp_usd_exchange()` in `lib.rs`. - -```candid title="src/send_http_get_rust_backend/send_http_get_rust_backend.did" -service : { - "get_icp_usd_exchange": () -> (text); -} -``` - -Open the `src/send_http_get_rust_backend/Cargo.toml` file in a text editor and replace the content with: - -```bash title="src/send_http_get_rust_backend/Cargo.toml" -[package] -name = "send_http_get_rust_backend" -version = "0.1.0" -edition = "2021" - -# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html - -[lib] -crate-type = ["cdylib"] +To use HTTPS outcalls you must update the canister's Candid file: -[dependencies] -candid = "0.10" -ic-cdk = "0.13" +```candid file=../../../../references/samples/rust/send_http_get/src/send_http_get_backend/send_http_get_backend.did ``` - - - +Update the `Cargo.toml` file to use the correct dependencies: -- #### Step 4: Test the dapp locally. - - - - -Deploy the dapp locally: - -```bash -dfx start --clean --background -dfx deploy +```toml file=../../../../references/samples/rust/send_http_get/src/send_http_get_backend/Cargo.toml ``` -If successful, the terminal should return canister URLs you can open: - -```bash -Deployed canisters. -URLs: - Frontend canister via browser - send_http_get_motoko_frontend: http://127.0.0.1:4943/?canisterId=asrmz-lmaaa-aaaaa-qaaeq-cai - Backend canister via Candid interface: - send_http_get_motoko_backend: http://127.0.0.1:4943/?canisterId=a3shf-5eaaa-aaaaa-qaafa-cai&id=avqkn-guaaa-aaaaa-qaaea-cai -``` - -Open the Candid web UI for the backend (the `send_http_get_motoko_backend` one) and call the `get_icp_usd_exchange()` method: - -![Candid web UI](../_attachments/https-get.webp) - - - - -Test the dapp locally. - -Deploy the dapp locally: - -```bash -dfx start --clean --background -dfx deploy -``` - -If successful, the terminal should return canister URLs you can open: - -```bash -Deployed canisters. -URLs: - Frontend canister via browser - send_http_get_rust_frontend: http://127.0.0.1:4943/?canisterId=ajuq4-ruaaa-aaaaa-qaaga-cai - Backend canister via Candid interface: - send_http_get_rust_backend: http://127.0.0.1:4943/?canisterId=aovwi-4maaa-aaaaa-qaagq-cai&id=a4tbr-q4aaa-aaaaa-qaafq-cai -``` - -Open the Candid web UI for the backend (the `send_http_get_rust_backend` one) and call the `get_icp_usd_exchange()` method: - -![Candid web UI](../_attachments/https-get-candid-3-rust.webp) - +:::caution -- #### Step 5: Test the dapp on mainnet. - - - - -Deploy the dapp locally: - -```bash -dfx deploy --network ic -``` - -If successful, the terminal should return canister URLs you can open: - -```bash -Committing batch. -Deployed canisters. -URLs: - Frontend canister via browser - send_http_get_motoko_frontend: https://ff5va-7qaaa-aaaap-qbona-cai.icp0.io/ - Backend canister via Candid interface: - send_http_get_motoko_backend: https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai -``` - - - - -Deploy the dapp to mainnet: - -```bash -dfx deploy --network ic -``` +Headers in the response may not always be identical across all nodes that process the request for consensus, causing the result of the call to be "No consensus could be reached." This particular error message can be hard to debug, but one method to resolve this error is to edit the response using the transform function. The transform function is run before consensus, and can be used to remove some headers from the response. -If successful, the terminal should return canister URLs you can open: +::: -```bash -Committing batch. -Deployed canisters. -URLs: - Frontend canister via browser - send_http_get_rust_frontend: https://ff5va-7qaaa-aaaap-qbona-cai.icp0.io/ - Backend canister via Candid interface: - send_http_get_rust_backend: https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai -``` -You can see play with the dapp's `get_icp_usd_exchange` method onchain here: [https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai](https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai). +You can see a deployed version of this canister and its `get_icp_usd_exchange` method onchain here: [https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai](https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai). - - ## Additional resources diff --git a/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-how-to-use.mdx b/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-how-to-use.mdx index 438a28dc85..62288f1a15 100644 --- a/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-how-to-use.mdx +++ b/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-how-to-use.mdx @@ -41,15 +41,7 @@ The following parameters should be supplied within the request: - `body`: Optional; The content of the request's body. -- `transform`: An optional function that transforms raw responses to [sanitized responses](https://en.wikipedia.org/wiki/HTML_sanitization), and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. An example written in Rust is shown below: - -```rust -async fn transform(raw: CanisterHttpResponsePayload) -> CanisterHttpResponsePayload { - let mut sanitized = raw.clone(); - sanitized.headers = vec![]; - sanitized -} -``` +- `transform`: An optional function that transforms raw responses to [sanitized responses](https://en.wikipedia.org/wiki/HTML_sanitization), and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. ### The response diff --git a/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-post.mdx b/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-post.mdx index e750ab4f23..59371eb8f4 100644 --- a/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-post.mdx +++ b/docs/developer-docs/smart-contracts/advanced-features/https-outcalls/https-outcalls-post.mdx @@ -14,35 +14,12 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; ## Overview -A minimal example of how to make a `POST` HTTP request. The purpose of this dapp is only to show how to make HTTP requests from a canister. - -The sample code is in both Motoko and Rust. This sample canister sends a `POST` request with some JSON to a free API where you can verify the headers and body were sent correctly. - -**The main intent of this canister is to show developers how to make idempotent `POST` requests.** - -This example takes less than 5 minutes to complete. +A minimal example of how to make a `POST` HTTP request. The purpose of this dapp is only to show how to make HTTP requests from a canister. It sends a `POST` request with some JSON to a free API where you can verify the headers and body were sent correctly. :::caution -The HTTPS outcalls feature only works for sending HTTP POST requests to servers or API endpoints that support **IPV6**. +The HTTPS outcalls feature only works for sending HTTP `POST` requests to servers or API endpoints that support **IPV6**. ::: -### Candid web UI of canister - -The canister in this tutorial will have only **one public method** which, when called, will trigger an HTTP `POST` request. The canister will not have a frontend (only a backend), but like all canisters, you can interact with its public methods via the Candid web UI, which will look like this: - -![Candid web UI](../_attachments/https-post-candid-2-motoko.webp) - -When you call the method, the canister will send an HTTP `POST` request with the following JSON in the response body: - -```json -{ - "name": "Grogu", - "force_sensitive": "true" -} -``` - -### Verifying the HTTP POST request - In order to verify that your canister sent the HTTP request you expected, this canister is sending HTTP requests to a [public API service](https://putsreq.com/aL1QS5IbaQd4NTqN3a81/inspect) where the HTTP request can be inspected. As you can see the image below, the `POST` request headers and body can be inspected to make sure it is what the canister sent. ![Public API to inspect POST request](../../_attachments/https-post-requestbin-result.webp) @@ -55,696 +32,42 @@ The recommended way for HTTP `POST` requests is to add the idempotency keys in t Developers should be careful that the destination server understands and uses idempotency keys. A canister can be coded to send idempotency keys, but it is ultimately up to the recipient server to know what to do with them. Here is an [example of an API service that uses idempotency keys](https://stripe.com/docs/api/idempotent_requests). -## Code structure - -Before you dive in, here is the structure of the code you will touch: +## `POST` example -```motoko no-repl - -//Import some custom types from `src/backend_canister/Types.mo` file -import Types "Types"; - -actor { - -//method that uses the HTTPS outcalls feature and returns a string - public func foo() : async Text { - - //1. DECLARE MANAGEMENT CANISTER - let ic : Types.IC = actor ("aaaaa-aa"); - - //2. SETUP ARGUMENTS FOR HTTP GET request - let request : Types.HttpRequestArgs = { - //construct the request - }; - - //3. ADD CYCLES TO PAY FOR HTTP REQUEST - //code to add cycles - - //4. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - let response : Types.HttpResponsePayload = await ic.http_request(request); - - //5. DECODE THE RESPONSE - //code to decode response - - //6. RETURN RESPONSE OF THE BODY - response - }; -}; -``` - -You will also create some custom types in `Types.mo`. This will look like this: - -```motoko no-repl -module Types { - - //type declarations for HTTP requests, HTTP responses, management canister, etc... - -} +```motoko no-repl file=../../../../references/samples/motoko/send_http_post/src/send_http_post_backend/main.mo ``` -```rust -//1. DECLARE MANAGEMENT CANISTER -use ic_cdk::api::management_canister::http_request::{ - http_request, CanisterHttpRequestArgument, HttpHeader, HttpMethod, HttpResponse, TransformArgs, - TransformContext, -}; - -//Update method using the HTTPS outcalls feature -#[ic_cdk::update] -async fn foo() { - //2. SETUP ARGUMENTS FOR HTTP GET request - let request = CanisterHttpRequestArgument { - //instantiate the request - }; - - //3. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - //Note: in Rust, `http_request()` already sends the cycles needed - //so no need for explicit Cycles.add() as in Motoko - match http_request(request).await { - - //4. DECODE AND RETURN THE RESPONSE - Ok((response,)) => { - //Ok case - } - Err((r, m)) => { - //error case - } - } -} +```rust file=../../../../references/samples/rust/send_http_post/src/send_http_post_backend/src/lib.rs ``` - - - -- #### Step 1: Create a new project by running the following command: +To use HTTPS outcalls you must update the canister's Candid file: - - - -```bash -dfx new send_http_post_motoko -cd send_http_post_motoko -npm install +```candid file=../../../../references/samples/rust/send_http_post/src/send_http_post_backend/send_http_post_backend.did ``` - - +Update the `Cargo.toml` file to use the correct dependencies: -```bash -dfx new send_http_post_rust --type=rust -cd send_http_post_rust -npm install -rustup target add wasm32-unknown-unknown +```toml file=../../../../references/samples/rust/send_http_post/src/send_http_post_backend/Cargo.toml ``` +:::caution +Headers in the response may not always be identical across all nodes that process the request for consensus, causing the result of the call to be "No consensus could be reached." This particular error message can be hard to debug, but one method to resolve this error is to edit the response using the transform function. The transform function is run before consensus, and can be used to remove some headers from the response. -- #### Step 2: Edit the backend canister's code. - - - - -Open the `src/send_http_post_motoko_backend/main.mo` file in a text editor and replace content with: - -```motoko no-repl title="src/send_http_post_motoko_backend/main.mo" -import Debug "mo:base/Debug"; -import Blob "mo:base/Blob"; -import Cycles "mo:base/ExperimentalCycles"; -import Array "mo:base/Array"; -import Nat8 "mo:base/Nat8"; -import Text "mo:base/Text"; - -//import the custom types you have in Types.mo -import Types "Types"; - -actor { - - //function to transform the response - public query func transform(raw : Types.TransformArgs) : async Types.CanisterHttpResponsePayload { - let transformed : Types.CanisterHttpResponsePayload = { - status = raw.response.status; - body = raw.response.body; - headers = [ - { - name = "Content-Security-Policy"; - value = "default-src 'self'"; - }, - { name = "Referrer-Policy"; value = "strict-origin" }, - { name = "Permissions-Policy"; value = "geolocation=(self)" }, - { - name = "Strict-Transport-Security"; - value = "max-age=63072000"; - }, - { name = "X-Frame-Options"; value = "DENY" }, - { name = "X-Content-Type-Options"; value = "nosniff" }, - ]; - }; - transformed; - }; - -//PUBLIC METHOD -//This method sends a POST request to a URL with a free API you can test. - public func send_http_post_request() : async Text { - - //1. DECLARE MANAGEMENT CANISTER - //You need this so you can use it to make the HTTP request - let ic : Types.IC = actor ("aaaaa-aa"); - - //2. SETUP ARGUMENTS FOR HTTP GET request - - // 2.1 Setup the URL and its query parameters - //This URL is used because it allows you to inspect the HTTP request sent from the canister - let host : Text = "putsreq.com"; - let url = "https://putsreq.com/aL1QS5IbaQd4NTqN3a81"; //HTTPS that accepts IPV6 - - // 2.2 Prepare headers for the system http_request call - - //idempotency keys should be unique so create a function that generates them. - let idempotency_key: Text = generateUUID(); - let request_headers = [ - { name = "Host"; value = host # ":443" }, - { name = "User-Agent"; value = "http_post_sample" }, - { name= "Content-Type"; value = "application/json" }, - { name= "Idempotency-Key"; value = idempotency_key } - ]; - - // The request body is an array of [Nat8] (see Types.mo) so do the following: - // 1. Write a JSON string - // 2. Convert ?Text optional into a Blob, which is an intermediate representation before you cast it as an array of [Nat8] - // 3. Convert the Blob into an array [Nat8] - let request_body_json: Text = "{ \"name\" : \"Grogu\", \"force_sensitive\" : \"true\" }"; - let request_body_as_Blob: Blob = Text.encodeUtf8(request_body_json); - let request_body_as_nat8: [Nat8] = Blob.toArray(request_body_as_Blob); // e.g [34, 34,12, 0] - - - // 2.2.1 Transform context - let transform_context : Types.TransformContext = { - function = transform; - context = Blob.fromArray([]); - }; - - // 2.3 The HTTP request - let http_request : Types.HttpRequestArgs = { - url = url; - max_response_bytes = null; //optional for request - headers = request_headers; - //note: type of `body` is ?[Nat8] so it is passed here as "?request_body_as_nat8" instead of "request_body_as_nat8" - body = ?request_body_as_nat8; - method = #post; - transform = ?transform_context; - // transform = null; //optional for request - }; - - //3. ADD CYCLES TO PAY FOR HTTP REQUEST - - //The management canister will make the HTTP request so it needs cycles - //See: /docs/current/motoko/main/canister-maintenance/cycles - - //The way Cycles.add() works is that it adds those cycles to the next asynchronous call - //See: /docs/current/references/ic-interface-spec#ic-http_request - Cycles.add(21_850_258_000); - - //4. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - //Since the cycles were added above, you can just call the management canister with HTTPS outcalls below - let http_response : Types.HttpResponsePayload = await ic.http_request(http_request); - - //5. DECODE THE RESPONSE - - //As per the type declarations in `Types.mo`, the BODY in the HTTP response - //comes back as [Nat8s] (e.g. [2, 5, 12, 11, 23]). Type signature: - - //public type HttpResponsePayload = { - // status : Nat; - // headers : [HttpHeader]; - // body : [Nat8]; - // }; - - // You need to decode that [Na8] array that is the body into readable text. - //To do this: - // 1. Convert the [Nat8] into a Blob - // 2. Use Blob.decodeUtf8() method to convert the Blob to a ?Text optional - // 3. Use Motoko syntax "Let... else" to unwrap what is returned from Text.decodeUtf8() - let response_body: Blob = Blob.fromArray(http_response.body); - let decoded_text: Text = switch (Text.decodeUtf8(response_body)) { - case (null) { "No value returned" }; - case (?y) { y }; - }; - - //6. RETURN RESPONSE OF THE BODY - let result: Text = decoded_text # ". See more info of the request sent at at: " # url # "/inspect"; - result - }; - - //PRIVATE HELPER FUNCTION - //Helper method that generates a Universally Unique Identifier - //this method is used for the Idempotency Key used in the request headers of the POST request. - //For the purposes of this exercise, it returns a constant, but in practice, it should return unique identifiers - func generateUUID() : Text { - "UUID-123456789"; - } -}; -``` - - - - -Open the `src/send_http_post_rust_backend/src/lib.rs` file in a text editor and replace the content with: - -```rust title="src/send_http_post_rust_backend/src/lib.rs" -//1. IMPORT MANAGEMENT CANISTER -//This includes all methods and types needed -use ic_cdk::api::management_canister::http_request::{ - http_request, CanisterHttpRequestArgument, HttpHeader, HttpMethod, HttpResponse, TransformArgs, - TransformContext, -}; - -use serde::{Serialize, Deserialize}; -use serde_json::{self, Value}; - -// This struct is legacy code and is not really used in the code. -#[derive(Serialize, Deserialize)] -struct Context { - bucket_start_time_index: usize, - closing_price_index: usize, -} - -//Update method using the HTTPS outcalls feature -#[ic_cdk::update] -async fn send_http_post_request() -> String { - //2. SETUP ARGUMENTS FOR HTTP GET request - - // 2.1 Setup the URL - let host = "putsreq.com"; - let url = "https://putsreq.com/aL1QS5IbaQd4NTqN3a81"; - - // 2.2 Prepare headers for the system http_request call - //Note that `HttpHeader` is declared in line 4 - let request_headers = vec![ - HttpHeader { - name: "Host".to_string(), - value: format!("{host}:443"), - }, - HttpHeader { - name: "User-Agent".to_string(), - value: "demo_HTTP_POST_canister".to_string(), - }, - //For the purposes of this exercise, Idempotency-Key" is hard coded, but in practice - //it should be generated via code and unique to each POST request. Common to create helper methods for this - HttpHeader { - name: "Idempotency-Key".to_string(), - value: "UUID-123456789".to_string(), - }, - HttpHeader { - name: "Content-Type".to_string(), - value: "application/json".to_string(), - }, - ]; - - //note "CanisterHttpRequestArgument" and "HttpMethod" are declared in line 4. - //CanisterHttpRequestArgument has the following types: - - // pub struct CanisterHttpRequestArgument { - // pub url: String, - // pub max_response_bytes: Option, - // pub method: HttpMethod, - // pub headers: Vec, - // pub body: Option>, - // pub transform: Option, - // } - //see: https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/struct.CanisterHttpRequestArgument.html - - //Where "HttpMethod" has structure: - // pub enum HttpMethod { - // GET, - // POST, - // HEAD, - // } - //See: https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/enum.HttpMethod.html - - //Since the body in HTTP request has type Option> it needs to look something like this: Some(vec![104, 101, 108, 108, 111]) ("hello" in ASCII) - //where the vector of u8s are the UTF. In order to send JSON via POST, do the following: - //1. Declare a JSON string to send - //2. Convert that JSON string to array of UTF8 (u8) - //3. Wrap that array in an optional - let json_string : String = "{ \"name\" : \"Grogu\", \"force_sensitive\" : \"true\" }".to_string(); - - //note: here, r#""# is used for raw strings in Rust, which allows you to include characters like " and \ without needing to escape them. - //You could have used "serde_json" as well. - let json_utf8: Vec = json_string.into_bytes(); - let request_body: Option> = Some(json_utf8); - - // This struct is legacy code and is not really used in the code. Need to be removed in the future - // The "TransformContext" function does need a CONTEXT parameter, but this implementation is not necessary - // the TransformContext(transform, context) below accepts this "context", but it does nothing with it in this implementation. - // bucket_start_time_index and closing_price_index are meaningless - let context = Context { - bucket_start_time_index: 0, - closing_price_index: 4, - }; - - let request = CanisterHttpRequestArgument { - url: url.to_string(), - max_response_bytes: None, //optional for request - method: HttpMethod::POST, - headers: request_headers, - body: request_body, - transform: Some(TransformContext::new(transform, serde_json::to_vec(&context).unwrap())), - }; - - //3. MAKE HTTP REQUEST AND WAIT FOR RESPONSE - - //Note: in Rust, `http_request()` already sends the cycles needed - //so no need for explicit Cycles.add() as in Motoko - match http_request(request).await { - //4. DECODE AND RETURN THE RESPONSE - - //See:https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/struct.HttpResponse.html - Ok((response,)) => { - //if successful, `HttpResponse` has this structure: - // pub struct HttpResponse { - // pub status: Nat, - // pub headers: Vec, - // pub body: Vec, - // } - - //You need to decode that Vec that is the body into readable text. - //To do this: - // 1. Call `String::from_utf8()` on response.body - // 3. Use a switch to explicitly call out both cases of decoding the Blob into ?Text - let str_body = String::from_utf8(response.body) - .expect("Transformed response is not UTF-8 encoded."); - ic_cdk::api::print(format!("{:?}", str_body)); - - //The API response will looks like this: - // { successful: true } - - //Return the body as a string and end the method - let result: String = format!( - "{}. See more info of the request sent at: {}/inspect", - str_body, url - ); - result - } - Err((r, m)) => { - let message = - format!("The http_request resulted into error. RejectionCode: {r:?}, Error: {m}"); - - //Return the error as a string and end the method - message - } - } - -} - -// Strips all data that is not needed from the original response. -#[ic_cdk::query] -fn transform(raw: TransformArgs) -> HttpResponse { - - let headers = vec![ - HttpHeader { - name: "Content-Security-Policy".to_string(), - value: "default-src 'self'".to_string(), - }, - HttpHeader { - name: "Referrer-Policy".to_string(), - value: "strict-origin".to_string(), - }, - HttpHeader { - name: "Permissions-Policy".to_string(), - value: "geolocation=(self)".to_string(), - }, - HttpHeader { - name: "Strict-Transport-Security".to_string(), - value: "max-age=63072000".to_string(), - }, - HttpHeader { - name: "X-Frame-Options".to_string(), - value: "DENY".to_string(), - }, - HttpHeader { - name: "X-Content-Type-Options".to_string(), - value: "nosniff".to_string(), - }, - ]; - - - let mut res = HttpResponse { - status: raw.response.status.clone(), - body: raw.response.body.clone(), - headers, - ..Default::default() - }; - - if res.status == 200 { - - res.body = raw.response.body; - } else { - ic_cdk::api::print(format!("Received an error from coinbase: err = {:?}", raw)); - } - res -} -``` - -- `send_http_post_request() -> String` returns a `String`, but this is not necessary. In this tutorial, this is done for easier testing. -- The `lib.rs` file used [http_request](https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/fn.http_request.html) which is a convenient Rust CDK method that already sends cycles to the management canister under the hood. It knows how many cycles to send for a 13-node subnet and most cases. If your HTTPS outcall needs more cycles, you should use [http_request_with_cycles()](https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/fn.http_request_with_cycles.html) method and explicitly call the cycles needed. -- The Rust CDK method `http_request` used above wraps the management canister method [`http_request`](/docs/current/references/ic-interface-spec#ic-http_request), but it is not strictly the same. - - - - - - -- #### Step 3: Edit the Type or Candid files. - - - - -Create the `src/send_http_post_motoko_backend/Types.mo` file in a text editor and replace content with: - -```motoko no-repl title="src/send_http_post_motoko_backend/Types.mo" -module Types { - - //1. Type that describes the Request arguments for an HTTPS outcall - //See: /docs/current/references/ic-interface-spec#ic-http_request - public type HttpRequestArgs = { - url : Text; - max_response_bytes : ?Nat64; - headers : [HttpHeader]; - body : ?[Nat8]; - method : HttpMethod; - transform : ?TransformRawResponseFunction; - }; - - public type HttpHeader = { - name : Text; - value : Text; - }; - - public type HttpMethod = { - #get; - #post; - #head; - }; - - public type HttpResponsePayload = { - status : Nat; - headers : [HttpHeader]; - body : [Nat8]; - }; - - //2. HTTPS outcalls have an optional "transform" key. These two types help describe it. - //"The transform function may, for example, transform the body in any way, add or remove headers, - //modify headers, etc. " - //See: /docs/current/references/ic-interface-spec#ic-http_request - - //2.1 This type describes a function called "TransformRawResponse" used in line 14 above - //"If provided, the calling canister itself must export this function." - //In this minimal example for a GET request, declare the type for completeness, but - //you do not use this function. You will pass "null" to the HTTP request. - public type TransformRawResponseFunction = { - function : shared query TransformArgs -> async HttpResponsePayload; - context : Blob; - }; - - //2.2 This type describes the arguments the transform function needs - public type TransformArgs = { - response : HttpResponsePayload; - context : Blob; - }; - - public type CanisterHttpResponsePayload = { - status : Nat; - headers : [HttpHeader]; - body : [Nat8]; - }; - - public type TransformContext = { - function : shared query TransformArgs -> async HttpResponsePayload; - context : Blob; - }; - - - //3. Declaring the management canister which is used to make the HTTPS outcall - public type IC = actor { - http_request : HttpRequestArgs -> async HttpResponsePayload; - }; - -} - -``` - - - - -Open the `src/hello_http_rust_backend/hello_http_rust_backend.did` file in a text editor and replace content with: - -Update the Candid interface file so it matches the method `send_http_post_request()` in `lib.rs`. - -```candid title="src/hello_http_rust_backend/hello_http_rust_backend.did" -service : { - "send_http_post_request": () -> (text); -} -``` - -Open the `src/send_http_post_rust_backend/Cargo.toml` file in a text editor and replace content with: - -```bash title="src/send_http_post_rust_backend/Cargo.toml" -[package] -name = "send_http_post_rust_backend" -version = "0.1.0" -edition = "2021" - -# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html - -[lib] -crate-type = ["cdylib"] - -[dependencies] -candid = "0.10" -ic-cdk = "0.13" -serde = "1.0.152" -serde_json = "1.0.93" -serde_bytes = "0.11.9" - -``` - - - - - - -- #### Step 4: Test the dapp locally. - - - - -Deploy the dapp locally: - -```bash -dfx start --clean --background -dfx deploy -``` - -If successful, the terminal should return canister URLs you can open: - -```bash -Deployed canisters. -URLs: - Backend canister via Candid interface: - send_http_post_motoko_backend: http://127.0.0.1:4943/?canisterId=dccg7-xmaaa-aaaaa-qaamq-cai&id=dfdal-2uaaa-aaaaa-qaama-cai -``` - -Open the candid web UI for the backend and call the `send_http_post_motoko_request()` method: - -![Candid web UI](../_attachments/https-post-candid-2-motoko.webp) - - - - -Deploy the dapp locally: - -```bash -dfx start --clean --background -dfx deploy -``` - -If successful, the terminal should return canister URLs you can open: - -```bash -Deployed canisters. -URLs: - Backend canister via Candid interface: - send_http_post_rust_backend: http://127.0.0.1:4943/?canisterId=dxfxs-weaaa-aaaaa-qaapa-cai&id=dzh22-nuaaa-aaaaa-qaaoa-cai -``` - -Open the candid web UI for backend and call the `send_http_post_request()` method: - -![Candid web UI](../_attachments/https-post-candid-2-motoko.webp) - - - - -- #### Step 5: Test the dapp on mainnet. - - - - -Deploy the dapp to mainnet: - -```bash -dfx deploy --network ic -``` - -If successful, the terminal should return canister URLs you can open: - -```bash -Committing batch. -Deployed canisters. -URLs: - Frontend canister via browser - send_http_post_motoko_frontend: https://fx3cz-taaaa-aaaap-qbooa-cai.icp0.io/ - Backend canister via Candid interface: - send_http_post_motoko_backend: https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fc4tu-siaaa-aaaap-qbonq-cai -``` - -You can see play with the dapp's `send_http_post_request` method onchain here: [https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fc4tu-siaaa-aaaap-qbonq-cai](https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fc4tu-siaaa-aaaap-qbonq-cai). - - - - -Deploy the dapp to mainnet: - -```bash -dfx deploy --network ic -``` - -If successful, the terminal should return canister URLs you can open: - -```bash -Committing batch. -Deployed canisters. -URLs: - Frontend canister via browser - send_http_post_rust_frontend: https://f6yjf-fiaaa-aaaap-qbopq-cai.icp0.io/ - Backend canister via Candid interface: - send_http_post_rust_backend: https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fzzpr-iqaaa-aaaap-qbopa-cai -``` - -You can see play with the dapp's `send_http_post_request` method onchain here: [https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai](https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai). +::: - - +You can see a deployed version of this canister's `send_http_post_request` method onchain here: [https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai](https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai). ## Additional resources diff --git a/docs/developer-docs/smart-contracts/advanced-features/serving-http-request.mdx b/docs/developer-docs/smart-contracts/advanced-features/serving-http-request.mdx deleted file mode 100644 index 51824af6f9..0000000000 --- a/docs/developer-docs/smart-contracts/advanced-features/serving-http-request.mdx +++ /dev/null @@ -1,272 +0,0 @@ ---- -keywords: [advanced, tutorial, serving http, http requests] ---- - -import TabItem from "@theme/TabItem"; -import { AdornedTabs } from "/src/components/Tabs/AdornedTabs"; -import { AdornedTab } from "/src/components/Tabs/AdornedTab"; -import { BetaChip } from "/src/components/Chip/BetaChip"; -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; - -# Serving an HTTP request - - - -## Overview - -Canisters can serve or handle an incoming HTTP request using the -[HTTP Gateway Protocol](/docs/current/references/http-gateway-protocol-spec). - -This allows developers to host web applications and APIs from a canister. - -## How it works - -An HTTP request from a client gets intercepted by the HTTP Gateway Protocol, -which identifies the target canister and encodes the request in Candid. This -encoded request is then sent to the canister for processing. Once processed, the -canister replies with an HTTP response. Finally, the HTTP Gateway Protocol -decodes the response using Candid and sends it back to the client, completing -the communication loop. - -For detailed information on how it works, please refer to the -[HTTP Gateway Protocol specification](/docs/current/references/http-gateway-protocol-spec). - -## How to make a request - -The following example returns 'Hello, World!' in the body at the `/hello` -endpoint. - - - - -```motoko -import HashMap = "mo:base/HashMap"; -import Blob = "mo:base/Blob"; -import Text "mo:base/Text"; -import Option "mo:base/Option"; - -actor { - - public type HttpRequest = { - body: Blob; - headers: [HeaderField]; - method: Text; - url: Text; - }; - - public type ChunkId = Nat; - public type SetAssetContentArguments = { - chunk_ids: [ChunkId]; - content_encoding: Text; - key: Key; - sha256: ?Blob; - }; - public type Path = Text; - public type Key = Text; - - public type HttpResponse = { - body: Blob; - headers: [HeaderField]; - status_code: Nat16; - }; - - public type HeaderField = (Text, Text); - - private func removeQuery(str: Text): Text { - return Option.unwrap(Text.split(str, #char '?').next()); - }; - - public query func http_request(req: HttpRequest): async (HttpResponse) { - let path = removeQuery(req.url); - if(path == "/hello") { - return { - body = Text.encodeUtf8("root page :" # path); - headers = []; - status_code = 200; - }; - }; - - return { - body = Text.encodeUtf8("404 Not found :" # path); - headers = []; - status_code = 404; - }; - }; -} -``` - - - - - -```rust -type HeaderField = (String, String); - -struct HttpResponse { - status_code: u16, - headers: Vec, - body: Cow<'static, Bytes>, -} - -struct HttpRequest { - method: String, - url: String, - headers: Vec<(String, String)>, - body: ByteBuf, -} - -#[query] -fn http_request(req: HttpRequest) -> HttpResponse { - let path = req.url.path(); - if path == "/hello" { - HttpResponse { - status_code: 200, - headers: Vec::new(), - body: b"hello, world!".to_vec(), - streaming_strategy: None, - upgrade: None, - } - } else { - HttpResponse { - status_code: 404, - headers: Vec::new(), - body: b"404 Not found :".to_vec(), - streaming_strategy: None, - upgrade: None, - } - } -} -``` - - - -}> - -```typescript -import { IDL, query } from 'azle'; - -type HeaderField = [string, string]; -const HeaderField = IDL.Tuple(IDL.Text, IDL.Text); - -type HttpResponse = { - status_code: number; - headers: HeaderField[]; - body: Uint8Array; -}; -const HttpResponse = IDL.Record({ - status_code: IDL.Nat16, - headers: IDL.Vec(HeaderField), - body: IDL.Vec(IDL.Nat8) -}); - -type HttpRequest = { - method: string; - url: string; - headers: HeaderField[]; - body: Uint8Array; -}; -const HttpRequest = IDL.Record({ - method: IDL.Text, - url: IDL.Text, - headers: IDL.Vec(HeaderField), - body: IDL.Vec(IDL.Nat8) -}); - -export default class { - @query([HttpRequest], HttpResponse) - getThing(req: HttpRequest): HttpResponse { - const path = new URL(req.url).pathname; - if (path === '/hello') { - return { - status_code: 200, - headers: [], - body: Buffer.from('hello, world!') - }; - } - return { - body: new TextEncoder().encode('404 Not found :'), - headers: [], - status_code: 404 - }; - } -} -``` - - - -}> - -```python -from kybra import blob, Func, nat16, Opt, query, Query, Record, Tuple, Variant, Vec - - -class HttpRequest(Record): - method: str - url: str - headers: Vec["Header"] - body: blob - - -class HttpResponse(Record): - status_code: nat16 - headers: Vec["Header"] - body: blob - streaming_strategy: Opt["StreamingStrategy"] - upgrade: Opt[bool] - - -Header = Tuple[str, str] - - -class StreamingStrategy(Variant): - Callback: "CallbackStrategy" - - -class CallbackStrategy(Record): - callback: "Callback" - token: "Token" - - -Callback = Func(Query[["Token"], "StreamingCallbackHttpResponse"]) - - -class StreamingCallbackHttpResponse(Record): - body: blob - token: Opt["Token"] - - -class Token(Record): - arbitrary_data: str - - -@query -def http_request(req: HttpRequest) -> HttpResponse: - path = req.url.path - if path == "/hello": - return { - "status_code": 200, - "headers": [], - "body": Buffer.from_text("hello, world!").encode("utf-8"), - } - else: - return { - "status_code": 404, - "headers": [], - "body": b"404 Not found :", -} -``` - -To learn more about serving an HTTP request in Python, refer to -[the Kybra book reference on incoming HTTP requests](https://demergent-labs.github.io/kybra/http.html). - - - - -## Additional examples - -The -[HTTP counter project](https://github.com/dfinity/examples/tree/master/motoko/http_counter) -is an example in Motoko of a 'counter' application that uses the `http_request` -method to read the current counter value or access some pre-stored data and the -`http_request_update` method to increment the counter and retrieve the updated -value. diff --git a/docs/developer-docs/smart-contracts/test/overview.mdx b/docs/developer-docs/smart-contracts/test/overview.mdx index 7ee17e05d8..65f8e6da18 100644 --- a/docs/developer-docs/smart-contracts/test/overview.mdx +++ b/docs/developer-docs/smart-contracts/test/overview.mdx @@ -34,7 +34,7 @@ For Rust unit testing, refer to the guide on [effective Rust containers](https:/ ### Integration testing -Integration testing refers to testing multiple components of a code or application together to ensure that they operate correctly. Integration testing often includes testing how several functions within a canister interact with one another, how different canisters may send and receive information from each other, or how a canister may obtain and digest external information from HTTPS outcalls. One common form of integration testing is continuous integration (CI) testing. CI testing uses an automated workflow to continuously test several components of an application. +Integration testing refers to testing multiple components of an application together to ensure that they operate correctly. Integration testing often includes testing how several functions within a canister interact with one another, how different canisters may send and receive information from each other, or how a canister may obtain and digest external information from HTTPS outcalls. One common form of integration testing is continuous integration (CI) testing. CI testing uses an automated workflow to continuously test several components of an application. GitHub CI testing is a common workflow used by developers that can be configured to run tests whenever new code is pushed to a repository. [Learn more about CI tests](https://docs.github.com/en/actions/using-workflows). @@ -63,8 +63,4 @@ PocketIC is a comprehensive testing suite that can be used for testing canisters Light Replica is a community-contributed tool designed to replicate the local testing environment that the Ethereum tools Hardhat or Truffle provide. Light Replica replicates the behavior of a real ICP node by running a local instance of a node, then providing additional logging and functionality to enable canister testing. Light Replica can be used to test any Wasm file as if it were a canister deployed on the mainnet. -[Learn more about Light Replica](https://github.com/icopen/lightic/tree/main). - -### Manual staging environments - -As an alternative to using the environments provided by PocketIC or Light Replica, you can also manually make a staging environment with `dfx` that you can test your code against. [Learn more about creating a staging environment with dfx](staging-environment.mdx). \ No newline at end of file +[Learn more about Light Replica](https://github.com/icopen/lightic/tree/main). \ No newline at end of file diff --git a/docs/developer-docs/smart-contracts/test/pocket-ic.mdx b/docs/developer-docs/smart-contracts/test/pocket-ic.mdx index b8a265f851..d3af3f91e9 100644 --- a/docs/developer-docs/smart-contracts/test/pocket-ic.mdx +++ b/docs/developer-docs/smart-contracts/test/pocket-ic.mdx @@ -42,27 +42,12 @@ While PocketIC can support integration with any language, Rust and Python will b ## PocketIC Rust -To use PocketIC, the latest PocketIC server binary must be downloaded from the PocketIC repo. You can download the latest version for macOS or Linux systems [here](https://github.com/dfinity/pocketic#download-the-pocketic-server). +To use PocketIC, the latest PocketIC server binary must be downloaded from the PocketIC repo. [Download the latest version for macOS or Linux systems](https://github.com/dfinity/pocketic?tab=readme-ov-file#download-the-pocketic-server). :::info PocketIC is currently not natively supported on Windows systems. ::: -Once the binary file is downloaded, unzip the file and make it executable: - -```bash -gzip -d pocket-ic.gz -chmod +x pocket-ic -``` - -:::info -On macOS systems, to bypass developer verification from Apple, you may need to run: - -``` -xattr -dr com.apple.quarantine pocket-ic -``` -::: - The PocketIC binary can be left in your working directory or you can specify the path to the binary by setting the `POCKET_IC_BIN` environmental variable. Add PocketIC Rust do your project with the command: @@ -83,37 +68,11 @@ Then, in your code you can create a new PocketIC instance with the code: let pic = PocketIc::new(); ``` -### Example: Single canister testing on a single subnet +### Single canister testing on a single subnet -The following is a simple, but complete testing example with a counter canister: +[View an example of testing a canister on a single subnet](https://crates.io/crates/pocket-ic). -```rust -use candid::encode_one; -use pocket_ic::PocketIc; - - #[test] - fn test_counter_canister() { - let pic = PocketIc::new(); - // Create an empty canister as the anonymous principal and add cycles. - let canister_id = pic.create_canister(); - pic.add_cycles(canister_id, 2_000_000_000_000); - - let wasm_bytes = load_counter_wasm(...); - pic.install_canister(canister_id, wasm_bytes, vec![], None); - // 'inc' is a counter canister method. - call_counter_canister(&pic, canister_id, "inc"); - // Check if it had the desired effect. - let reply = call_counter_canister(&pic, canister_id, "read"); - assert_eq!(reply, WasmResult::Reply(vec![0, 0, 0, 1])); - } - -fn call_counter_canister(pic: &PocketIc, canister_id: CanisterId, method: &str) -> WasmResult { - pic.update_call(canister_id, Principal::anonymous(), method, encode_one(()).unwrap()) - .expect("Failed to call counter canister") -} -``` - -### Example: Multi-subnet testing +### Multi-subnet testing Versions of PocketIC `v2.0.0` and newer support multi-subnet testing. Multi-subnet testing allows for simulating multiple, different types of subnets locally. For example, to create an IC instance with an NNS subnet and two application subnets, use the code: @@ -160,26 +119,7 @@ For a larger, more complex example that uses cross canister calls on two differe ## PocketIC Python -To use PocketIC, the latest PocketIC server binary must be downloaded from the PocketIC repo. You can download the latest version for macOS or Linux systems [here](https://github.com/dfinity/pocketic#download-the-pocketic-server). - -:::info -PocketIC is currently not supported natively on Windows systems. -::: - -Once the binary file is downloaded, unzip the file and make it executable: - -```bash -gzip -d pocket-ic.gz -chmod +x pocket-ic -``` - -:::info -On macOS systems, to bypass developer verification from Apple, you may need to run: - -```bash -xattr -dr com.apple.quarantine pocket-ic -``` -::: +To use PocketIC, the latest PocketIC server binary must be downloaded from the PocketIC repo. [Download the latest version for macOS or Linux systems](https://github.com/dfinity/pocketic#download-the-pocketic-server). The PocketIC binary can be left in your working directory or you can specify the path to the binary by setting the `POCKET_IC_BIN` environmental variable. @@ -195,22 +135,9 @@ To import PocketIC into your Python code, use the line: from pocket_ic import PocketIC ``` -### Example: Create a canister with cycles +### Create a canister with cycles and call the canister -```python -from pocket_ic import PocketIC - -pic = PocketIC() -canister_id = pic.create_canister() -pic.add_cycles(canister_id, 2_000_000_000_000) -``` - -### Make canister calls - -```python -response = pic.update_call(canister_id, method="greeting", ...) -assert(response == 'Hello, PocketIC!') -``` +[View an example of how to create a canister with cycles, then call the canister](https://pypi.org/project/pocket-ic/). ### Make a call directly with a canister object @@ -225,49 +152,9 @@ response = my_canister.greeting() assert(response == 'Hello, PocketIC!') ``` -### Example: Single canister testing on a single subnet +### Single canister testing on a single subnet -The following is a simple, but complete testing example with a counter canister: - -```python title="src/example_backend/src/main.py" -import sys -import os -import unittest -import ic -from pocket_ic import PocketIC - - -class CounterCanisterTests(unittest.TestCase): - def test_counter_canister(self): - pic = PocketIC() - canister_id = pic.create_canister() - pic.add_cycles(canister_id, 1_000_000_000_000_000_000) - - with open("counter.wasm", "rb") as wasm_file: - wasm_module = wasm_file.read() - pic.install_code(canister_id, bytes(wasm_module), []) - - self.assertEqual( - pic.update_call(canister_id, "read", ic.encode([])), - [0, 0, 0, 0], - ) - self.assertEqual( - pic.update_call(canister_id, "write", ic.encode([])), - [1, 0, 0, 0], - ) - self.assertEqual( - pic.update_call(canister_id, "write", ic.encode([])), - [2, 0, 0, 0], - ) - self.assertEqual( - pic.update_call(canister_id, "read", ic.encode([])), - [2, 0, 0, 0], - ) - - -if __name__ == "__main__": - unittest.main() -``` +[View a complete example of testing a single canister on a single subnet](https://github.com/dfinity/pocketic-py/blob/main/examples/counter_canister/counter_canister_test.py). To see more examples and run them locally, clone the [PocketIC Python repo](https://github.com/dfinity/pocketic-py), then run the following command, replacing the example name with the example you'd like to run. For example, to run the `counter_canister` example, use the command: diff --git a/docs/developer-docs/smart-contracts/test/staging-environment.mdx b/docs/developer-docs/smart-contracts/test/staging-environment.mdx deleted file mode 100644 index 8cd50071c3..0000000000 --- a/docs/developer-docs/smart-contracts/test/staging-environment.mdx +++ /dev/null @@ -1,81 +0,0 @@ ---- -keywords: [intermediate, test, tutorial, staging environment, staging] ---- - - -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -import { GlossaryTooltip } from "/src/components/Tooltip/GlossaryTooltip"; - - -# Staging environment - - - -## Overview - -Having a separate deployment environment available where features can be end-to-end tested before they get deployed to the production environment can be helpful. While local deployments mirror the live mainnet as closely as possible, it is not the same. The local instance only runs as a single subnet. If you want to test canisters of two different subnets connecting to each other, you cannot do this locally. Other benefits of using a staging environment include being able to test an integration with other services, deployment workflows, and estimating costs before setting a feature live. - -## Setting up a staging environment - -In a staging environment it is possible to run any `dfx` command that would otherwise take `--network ic` but using `--network myStaging` instead. `myStaging` can be replaced with any other name, except the three reserved ones: `ic`, `local`, and `playground`. - -:::info -Deploying to the playground is ICP's equivalent of deploying to a testnet network. -::: - - -Networks are defined in two ways: assumed and explicitly configured. dfx only contains the `ic` network as an assumed environment. The flag `--network ic` can be used to run any command on the mainnet. - -All other networks are explicitly configured in `dfx.json`. The `"networks"` section should contain at least the `local` network, which gets chosen by default if no other network is specified with the `--network` flag. - -## Network definition - -To add a staging network named `myStaging` to `dfx.json`, add this code under `"networks"` in your `dfx.json`: - -``` json title="dfx.json" -"myStaging": { - "providers": [ - "https://icp0.io" - ], - "type": "persistent" -} -``` - -This value for `"providers"` tells `dfx` where to connect to the network. It is identical to the one in the hard-coded `ic` network. -The type `persistent` tells `dfx` that the canisters on this network will stay there and the canister identifiers will be saved in the `canister_ids.json` file. - -### Configuring a wallet - -If you are using a cycles wallet, the cycles wallet for each network is stored separately. The newly created `myStaging` network has no wallet configured yet. - -:::caution -Please note that the cycles wallet will be removed from dfx in a future release. - -It is recommended to use the cycles ledger instead. -::: - -To use the same cycles wallet as on the main `ic` network, first make sure the correct identity is set by running the command: - -```bash -dfx identity use -``` - -Then, read the `ic` network's currently configured wallet using: - -```bash -dfx identity get-wallet --network ic -``` - -Set the wallet for the newly defined network with the command: - -```bash -dfx identity set-wallet --network myStaging -``` - -These commands can be used together, such as: - -```bash -dfx identity set-wallet "$(dfx identity get-wallet --network ic)" --network myStaging -``` - -If you prefer to use a separate cycles wallet for the staging environment, follow the instructions in the step 'Creating a cycles wallet' in the [deploying to the mainnet guide](/docs/current/developer-docs/getting-started/deploy-and-manage). diff --git a/docs/developer-docs/smart-contracts/test/troubleshooting.mdx b/docs/developer-docs/smart-contracts/test/troubleshooting.mdx deleted file mode 100644 index b0baf73ca8..0000000000 --- a/docs/developer-docs/smart-contracts/test/troubleshooting.mdx +++ /dev/null @@ -1,162 +0,0 @@ ---- -keywords: [intermediate, test, tutorial, troubleshooting] ---- - -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; - -# Troubleshooting resources - - - -## Overview - -This section provides information to help you troubleshoot and resolve or work around common issues that are related to the following tasks: - -- Downloading and installing the IC SDK. - -- Creating, building, or deploying canisters. - -- Using the [IC SDK](/docs/current/developer-docs/getting-started/install). - -- Running the local canister execution environment. - -## Migrating an existing project - -Currently, there is no automatic migration or backward compatibility for any projects that you might have created using previous versions of the IC SDK. After upgrading to the latest version, you might see error or failure messages if you attempt to build or install a project created with a previous version of the IC SDK. - -In many cases, however, you can continue to work with projects from a previous release by manually changing the `dfx` setting in the `dfx.json` configuration file, then rebuilding the project to be compatible with the version of the IC SDK you have currently installed. - -For example, if you have a project that was created with IC SDK version `0.8.0`, open the `dfx.json` file in a text editor and change the `dfx` setting to the latest version or remove the section entirely. - -## Restarting the local canister execution environment - -In some cases, starting the local canister execution environment fails due to stale state. If you encounter issues when running `dfx start` to start the local canister execution environment: - -- #### Step 1: In the terminal that displays the ICP emulation the execution environment uses, press Control-C to interrupt the local canister execution environment process. - -- #### Step 2: Stop the local canister execution environment by running the following command: - -```bash -dfx stop -``` - -- #### Step 3: Restart the local canister execution environment in a clean state by running the following command: - -```bash -dfx start --clean -``` - -The `--clean` option removes checkpoints and stale state information from your project’s cache so that you can restart the local canister execution environment and web server processes in a clean state. - -:::caution -Keep in mind, however, that if you reset the state information by running `dfx start --clean`, your existing canisters are also removed. -::: - -- #### Step 4: After running `dfx start --clean`, recreate your canisters by running the following commands: - -```bash -dfx canister create --all -dfx build -dfx canister install --all -``` - -## Removing the canisters directory - -If you run into problems building or deploying canisters after successfully connecting to ICP and registering canister identifiers, you should remove the `canisters` directory before attempting to rebuild or redeploy the canisters. - -You can remove the `canisters` directory for a project by running the following command in the project’s root directory: - -```bash -rm -rf ./.dfx/* canisters/* -``` - -## Reinstalling dfx - -Many of the bugs you might encounter can be addressed by uninstalling and reinstalling the IC SDK. Here are a few ways to reinstall the IC SDK. - -If you only have one version of the IC SDK installed in your development environment, you can usually run the following command to uninstall and reinstall the latest version of IC SDK: - -```bash -~/.cache/dfinity/uninstall.sh && sh -ci "$(curl -sSL https://internetcomputer.org/install.sh)" -``` - -If you have modified the location of the IC SDK binary (the binary is titled `dfx`), you might want run the following command to uninstall the version of the IC SDK that is in your PATH, then reinstall the latest version of the IC SDK: - -```bash -rm -rf ~/.cache/dfinity && rm $(which dfx) && sh -ci "$(curl -sSL https://internetcomputer.org/install.sh)" -``` - -## Xcode prerequisite - -Some versions of the IC SDK prompted you to install Xcode when creating a new project on a macOS computer. The prompt has been removed and the `dfx new` command does not require you to install any macOS developer tools. However, you should have **Developer Command Line Tools** installed if you want to create a Git repository for your project. - -You can check whether you have the developer tools installed by running `xcode-select -p`. You can install the developer tools by running `xcode-select --install`. - -## Apple ARM silicon -If you are using a Mac with Apple silicon and are having issues (such as `bad CPU type in executable: dfx`), you may need to install Rosetta. - -```bash -softwareupdate --install-rosetta -``` - -## Failed build when using VMs - -If you are running the IC SDK using a virtual machine image on Ubuntu or CentOS, you might see an error message that looks like this when you attempt to run the `dfx build` command: - -```bash -Building hello... -An error occurred: -Io( - Os { - code: 2, - kind: NotFound, - message: "No such file or directory", - }, -) -``` - -## Address in use error or orphan processes - -If you are developing projects locally, you often have the local replica running either in a separate terminal or in the background. If the local environment processes do not get properly terminated, you might see operating system errors indicating that an address is already in use or or be unable to stop processes normally using the `dfx stop` command. - -There are several scenarios in which you might encounter this issue. For example, if you run `dfx start` in a local project directory then change to a different local project directory without first stopping the canister execution environment processes, you might see this issue. - -If you encounter an issue where you suspect or you receive a message that an address is already in use or that a process is already running in the background, perform the following steps: - -- #### Step 1: Run the following command to see which process is listening to the 4943 port if you are using the default binding to localhost: - -```bash -lsof -i tcp:4943 -``` - -- #### Step 2: Run the following command to terminate any orphan processes: - -```bash -killall dfx replica -``` - -- #### Step 3: Close the current terminal and open a new terminal window. - -- #### Step 4: In the new terminal, run the following command to run the local canister execution environment in a clean state: - -```bash -dfx start --clean --background -``` - -## Memory leak - -Fixing memory leaks is an ongoing process. If you encounter any error messages related to memory leaks, you should do the following: - -- #### Step 1: Run `dfx stop` to stop currently running processes. - -- #### Step 2: Uninstall the IC SDK to prevent further degradation. - -- #### Step 3: Re-install the IC SDK - -- #### Step 4: Run `dfx start` to restart replica processes. - -Alternatively, you can remove the `.cache/dfinity` directory and re-install the latest IC SDK `dfx` binary. For example: - -```bash -rm -rf ~/.cache/dfinity && sh -ci "$(curl -sSL https://internetcomputer.org/install.sh)" -``` diff --git a/docs/developer-docs/smart-contracts/topping-up/cycles_management_services.mdx b/docs/developer-docs/smart-contracts/topping-up/cycles_management_services.mdx index 3a4d7eb73a..378ac9766b 100644 --- a/docs/developer-docs/smart-contracts/topping-up/cycles_management_services.mdx +++ b/docs/developer-docs/smart-contracts/topping-up/cycles_management_services.mdx @@ -33,144 +33,4 @@ Popular cycles management services include: ## Creating custom autonomous top-up solutions -To create an autonomous canister that doesn't rely on a third-party service, you can create your own autonomous canister top-up solution. One option is to use a similar architecture implemented by [CycleOps](https://github.com/CycleOperators/cycles-manager/tree/main), where one canister acts as the cycles manager for the project and provides the API used to request more cycles when a canister within the project has its cycles balance dip below a preset threshold. Below is a sample code for this architecture, where 'CyclesManager' is the management canister and 'Child' is another canister in the project that requests cycles from the 'CyclesManager'. - -:::caution - -You will still need to top up the CyclesManager canister manually. - -::: - -Here is example code for the 'CyclesManager' canister: - -```motoko no-repl title="CyclesManager.mo" -// Import packages -import { logand } "mo:base/Bool"; -import { trap } "mo:base/Debug"; -import Principal "mo:base/Principal"; -import { endsWith; size } "mo:base/Text"; - -// Import the Mops package CyclesManager -import CyclesManager "mo:cycles-manager/CyclesManager"; - -// Implements the cycles_manager_transferCycles API of the CyclesManager.Interface - -actor CyclesManager { -  // Initializes a cycles manager -  stable let cyclesManager = CyclesManager.init({ -    // By default, with each transfer request 500 billion cycles will be transferred -    // to the requesting canister, provided they are permitted to request cycles -    // -    // This means that if a canister is added with no quota, it will default to the quota of #fixedAmount(500) -    defaultCyclesSettings = { -      quota = #fixedAmount(500_000_000_000); -    }; -    // Allow an aggregate of 10 trillion cycles to be transferred every 24 hours -    aggregateSettings = { -      quota = #rate({ -        maxAmount = 10_000_000_000_000; -        durationInSeconds = 24 * 60 * 60; -      }); -    }; -    // 50 billion is a good default minimum for most low use canisters -    minCyclesPerTopup = ?50_000_000_000; -  }); - -  // @required - IMPORTANT!!! -  // Allows canisters to request cycles from this "manager canister" that implements -  // the cycles manager -  public shared ({ caller }) func cycles_manager_transferCycles( -    cyclesRequested: Nat -  ): async CyclesManager.TransferCyclesResult { -    if (not isCanister(caller)) trap("Calling principal must be a canister"); - -    let result = await* CyclesManager.transferCycles({ -      cyclesManager; -      canister = caller; -      cyclesRequested; -    }); -    result; -  }; - -  // A very basic example of adding a canister to the cycles manager -  // This adds a canister with a 1 trillion cycles allowed per 24 hours cycles quota -  // -  // IMPORTANT: You must add authorization for production implementations so that not just any canister can add themselves -  public shared func addCanisterWith1TrillionPer24HoursLimit(canisterId: Principal) { -    CyclesManager.addChildCanister(cyclesManager, canisterId, { -      // This top up rule 1 Trillion every 24 hours -      quota = ?(#rate({ -        maxAmount = 1_000_000_000_000; -        durationInSeconds = 24 * 60 * 60; -      })); -    }) -  }; - -  // **DO NOT USE IN PRODUCTION** - for developer debugging and testing purposes only -  public func toText() : async Text { -    let result = CyclesManager.toText(cyclesManager); -    result; -  }; - -  func isCanister(p : Principal) : Bool { -    let principal_text = Principal.toText(p); -    // Canister principals have 27 characters -    size(principal_text) == 27 -    and -    // Canister principals end with "-cai" -    endsWith(principal_text, #text "-cai"); -  }; -} -``` - -Here is example code for the 'Child' canister: - -```motoko no-repl Child.mo - -// Import the Mops package CyclesManager -import CyclesRequester "mo:cycles-manager/CyclesRequester"; -import CyclesManager "mo:cycles-manager/CyclesManager"; - -import { print } "mo:base/Debug"; - -actor Child { -  // Stable variable holding the cycles requester -  stable var cyclesRequester: ?CyclesRequester.CyclesRequester = null; - -  // A simple counter, for the purposes of this example -  stable var counter: Nat = 0; - -  // Initialize the cycles requester -  // As an alternative, you can also initialize the cycles requester in the constructor -  public func initializeCyclesRequester( -    batteryCanisterPrincipal: Principal, -    topupRule: CyclesRequester.TopupRule, -  ) { -    cyclesRequester := ?CyclesRequester.init({ -      batteryCanisterPrincipal; -      topupRule -    }); -  }; - -  // An example of adding cycles request functionality to an arbitrary update function -  public func justAnotherCounterExample(): async () { -    // before doing something, check if we need to request cycles -    let result = await* requestTopupIfLow(); -    print(debug_show(result)); - -    // Do something in the rest of the function; -    counter += 1; -  }; - -  // Local helper function you can use in your actor if the cyclesRequester could possibly be null -  func requestTopupIfLow(): async* CyclesManager.TransferCyclesResult { -    switch(cyclesRequester) { -      case null #err(#other("CyclesRequester not initialized")); -      case (?requester) await* CyclesRequester.requestTopupIfBelowThreshold(requester); -    } -  }; -} -``` - -[View the full example on GitHub](https://github.com/CycleOperators/cycles-manager/tree/main/example). - +To create an autonomous canister that doesn't rely on a third-party service, you can create your own autonomous canister top-up solution. One option is to use a similar architecture implemented by [CycleOps](https://github.com/CycleOperators/cycles-manager/tree/main), where one canister acts as the cycles manager for the project and provides the API used to request more cycles when a canister within the project has its cycles balance dip below a preset threshold. Below is a sample code for this architecture, where 'CyclesManager' is the management canister and 'Child' is another canister in the project that requests cycles from the 'CyclesManager'. \ No newline at end of file diff --git a/docs/developer-docs/smart-contracts/topping-up/topping-up-canister.mdx b/docs/developer-docs/smart-contracts/topping-up/topping-up-canister.mdx index bb2f56208d..7cc81e5220 100644 --- a/docs/developer-docs/smart-contracts/topping-up/topping-up-canister.mdx +++ b/docs/developer-docs/smart-contracts/topping-up/topping-up-canister.mdx @@ -17,9 +17,9 @@ A canister needs to have cycles deposited con Anyone can top up any canister deployed to ICP; you do not need to be the owner or controller of the canister. All you need is the canister's principal. -There are a few different ways to top up canisters, such as: +There are a few different ways to top up canisters: -- Using [`dfx`](/docs/current/developer-docs/developer-tools/cli-tools/cli-reference/). +- Using [`dfx cycles top-up`](/docs/current/developer-docs/developer-tools/cli-tools/cli-reference/dfx-cycles#dfx-cycles-top-up). - Through the [NNS frontend dapp](https://nns.ic0.app). @@ -47,13 +47,14 @@ The following examples will demonstrate how to interact with and top up a canist ### Checking the cycles balance of a canister -Canister cycles balances are not exposed publicly by default; you can only see them if you are the controller of a canister. Using `jqylk-byaaa-aaaal-qbymq-cai` as an example, you can query it by calling: +Canister cycles balances are available through the [`dfx canister status`](/docs/current/developer-docs/developer-tools/cli-tools/cli-reference/dfx-canister#dfx-canister-status) command. They are not exposed publicly by default; you can only see them if you are the controller of a canister: ```bash -dfx canister status jqylk-byaaa-aaaal-qbymq-cai --network ic +dfx canister status jqylk-byaaa-aaaal-qbymq-cai --network=ic ``` -The output will look like this: +
+Output ```bash Canister status call result for jqylk-byaaa-aaaal-qbymq-cai. @@ -67,91 +68,46 @@ Balance: 9_811_813_913_485 Cycles Module hash: 0xe7866e1949e3688a78d8d29bd63e1c13cd6bfb8fbe29444fa606a20e0b1e33f0 ``` -Canisters can check their balance programmatically. For example: +
+ +Canisters can check their balance programmatically. -Motoko canisters can use the [ExperimentalCycles library](/docs/current/motoko/main/base/ExperimentalCycles) to interact with cycles: - -``` motoko no-repl -import Cycles "mo:base/ExperimentalCycles"; -import Debug "mo:base/Debug"; - -actor { - public func main() : async() { - Debug.print("Main balance: " # debug_show(Cycles.balance())); - }; -} -``` +Motoko canisters can use the [ExperimentalCycles library](/docs/current/motoko/main/base/ExperimentalCycles) to interact with cycles. -Rust canisters can use the function `canister_balance128()` from the [Rust IC CDK](https://docs.rs/ic-cdk/latest/ic_cdk/api/fn.canister_balance128.html): - -```rust -let current_canister_balance = ic_cdk::api::canister_balance(); -``` +Rust canisters can use the function `canister_balance128()` from the [Rust IC CDK](https://docs.rs/ic-cdk/latest/ic_cdk/api/fn.canister_balance128.html). ### Option 1: If you have ICP on your account -If you have ICP on the account associated with a `dfx` identity, you can convert ICP into cycles, then top up a canister with those cycles: - -```bash -dfx ledger account-id -dfx ledger balance --network ic -dfx ledger top-up --icp 1 jqylk-byaaa-aaaal-qbymq-cai --network ic -``` - --   The `dfx ledger account-id` returns the ledger account ID of the current `dfx` identity used. --   The `dfx ledger --network ic balance` command checks the balance on the `account` associated with the current `dfx` identity used. --   `top-up --icp 1 jqylk-byaaa-aaaal-qbymq-cai` command converts 1 ICP into cycles and uses them to refill canister `jqylk-byaaa-aaaal-qbymq-cai`. --   `--network ic` tells `dfx` to use the mainnet as the network (rather than the local replica). +If you have ICP associated with your [developer identity](/docs/current/developer-docs/getting-started/identities) you can use [`dfx ledger top-up`](/docs/current/developer-docs/developer-tools/cli-tools/cli-reference/dfx-ledger) to convert some ICP into cycles, then top up a canister with those cycles: ### Option 2: If you have cycles -If you already have cycles, you can top up a canister with the command `dfx cycles top-up `: +If you already have cycles, you can top up a canister with the command [`dfx cycles top-up`](/docs/current/developer-docs/developer-tools/cli-tools/cli-reference/dfx-cycles#dfx-cycles-top-up): ```bash -dfx cycles balance --network ic dfx cycles top-up jqylk-byaaa-aaaal-qbymq-cai 1T --network ic ``` -:::warning -This workflow uses the cycles ledger, a feature that requires `dfx` version `0.19.0`. To enable this feature, you will need to set the following environmental variable: - -```bash -DFX_CYCLES_LEDGER_SUPPORT_ENABLE=1 -``` -::: - --   The `cycles balance --network ic` command checks the cycles balance of your identity on the mainnet. --   The `cycles top-up` command transfers cycles to the canister of your choice. - ### Special case: topping up a cycles wallet -:::info -Please note that the cycles wallet will be removed from `dfx` in a future release. +:::danger +The cycles wallet will be removed from `dfx` in a future release. It is recommended to use the cycles ledger instead. ::: -Cycles wallets are canisters like any other, so you can top them up as well. If you have a cycles wallet you control via dfx, there is another option as well for sending cycles from your cycles wallet to a canister of your choice: `dfx wallet send [OPTIONS] `. +Cycles wallets are canisters like any other. If you have a cycles wallet you control via `dfx`, there is another option for sending cycles from your cycles wallet to a canister of your choice: [`dfx wallet send [OPTIONS] `](/docs/current/developer-docs/developer-tools/cli-tools/cli-reference/dfx-wallet#dfx-wallet-send). -In this example, it is assumed you are sending cycles to a cycles wallet with the principal ID `dfds-sddds-aaaal-qbsms-cai`. - -```bash -dfx wallet balance --network ic -dfx wallet send 1T dfds-sddds-aaaal-qbsms-cai --network ic -``` - --   The `wallet balance --network ic` checks the cycles balance of your cycles wallet on the mainnet. --   The `wallet send 1T dfds-sddds-aaaal-qbsms-cai --network ic` takes one trillion cycles from your cycles wallet and sends them to cycles wallet `dfds-sddds-aaaal-qbsms-cai`. ## Topping up a canister with the NNS frontend dapp @@ -167,26 +123,4 @@ You can also top up any canister via the [NNS frontend dapp](https://nns.ic0.app ICP features a mechanism to prevent canisters from running out of cycles. Canisters have a configurable `freezing_threshold`, dynamically evaluated in cycles. -In the output of the `dfx canister status` command, you may have noticed the `freezing threshold` value. The freezing threshold is a value defined in seconds, which is used to calculate how many cycles a canister must retain in its cycles balance. This calculation is based on the canister's memory consumption. The default freezing threshold value is `2_592_000s`, which corresponds to 30 days. - -The freezing threshold is important because if a canister runs out of cycles, it will be uninstalled. The freezing threshold protects it from deletion since if the cycles balance dips below the threshold, the canister will stop processing any new requests; however, it will continue to reply to existing requests. - -For example, to set a freezing threshold for your `poll_backend` canister, use the command: - -``` -dfx canister update-settings poll_backend --freezing-threshold 3472000 -``` - -Then, confirm that this threshold has been set by running the `dfx canister status poll_backend --network ic ` command again: - -```bash -Canister status call result for poll_backend. -Status: Stopped -Controllers: lalyb-uhvmt-p7ubs-u5t7l-hce6v-lp7c5-dlmj5-wi2gc-depab-wtgi3-pae -Memory allocation: 0 -Compute allocation: 0 -Freezing threshold: 3_472_000 -Memory Size: Nat(2363181) -Balance: 3_100_000_000_000 Cycles -Module hash: 0xf8680eb74022a1079012b7e9c644d1156580002a6126305791811533d3fd6f17 -``` +[Learn more about freezing thresholds](/docs/current/developer-docs/smart-contracts/maintain/settings#freezing-threshold). diff --git a/docs/other/updates/release-notes/sdk-release-notes.md b/docs/other/updates/release-notes/sdk-release-notes.md index b1044e7e2b..bd12238016 100644 --- a/docs/other/updates/release-notes/sdk-release-notes.md +++ b/docs/other/updates/release-notes/sdk-release-notes.md @@ -26,7 +26,7 @@ To see a list of known problems with `dfx` or to report a problem you discovered Joining the [DFINITY Developer Forum](https://forum.dfinity.org/) is an effective way to learn from community members, ask questions, solicit help from other developers, and provide insight and feedback about your experiences. -If you have questions that aren’t answered by the community, you might also want to check out [Troubleshooting](/docs/current/developer-docs/smart-contracts/test/troubleshooting) topics for information about common issues, workarounds for known issues, or help troubleshooting warnings or errors. +If you have questions that aren’t answered by the community, you might also want to check out [Troubleshooting](/docs/current/developer-docs/getting-started/troubleshooting) topics for information about common issues, workarounds for known issues, or help troubleshooting warnings or errors. For technical support, send email to [DFINITY Support](mailto:support@dfinity.org). diff --git a/docusaurus.config.js b/docusaurus.config.js index c7cd65939e..3dc633f7ff 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -106,13 +106,11 @@ const subnavItems = [ label: "Awesome Internet Computer", href: "https://github.com/dfinity/awesome-internet-computer#readme", }, - { label: "Sample Code", to: "/samples" }, { label: "SDK Release Notes", type: "doc", docId: "other/updates/release-notes/release-notes", }, - { label: "Developer Tools", to: "/tooling" }, { label: "Developer Grants", href: "https://dfinity.org/grants" }, { label: "Playground", diff --git a/plugins/utils/redirects.js b/plugins/utils/redirects.js index fd63a56ab7..c9b8f85822 100644 --- a/plugins/utils/redirects.js +++ b/plugins/utils/redirects.js @@ -84,7 +84,7 @@ const redirects = ` /docs/current/developer-docs/build/project-setup/cycles-wallet /docs/current/developer-docs/defi/cycles/cycles-wallet /docs/current/developer-docs/build/project-setup/manage-canisters /docs/current/developer-docs/smart-contracts/maintain/settings /docs/current/developer-docs/build/project-setup/design-dapps /docs/current/developer-docs/web-apps/design-dapps - /docs/current/developer-docs/build/troubleshooting /docs/current/developer-docs/smart-contracts/test/troubleshooting + /docs/current/developer-docs/build/troubleshooting /docs/current/developer-docs/getting-started/troubleshooting /docs/current/developer-docs/build/agents/ /docs/current/developer-docs/developer-tools/off-chain/agents/overview /docs/current/developer-docs/build/agents/javascript/javascript-intro /docs/current/developer-docs/developer-tools/off-chain/agents/javascript-agent /docs/current/developer-docs/build/agents/javascript/* /docs/current/developer-docs/developer-tools/off-chain/agents/javascript-agent @@ -107,7 +107,7 @@ const redirects = ` /docs/developers-guide/reinstalling-dfx /docs/current/developer-docs/getting-started/install /docs/developers-guide/sample-apps /samples /docs/developers-guide/sdk-guide /docs/current/developer-docs/getting-started/install - /docs/developers-guide/troubleshooting /docs/current/developer-docs/smart-contracts/test/troubleshooting + /docs/developers-guide/troubleshooting /docs/current/developer-docs/getting-started/troubleshooting /docs/developers-guide/tutorials-intro /docs/current/motoko/main/getting-started/motoko-introduction /docs/developers-guide/tutorials/default-frontend /docs/current/developer-docs/web-apps/application-frontends/default-frontend /docs/developers-guide/tutorials/my-contacts /docs/current/developer-docs/web-apps/application-frontends/add-stylesheet @@ -327,7 +327,7 @@ const redirects = ` /docs/current/developer-docs/setup/pocket-ic /docs/current/developer-docs/smart-contracts/test/pocket-ic /docs/current/developer-docs/production/staging-environment /docs/current/developer-docs/smart-contracts/deploy/custom-testnets /docs/current/developer-docs/backend/reproducible-builds /docs/current/developer-docs/smart-contracts/best-practices/reproducible-builds - /docs/current/developer-docs/backend/troubleshooting /docs/current/developer-docs/smart-contracts/test/troubleshooting + /docs/current/developer-docs/backend/troubleshooting /docs/current/developer-docs/getting-started/troubleshooting /docs/current/developer-docs/integrations/t-ecdsa /docs/current/developer-docs/smart-contracts/signatures/t-ecdsa /docs/current/developer-docs/integrations/vetkeys/ /docs/current/developer-docs/smart-contracts/encryption/vetkeys /docs/current/developer-docs/integrations/vetkeys/using-vetkeys /docs/current/developer-docs/smart-contracts/encryption/using-vetkeys @@ -639,6 +639,9 @@ const redirects = ` /docs/current/developer-docs/getting-started/deploy/local /docs/current/developer-docs/getting-started/deploy-and-manage /docs/current/developer-docs/getting-started/deploy/testnet /docs/current/developer-docs/getting-started/deploy-and-manage /docs/current/developer-docs/getting-started/deploy-and-manage /docs/current/developer-docs/getting-started/deploy-and-manage + /docs/current/developer-docs/smart-contracts/test/troubleshooting /docs/current/developer-docs/getting-started/troubleshooting + /docs/current/developer-docs/smart-contracts/advanced-features/http-gateways /docs/current/developer-docs/smart-contracts/advanced-features/handling-get-post-requests + /docs/current/developer-docs/smart-contracts/advanced-features/serving-http-request /docs/current/developer-docs/smart-contracts/advanced-features/handling-get-post-requests /docs/current/developer-docs/smart-contracts/write/default-template /docs/current/developer-docs/smart-contracts/write/overview /developers /docs/current/home ` diff --git a/sidebars.js b/sidebars.js index 651b7ea0aa..a2efbe15a6 100644 --- a/sidebars.js +++ b/sidebars.js @@ -183,9 +183,10 @@ const sidebars = { id: "developer-docs/smart-contracts/test/overview", }, "developer-docs/smart-contracts/test/pocket-ic", - "developer-docs/smart-contracts/test/troubleshooting", ], }, + ], + }, { type: "category", label: "Advanced features", @@ -200,27 +201,11 @@ const sidebars = { label: "Composite queries", id: "developer-docs/smart-contracts/advanced-features/composite-query", }, - { - type: "category", - label: "Incoming HTTP requests", - items: [ { type: "doc", - label: "GET/POST requests", + label: "HTTP gateways & incoming requests", id: "developer-docs/smart-contracts/advanced-features/handling-get-post-requests", }, - { - type: "doc", - label: "Serving requests", - id: "developer-docs/smart-contracts/advanced-features/serving-http-request", - }, - { - type: "doc", - label: "HTTP gateways", - id: "developer-docs/smart-contracts/advanced-features/http-gateways", - }, - ], - }, { type: "category", label: "HTTPS outcalls", @@ -328,8 +313,6 @@ const sidebars = { }, ], }, - ], - }, { type: "category", label: "Developer tools",