rest_api: add an example to the incremental load section (#1502)

docs/website/docs/dlt-ecosystem/verified-sources/rest_api.md

@@ -560,49 +560,110 @@ This will include the `id`, `title`, and `created_at` fields from the `issues` r
 Some APIs provide a way to fetch only new or changed data (most often by using a timestamp field like `updated_at`, `created_at`, or incremental IDs).
 This is called [incremental loading](../../general-usage/incremental-loading.md) and is very useful as it allows you to reduce the load time and the amount of data transferred.
 
-When the API endpoint supports incremental loading, you can configure the source to load only the new or changed data using these two methods:
+When the API endpoint supports incremental loading, you can configure dlt to load only the new or changed data using these two methods:
 
-1. Defining a special parameter in the `params` section of the [endpoint configuration](#endpoint-configuration):
+1. Defining a special parameter in the `params` section of the [endpoint configuration](#endpoint-configuration).
+2. Specifying the `incremental` field in the endpoint configuration.
 
-    ```py
-    {
-        "<parameter_name>": {
-            "type": "incremental",
-            "cursor_path": "<path_to_cursor_field>",
-            "initial_value": "<initial_value>",
-        },
-    }
-    ```
+Let's start with the first method.
 
-    For example, in the `issues` resource configuration in the GitHub example, we have:
+### Incremental loading in `params`
 
-    ```py
-    {
-        "since": {
+Imagine we have the following endpoint `https://api.example.com/posts` and it:
+1. Accepts a `created_since` query parameter to fetch posts created after a certain date.
+2. Returns a list of posts with the `created_at` field for each post.
+
+For example, if we query the endpoint with `https://api.example.com/posts?created_since=2024-01-25`, we get the following response:
+
+```json
+{
+    "results": [
+        {"id": 1, "title": "Post 1", "created_at": "2024-01-26"},
+        {"id": 2, "title": "Post 2", "created_at": "2024-01-27"},
+        {"id": 3, "title": "Post 3", "created_at": "2024-01-28"}
+    ]
+}
+```
+
+To enable the incremental loading for this endpoint, you can use the following endpoint configuration:
+
+```py
+{
+    "path": "posts",
+    "data_selector": "results",  # Optional JSONPath to select the list of posts
+    "params": {
+        "created_since": {
             "type": "incremental",
-            "cursor_path": "updated_at",
-            "initial_value": "2024-01-25T11:21:28Z",
+            "cursor_path": "created_at", # The JSONPath to the field we want to track in each post
+            "initial_value": "2024-01-25",
         },
-    }
-    ```
+    },
+}
+```
 
-    This configuration tells the source to create an incremental object that will keep track of the `updated_at` field in the response and use it as a value for the `since` parameter in subsequent requests.
+After you run the pipeline, dlt will keep track of the last `created_at` from all the posts fetched and use it as the `created_since` parameter in the next request.
+So in our case, the next request will be made to `https://api.example.com/posts?created_since=2024-01-28` to fetch only the new posts created after `2024-01-28`.
 
-2. Specifying the `incremental` field in the [endpoint configuration](#endpoint-configuration):
+Let's break down the configuration.
 
-    ```py
-    {
-        "incremental": {
-            "start_param": "<parameter_name>",
-            "end_param": "<parameter_name>",
-            "cursor_path": "<path_to_cursor_field>",
-            "initial_value": "<initial_value>",
-            "end_value": "<end_value>",
-        }
+1. We explicitly set `data_selector` to `"results"` to select the list of posts from the response. This is optional, if not set, dlt will try to auto-detect the data location.
+2. We define the `created_since` parameter as an incremental parameter with the following fields:
+
+```py
+{
+    "created_since": {
+        "type": "incremental",
+        "cursor_path": "created_at",
+        "initial_value": "2024-01-25",
+    },
+}
+```
+
+- `type`: The type of the parameter definition. In this case, it must be set to `incremental`.
+- `cursor_path`: The JSONPath to the field within each item in the list. The value of this field will be used in the next request. In the example above our items look like `{"id": 1, "title": "Post 1", "created_at": "2024-01-26"}` so to track the created time we set `cursor_path` to `"created_at"`. Note that the JSONPath starts from the root of the item (dict) and not from the root of the response.
+- `initial_value`: The initial value for the cursor. This is the value that will initialize the state of incremental loading. In this case, it's `2024-01-25`. The value type should match the type of the field in the data item.
+
+### Incremental loading using the `incremental` field
+
+The alternative method is to use the `incremental` field in the [endpoint configuration](#endpoint-configuration). This method is more flexible and allows you to specify the start and end conditions for the incremental loading.
+
+Let's take the same example as above and configure it using the `incremental` field:
+
+```py
+{
+    "path": "posts",
+    "data_selector": "results",
+    "incremental": {
+        "start_param": "created_since",
+        "cursor_path": "created_at",
+        "initial_value": "2024-01-25",
+    },
+}
+```
+
+Note that we specify the query parameter name `created_since` in the `start_param` field and not in the `params` section.
+
+The full available configuration for the `incremental` field is:
+
+```py
+{
+    "incremental": {
+        "start_param": "<start_parameter_name>",
+        "end_param": "<end_parameter_name>",
+        "cursor_path": "<path_to_cursor_field>",
+        "initial_value": "<initial_value>",
+        "end_value": "<end_value>",
     }
-    ```
+}
+```
+
+The fields are:
 
-    This configuration is more flexible and allows you to specify the start and end conditions for the incremental loading.
+- `start_param` (str): The name of the query parameter to be used as the start condition. If we use the example above, it would be `"created_since"`.
+- `end_param` (str): The name of the query parameter to be used as the end condition. This is optional and can be omitted if you only need to track the start condition. This is useful when you need to fetch data within a specific range and the API supports end conditions (like `created_before` query parameter).
+- `cursor_path` (str): The JSONPath to the field within each item in the list. This is the field that will be used to track the incremental loading. In the example above, it's `"created_at"`.
+- `initial_value` (str): The initial value for the cursor. This is the value that will initialize the state of incremental loading.
+- `end_value` (str): The end value for the cursor to stop the incremental loading. This is optional and can be omitted if you only need to track the start condition. If you set this field, `initial_value` needs to be set as well.
 
 See the [incremental loading](../../general-usage/incremental-loading.md#incremental-loading-with-a-cursor-field) guide for more details.