Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IBX-8963: AI Actions #315

Merged
merged 11 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/ai_actions/ai_actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
description: Learn about generative AI and its capabilities.
---

# AI Actions

AI Actions is a set of capabilities that enhance the efficiency and creativity of content editors.
The feature is not included in [[= product_name =]] by default.
It's an [LTS Update](https://www.ibexa.co/blog/continuous-innovation-how-ibexa-dxp-s-lifecycle-enhances-feature-delivery) that requires separate [installation and configuration]([[= developer_doc =]]/ai_actions/install_ai_actions/) by the system administrator.
dabrt marked this conversation as resolved.
Show resolved Hide resolved

Once AI Actions are installed and configured, content editors with `Action configuration/View` and `Action configuration/Execute` permissions have access to AI-driven tools.
For example, they can [enhance or improve the text](create_edit_content_items.md#ai-assistant) that they have written, or [generate alternative text](upload_images.md#ai) for images.

If you have administrator permissions, you can decide what capabilities are available to the editors by [configuring AI actions](work_with_ai_actions.md).
You can also modify the parameters of individual AI actions, to control the creativity of the AI or the cost involved with using it.
Binary file added docs/ai_actions/img/ai_action_details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/ai_actions/img/ai_action_options.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/ai_actions/img/ai_actions_list.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/ai_actions/img/ai_assistant.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
100 changes: 100 additions & 0 deletions docs/ai_actions/work_with_ai_actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
description: Create new AI actions or modify existing ones to work faster and increase creativity.
---

# Work with AI actions

AI actions define what results are available to editors in AI-enabled areas, such as, for example, the AI Assistant.
If the AI Actions LTS update has been installed and configured in your application, and you have administrator permissions, you can reconfigure the existing AI actions, and create new ones.
dabrt marked this conversation as resolved.
Show resolved Hide resolved

## View AI actions

With administrator permissions, you can view all AI actions configured in the application by navigating to the Admin Panel and selecting **AI actions**.
dabrt marked this conversation as resolved.
Show resolved Hide resolved

![AI actions in Admin Panel](img/ai_actions_list.png)

You can narrow down the list of AI actions by filtering it by the status, either Enabled or Disabled, or by the type.
Out of the box, there are two types of AI Actions present in the system:

- **Refine text** - used primarily in [online editor](create_edit_content_items.md#ai-assistant) for performing text-to-text actions, for example: "Rewrite text in formal tone"
dabrt marked this conversation as resolved.
Show resolved Hide resolved
- **Generate alternative text** - used in the [image asset editing screen](upload_images.md#ai) to perform image-to-text actions, for example: "Generate short alternative description of an image"
dabrt marked this conversation as resolved.
Show resolved Hide resolved

It may happen that a set of sample AI actions has been [installed with the AI Actions package]([[= developer_doc =]]/ai_actions/install_ai_actions/#install-sample-ai-action-configurations-optional), and there is a number of existing AI actions that you can modify and clone.

!!! note "Custom action types"

In your specific case, the types available can be different, and your organization's development team can create custom AI action types.
For more information, see [developer documentation]([[= developer_doc =]]/ai_actions/).

### View AI action details

Navigate to the Admin Panel and select **AI actions**.
In the **AI actions** list, click the name of an AI action to review its details.
For example, in the **Properties** tab, you can see specific settings that modify the prompt that is sent to an AI service.

![AI action details](img/ai_action_details.png)

## Edit existing AI actions

You can modify the existing AI actions.

1\. Navigate to the Admin Panel and select **AI actions**.

2\. In the **AI actions** list, click the **Edit** icon next to a name of the AI action that you want to modify.

3\. In the **Global properties** section, you can change the name and description of the AI action. You can also toggle the availability of the AI action between disabled and enabled.

4\. In the **Settings** area, change the settings that modify the behavior of an AI service that executes an AI action, for example:

- **Prompt** - modifies the default request by passing a verbal command, for example, "Make it short and formal."

!!! note "Default request"

The default request can be seen at the top of the settings area, on a light blue background.

- **Max tokens** - sets a maximum number of "[words](https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them)" or tokens that can be used in a single request by both the request and the response

- **Length of prompt output** - sets a maximum number of words or tokens that can be used by the response
dabrt marked this conversation as resolved.
Show resolved Hide resolved

- **Temperature** - controls the randomness of the response. The higher the temperature, the more creative is the output

![AI action options](img/ai_action_options.png)

!!! note "Action settings availability"

Action settings differ depending on the AI service used, model implementation and actual action type.
Therefore the settings visible in your installation may vary from the ones presented above.

5\. Click **Save and close** to apply the changes or **Discard** to discard them and close the window.

## Create new AI actions

You can create AI actions that perform actions of different types, using different models, or action handlers.

!!! note "AI action models"

Before you can work with AI actions, models must be configured and enabled by your organization's development team.
If there are more AI service connectors available, you might be able to create AI actions that perform the same type of actions but use different models.
For more information, see [developer documentation]([[= developer_doc =]]/ai_actions/ai_actions_guide/#model).

1. Navigate to the Admin Panel and select **AI actions**.

1. In the **AI actions** list, click **Create**.

1. In the slide-out pane, make initial choices in the following fields, and click **Create**:

- **Language** - sets the base language for the AI action
- **Action type** - sets an action type to serve as a template for the AI action, for example, **Refine action**
- **Model** - sets the AI model used to process the requests resulting from this AI action

1. In the **Global properties** section, set the name and identifier of the AI action.

1. Optionally, provide a description of the AI action.

1. When ready, toggle the status of the AI action to enabled.

1. In the **Settings** area.
For a list of available settings, see [Edit existing AI actions](#edit-existing-ai-actions).

1. Click **Save and close** to apply the changes or **Discard** to discard them and close the window.

7 changes: 7 additions & 0 deletions docs/content_management/block_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,13 @@ On the **Properties** tab, set values in the following fields:
- **Content** — Enter text, images, videos, etc. using Online Editor.
For details, see [Editing Rich Text Fields](create_edit_content_items.md#edit-rich-text-fields).

!!! note "Use AI to improve text"

If the AI Actions LTS update has been installed and configured in your application, the **Improve text** button is visible.
If you have necessary permissions, you can click it to request that the AI Assistant performs specific tasks with your text.

For more information, see [AI Assistant](create_edit_content_items.md#ai-assistant).

## Video block

Integrates a video into the Page with standard playback controls.
Expand Down
28 changes: 28 additions & 0 deletions docs/content_management/create_edit_content_items.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ You can also add new elements to the Field. To do it, choose one of the availabl
Each of these elements can have its own settings, such as text formatting.
The option bar also lets you reorder or remove any elements in the Rich Text Field.



#### Edit embedded content items

You can edit embedded content items without leaving current window.
Expand Down Expand Up @@ -184,6 +186,32 @@ add or delete rows and columns in any part of the table, as well as merge and sp
Your installation can also have custom elements available in the Rich Text field.
Contact your website administrator about the details of using them.

### AI Assistant

If the [AI Actions](ai_actions.md) LTS update has been installed and configured in your application, when writing content, the **Improve text** button is visible in the toolbox.
If you have `Action configuration/View` and `Action configuration/Execute` permissions, you can request that the AI Assistant performs specific tasks with your text.
dabrt marked this conversation as resolved.
Show resolved Hide resolved
dabrt marked this conversation as resolved.
Show resolved Hide resolved
For example, you can highlight a passage and instruct the AI Assistant to extend the passage or adjust its tone to suit your needs.

The **Go to AI actions** button can also visible to users with access to the **Admin UI**.
By clicking it, you can quickly navigate to [AI actions configuration](work_with_ai_actions.md).

![Improve text button and AI actions](ai_assistant.png)

After you select an action, the AI Assistant modal appears, where you can observe the results of AI service's work.
When response is displayed on the screen, depending on whether you are satisfied with the result, you can select one of the options:

- **Replace** - replaces the source text with the output
- **Insert below** - inserts the output text directly after the source passage
- **Try again** - makes another attempt at generating the output

![AI Assistant](ai_assistant_options.png)

!!! note "Lengthy processing of complex requests"

Depending on the length of the source text and the complexity of request, processing may take a while.
You may cancel the processing if it takes too long by clicking **Stop**.


### SeenThis! streaming

!!! note
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
40 changes: 40 additions & 0 deletions docs/getting_started/content_tree/content_tree.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
description: Learn about the Content Tree.
---

# Content Tree

Content Tree is the place where you can navigate through all the content items in your application.
If you want to browse your application's content items, in the main menu, go to the **Content** -> **Content structure**.
The **Content Tree** area is an expandable content menu of your website.

![Content Tree in the menu](../img/left_menu_tree.png "Content Tree in the menu")

Unique icons for each content type instantly show you what type of content you are selecting. To add custom icons to your Content Tree, follow [configuration tutorial in developer documentation]([[= developer_doc =]]/administration/back_office/back_office_elements/custom_icons/#customize-content-type-icons).

For more information about custom configuration, go to [Content Tree]([[= developer_doc =]]/administration/back_office/content_tree/) in developer documentation.

Win the Content Tree, hidden content is greyed out in the tree view.

To simplify the Content Tree, big lists are collapsed and include a **Show more** icon.
Click it to expand the branches of the tree.
**Collapse all** option, which is available in the context menu, closes all expanded sections.

### Add folders

You can organize the tree by creating additional folders and subfolders of the **Content structure**.
The same procedure applies to other areas of the Content menu, such as **Media** or **Forms**.

1\. Go to **Content** -> **Content structure**.

2\. In the upper-right corner click **Create content**.

3\. From the available content types, select **Folder**, and click **Create**.

4\. Enter name for the new folder.

5\. (Optional) Add folder description.

6\. Click **Publish**.

The newly created folder is now visible in the Content Tree.
18 changes: 5 additions & 13 deletions docs/getting_started/discover_ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ It's the default dashboard and contains selected blocks.
You can customize the dashboard by changing the blocks and the layout.
For more information, see [Work with dashboard](work_with_dashboard.md).

## Main menu
## Main menu

The main menu allows you to move between important sections of the application.

Expand All @@ -66,20 +66,12 @@ It lets you navigate the Content Tree, create, edit, move, copy, delete content,

## Content Tree

If you want to navigate through your website with a menu, in the main menu, go to the **Content** -> **Content structure**.
The **Content Tree** area is an expandable content menu of your website.
Content Tree is the place where you can navigate through all the content items in your application.
It is available in the Back Office and allows you to browse your content, move content items around, hide them, send them to trash, and perform other actions.

![Content Tree in the menu](img/left_menu_tree.png "Content Tree in the menu")

Unique icons for each content type instantly show you what type of content you are selecting. To add custom icons to your Content Tree, follow [configuration tutorial in developer documentation.]([[= developer_doc =]]/administration/back_office/back_office_elements/custom_icons/#customize-content-type-icons)

Hidden content is greyed out in the tree view.

To simplify the Content Tree, big lists are collapsed and include a **Show more** icon.
You can select it to expand the branches of the tree.
**Collapse all** option, which is available in the context menu, closes all expanded sections.

For more information on custom configuration, go to [Content Tree]([[= developer_doc =]]/administration/back_office/content_tree/) in developer documentation.
For more information, see [Content Tree](content_tree.md).

## Content browser

Expand All @@ -102,7 +94,7 @@ You can also use the **Content Browser** to search the Repository for content, e
If you have administrator permissions, you can also view and manage User accounts and site skeletons in the **Admin** tab.

For more information about users and permissions, see [Users](../permission_management/permissions_and_users.md).

For more information about site skeletons, see [Site skeletons]([[= developer_doc =]]/multisite/site_factory/site_factory_configuration/#site-skeletons).

## Focus mode
Expand Down
54 changes: 44 additions & 10 deletions docs/image_management/edit_images.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,60 @@ description: Edit images in the Image Editor to flip, crop and select a focal po

# Edit images

When you browse the [Media library](content_model.md#content-and-media),
or create or edit a content item that contains an *Image*
or *Image asset* Field, you can perform basic image editing functions by using the Image Editor.
You can edit the image asset's properties.
You can also edit the image itself with the Image Editor.

The Image Editor enables:
!!! note "Editing tool availability"

* image cropping
The editing tools are available when you work in the back office of your application, for example, to:

- browse the Media root folder of the Content Tree
- add an image or an image asset to a content item by selecting it from a Content browser or an Image picker
- insert an image into a [Rich Text Field](create_edit_content_items.md#edit-rich-text-fields)

## Edit image asset

You can edit the image asset's properties, for example, to provide a different caption for the image, or change the alternative text for the image.
To edit an existing image asset, in the back office, the main menu, go to **Content** -> **Media** -> **Images**, choose the image asset that you want to edit and click **Edit**.

Here, you can:

- Change the name and caption for the image.

- Edit the image itself in [Image Editor](#edit-image-in-image-editor).

- Change the alternative text, used by screen readers and in SEO.

!!! note "Use AI to generate alternative text"

If the AI Actions LTS update has been installed and configured in your application, the **Use AI** button is visible.
If you have `Action configuration/View` and `Action configuration/Execute` permissions, you can click it to have the alternative text generated automagically.
The **Go to AI actions** button can also visible to users with access to the **Admin UI**.
By clicking it, you can quickly navigate to [AI actions configuration](work_with_ai_actions.md).

![Generate alternative text](img/alt_text_use_ai.png)

- Change the keywords assigned to the image.
Tagging image assets helps keep the media library organized and enables searching images by tags in the DAM.

### Edit image in Image Editor

When you edit the image asset, you can edit the image itself by using the Image Editor that enables:

* image cropping
* image flipping
* setting a point of focus within an image

Image Editor is available whenever you see the **Edit** icon on the preview.

![Image Editor icon](image_editor_icon.png)

## Flip
#### Flip

With the Flip feature you mirror the image along a horizontal or vertical axis.
Click either the **Horizontal** or **Vertical** button to flip an image.

## Crop
#### Crop

With the Crop feature you can cut the image down to a desired aspect ratio and dimensions.
You can either choose one of the preset aspect ratio options (square, vertical or horizontal rectangle, or widescreen), or click **Custom**.
Expand All @@ -40,7 +74,7 @@ When you confirm the change, the preview refreshes to display a cropped image.
Confirming the crop operation does not mean that it has been saved.
You must click **Save** to apply the changes to the original image.

## Focal point
#### Focal point

If your page contains an image that is larger that the current viewport, for example, when a user accesses the page from a mobile phone,
you can select a point on the image that the view should focus on.
Expand All @@ -58,13 +92,13 @@ Click the **Show point** button to see a target on the preview.
Drag the target to a point on the image that you want to focus on.
Click the **X** icon to restore the original position of the target.

## Undo changes
#### Undo changes

Click the left or right arrow button to undo or redo the most recent change.
Click **Reset image** to restore the original appearance of the image.

![Image Editor: Undo](image_editor_undo.png)

## Save changes
#### Save changes

Click **Save** to apply your edits to the original image and exit the Image Editor screen.
Binary file added docs/image_management/img/alt_text_use_ai.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading