[DONE] Add the AI Enhancement application (#1025)

* Create the AI Enhancement application * Add the connection to the API * Create the function to connect to the API * Create the function to get the enrichment * Create the function to get a special enrichment * Add some tests * Add unit tests & view * Fix for Flake8 * Create the AIEnrichment models & test it * Add the USE_AI_ENHANCEMENT setting * Create the function to create an enrichmenet from an URL * Create the method to get the latest version of enrichment * Create the method to get the versions of an enrichment * Create the method to get a specific version of an enrichment * Update the unit tests * Create the AI_ENRICHMENT_CLIENT_ID & AI_ENRICHMENT_SECRET settings * Add the AI button * Begin to add the AI enhancement creation * Add the init file for the migraton * Add the enrichment creation with the button * Edit the video view to use the AI if the user ask it * Create the title and description field for the choice form * Fix the admin, model & template tags * Create the view to choose AI generated or human created field for the video informations * Fix flake8 * Renomme template * Rename variable to snake case * Create the view to choose AI generated or humain created field for the transcription * Change console.error for showalert * Add the view to toggle webhook * Add redirection when the user create an ai enhancement * Add the extract_json_from_str util function * Fix the decode bug in the choice form * Fix the discipline field in the choice form * Fix flake8 * Add unit tests for the extract_json_from_str function * Put the CSS code in css file * Add the AI Enhancement app in configuration.json * Update the subtitles choice with IA * Fix flake8 * Fix the empty discipline bug * Add pydoc * Realize the TODO * Add the deletion of the enrichment at the end * Fix some problem * Add the translations * Fix flake8 * Remove the AI_ENRICHMENT_DIR setting * Add the docs for the settings * Add a line at the end of the file * Fix flake8 * Add JS Doc & remove enrich-transcript.js file * Add & rename some settings & add the settings in the file test * Update the description * Remove print * Change 'enrichment' by 'enhancement' * Fix template tags * Update forms.py * Remove unused aria-label & some blank lines * Add global docs * Move some util functions to the main package * Fix the tilte page * Change 'warning' to 'success' * Translate * Change 'Aristotle' to 'Aristote' * Add the notify for thrid-party service * Fix the link * Fix flake8 * Make the translations * Change the default value for USE_AI_ENHANCEMENT * Rename user_can_enrich_video by user_can_enhance_video * Rename enrich_video & enrich_video_json by enhance_video & enhance_video_json * Rename enrich_subtitles by enhance_subtitles * Rename enrich_form by enhance_form * Rename settings * Update test settings * Restrict AI enhancement to the staff only * Remove title * Remove aria-label * Update the create_enhancement template * Update views * Remove console.log * Update the configuration file * Update the documentation * Update str in forms * Make enhancements available for staff only * Add help texts in the AIEnhancement model * Add version * Use bootstrap * Replace str by slug * Change 'my' bu 'the' * data-bs-toggle & data-bs-placement * Remove convert_time & pad_zero function * Move the stylesheet link in the page_extra_head block * Make lang * Update translations * Compile lang * Fix some bugs * Simplify the video view * Add some imports * 🐛 Fix the create_enhancement_from_url method * Handle the errors * ✅ Add unit tests for the get_token function * Add __str__ method & sites property * Make translations * Fix flake8 * ✅ Add unit tests for the model * ✅ Add unit tests for the enhance video route * 👕 Remove unnecessary imports * Make the Badatos requested changes * Add filter dans search fields for AIEnhancement * Change ForeignKey to OneToOneField * Add AI_ENHANCEMENT_FIELDS_HELP_TEXT setting * remove protocole to fetch, maybe get url in ai-enhancement/enrich-video template * mark False as default value tu USE_AI_ENHANCEMENT * add notify user at the end of IA improvement * add translations * add restrict access to ai enhancement - remove delete ai enhancement * add signal to delete enhancement - use update or create to create one - remove delete calling from video application and remove tag to show tooltips * add link to go to subtitle part and return to video - fix link to fetch * add translation * fix translation and unit test * fix unit test and check if ai enhancement exist before deletion * remove quote from ai title and use client get uel in test view * add login in test views * fix translation * Change MP4 format to MP3 format for Aristote * 🐛 Fix the media_types tab * refactor get video mp3 url * add link to get more information about third service * Feat implement quiz in Aristote enhancement * add AI_ENHANCEMENT_CGU_URL in configuration * 🐛 Fix the keywords field * Update JSDoc * Fix review requests * Fix missing function error * Fix translations * 🚸 Remove tag when it is in input tag in Aristote form * 🚸 Modify video position in quiz page * 🚸 Improve quiz access in video page * 🚸 Add scroll when click on collapse button * use hashlib to pseudonymise user with AI * 💄 Improve ui of some quiz pages * ✨ Add the button in the Aristote page to delete the enhancement * 🎨 Improve css quiz code * 🐛 Fix an error when submitting an empty quiz * 🌐 Add translations to delete the enhancement * delete all instance of enhancement when video encoding * 🧑‍💻 Add filter to get a value in dict * 📝 Update PyDoc * 📝 Update version in doc * 💄 Add border color when submitting quiz * ♻️ Create function to select an element * 💄 Improve question infos when submitting * 🚸 Improve buttons in quiz page * 🎨 Move decodeString & removeAccentsAndLowerCase function in main.js * 🧑‍💻 Remove log * 💄 Replace success by primary * 🌐 Change IA to AI * ⚡ Add penalty when select incorrect anwer in multiple choice question * 📝 Add commentary * 🌐 Update the french translation * 🔊 Add logs * 🚨 Apply Flake8 & black * 🚑 Fix bug that prevented deleting a question in a quiz * 🚸 Add delete quiz button in aside menu in video page * ♻️ Simplify update_questions function * 🚸 Add d-none class for the deleteIpnuts * 🔥 Remove unnecessary code * fix btn secondary and translation * 🐛 Fix non-creation question * add new line at the end of file * move import quizz to utils and add redirect to quiz edit * add question id in quizz form to update or create question * replace go to by import * fix dlete and create question for quizz * fix add quizz * put the default value of create quiz to 1 and fix bug when deleting new form * play video when show reponse quiz * keep user answer when submit quizz, show question form error, add play for video player * fix extra formset to 1 * fix translation and add pydoc and improve get question to prevent exception --------- Co-authored-by: ptitloup <[email protected]> Co-authored-by: Aymeric Jakobowski <[email protected]>
EsupPortail · Jun 4, 2024 · 26af203 · 26af203
1 parent 619eed0
commit 26af203
Show file tree

Hide file tree

Showing 56 changed files with 4,410 additions and 628 deletions.
diff --git a/pod/ai_enhancement/__init__.py b/pod/ai_enhancement/__init__.py
diff --git a/pod/ai_enhancement/admin.py b/pod/ai_enhancement/admin.py
@@ -0,0 +1,27 @@
+"""Esup-Pod AI enhancement admin."""
+from django.contrib import admin
+
+from .models import AIEnhancement
+
+
+@admin.register(AIEnhancement)
+class AIEnhancementAdmin(admin.ModelAdmin):
+    """AIEnhancement admin page."""
+
+    date_hierarchy = "updated_at"
+    list_display = (
+        "id",
+        "ai_enhancement_id_in_aristote",
+        "video",
+        "is_ready",
+        "created_at",
+        "updated_at",
+    )
+    list_display_links = ("id", "ai_enhancement_id_in_aristote")
+    list_filter = ("is_ready", "created_at", "updated_at")
+    search_fields = [
+        "ai_enhancement_id_in_aristote",
+        "video__title",
+        "video__id",
+        "video__slug",
+    ]
diff --git a/pod/ai_enhancement/apps.py b/pod/ai_enhancement/apps.py
@@ -0,0 +1,11 @@
+"""Esup-Pod AI Enhancement apps."""
+from django.apps import AppConfig
+from django.utils.translation import gettext_lazy as _
+
+
+class IaEnhancementConfig(AppConfig):
+    """AI Enhancement app configuration."""
+
+    name = "pod.ai_enhancement"
+    default_auto_field = "django.db.models.BigAutoField"
+    verbose_name = _("Artificial Intelligence Enhancement")
diff --git a/pod/ai_enhancement/context_processors.py b/pod/ai_enhancement/context_processors.py
@@ -0,0 +1,16 @@
+"""Esup-Pod ai_enhancement context_processors."""
+from django.conf import settings as django_settings
+
+USE_AI_ENHANCEMENT = getattr(
+    django_settings,
+    "USE_AI_ENHANCEMENT",
+    False,
+)
+
+
+def context_settings(request):
+    """Return all context settings for ai_enhancement app"""
+    new_settings = {
+        "USE_AI_ENHANCEMENT": USE_AI_ENHANCEMENT,
+    }
+    return new_settings
diff --git a/pod/ai_enhancement/forms.py b/pod/ai_enhancement/forms.py
@@ -0,0 +1,264 @@
+"""Forms used in ai_enhancement application."""
+from django import forms
+from django.conf import settings
+from django.utils.translation import ugettext_lazy as _
+from tagging.fields import TagField
+
+from pod.main.forms_utils import add_placeholder_and_asterisk
+from pod.video.models import Video, Discipline
+
+AI_ENHANCEMENT_FIELDS_HELP_TEXT = getattr(
+    settings,
+    "AI_ENHANCEMENT_FIELDS_HELP_TEXT",
+    {
+        "title": {
+            "choose_information_string": _("Choose the title."),
+            "initial_information": _("The initial title is loading…"),
+            "initial_information_t": _("Your initial title."),
+            "ai_information": _("The title proposed by the Aristote AI is loading…"),
+            "ai_information_t": _("The title proposed by the Aristote AI."),
+        },
+        "description": {
+            "choose_information_string": _("Choose the description."),
+            "initial_information": _("The initial description is loading…"),
+            "initial_information_t": _("Your initial description."),
+            "ai_information": _(
+                "The description proposed by the Aristote AI is loading…"
+            ),
+            "ai_information_t": _("The description proposed by the Aristote AI."),
+        },
+        "tags": {
+            "choose_information_string": _("Choose the tags."),
+            "ai_information": _("The tags proposed by the Aristote AI is loading…"),
+        },
+        "discipline": {
+            "choose_information_string": _("Choose the discipline."),
+            "initial_information": _("The initial discipline is loading…"),
+            "initial_information_t": _("Your initial discipline."),
+            "ai_information": _("The discipline proposed by the Aristote AI is loading…"),
+            "ai_information_t": _("The discipline proposed by the Aristote AI."),
+        },
+    },
+)
+AI_ENHANCEMENT_CGU_URL = getattr(
+    settings,
+    "AI_ENHANCEMENT_CGU_URL",
+    "",
+)
+
+
+class AIEnhancementChoice(forms.ModelForm):
+    """Form class for choosing the title of a video with the AI enhancement."""
+
+    class Meta:
+        """Meta class."""
+
+        model = Video
+        fields = [
+            "title",
+            "description",
+            "tags",
+            "discipline",
+        ]
+
+    title = forms.CharField(
+        label=_("Title"),
+        widget=forms.TextInput(
+            attrs={
+                "aria-describedby": "id_titleHelp",
+            },
+        ),
+        help_text=_(
+            """
+            Please choose a title between 1 and 250 characters.
+        """
+        ),
+    )
+
+    description = forms.CharField(
+        label=_("Description"),
+        widget=forms.Textarea(
+            attrs={
+                "aria-describedby": "id_descriptionHelp",
+            },
+        ),
+        required=False,
+        help_text=_("Please choose a description."),
+    )
+
+    tags = TagField(
+        help_text=_(
+            """
+            Please choose tags for your video.
+            Separate tags with spaces, enclose the tags consist of several words in quotation marks.
+        """
+        ),
+        verbose_name=_("Tags"),
+    )
+
+    disciplines = forms.ModelChoiceField(
+        label=_("Discipline"),
+        queryset=Discipline.objects.all(),
+        required=False,
+        help_text=_("Please choose the discipline of your video."),
+    )
+
+    fieldsets = [
+        (
+            "choose_title",
+            {
+                "legend": f"<i class='bi bi-info-lg' aria-hidden='true'></i>&nbsp;\
+                    {AI_ENHANCEMENT_FIELDS_HELP_TEXT['title']['choose_information_string']}<br>\
+                    <div>\
+                        <div class='row'>\
+                            <div id='initial-version-title' class='col'>\
+                                <div\
+                                    class='border-d rounded-4 p-3 mb-3 mt-3 blockquote'\
+                                    title='{AI_ENHANCEMENT_FIELDS_HELP_TEXT['title']['initial_information_t']}'>\
+                                        {AI_ENHANCEMENT_FIELDS_HELP_TEXT['title']['initial_information']}\
+                                </div>\
+                            </div>\
+                            <div id='ai-version-title' class='col'>\
+                                <div\
+                                    class='border-d rounded-4 p-3 mb-3 mt-3 blockquote'\
+                                    title={AI_ENHANCEMENT_FIELDS_HELP_TEXT['title']['ai_information_t']}>\
+                                        {AI_ENHANCEMENT_FIELDS_HELP_TEXT['title']['ai_information']}\
+                                </div>\
+                            </div>\
+                        </div>\
+                    </div>",
+                "fields": ["title"],
+            },
+        ),
+        (
+            "choose_description",
+            {
+                "legend": f"<i class='bi bi-info-lg' aria-hidden='true'></i>&nbsp;\
+                {AI_ENHANCEMENT_FIELDS_HELP_TEXT['description']['choose_information_string']}<br>\
+                <div>\
+                    <div class='row'>\
+                        <div id='initial-version-description' class='col'>\
+                            <div\
+                                class='border-d rounded-4 p-3 mb-3 mt-3 blockquote'\
+                                title='{AI_ENHANCEMENT_FIELDS_HELP_TEXT['description']['initial_information_t']}'>\
+                                    {AI_ENHANCEMENT_FIELDS_HELP_TEXT['description']['initial_information']}\
+                            </div>\
+                        </div>\
+                        <div id='ai-version-description' class='col'>\
+                            <div\
+                                class='border-d rounded-4 p-3 mb-3 mt-3 blockquote'\
+                                title={AI_ENHANCEMENT_FIELDS_HELP_TEXT['description']['ai_information_t']}>\
+                                    {AI_ENHANCEMENT_FIELDS_HELP_TEXT['description']['ai_information']}\
+                            </div>\
+                        </div>\
+                    </div>\
+                </div>",
+                "fields": ["description"],
+            },
+        ),
+        (
+            "choose_tags",
+            {
+                "legend": f"<i class='bi bi-info-lg' aria-hidden='true'></i>&nbsp;\
+            {AI_ENHANCEMENT_FIELDS_HELP_TEXT['tags']['choose_information_string']}<br>\
+            <div id='tags-container'>\
+                <div class='row'>\
+                    <div class='col' id='tags-informations-text'>\
+                        <div\
+                            class='border-d rounded-4 p-3 mb-3 mt-3 blockquote'>\
+                                {AI_ENHANCEMENT_FIELDS_HELP_TEXT['tags']['ai_information']}\
+                        </div>\
+                    </div>\
+                </div>\
+            </div>",
+                "fields": ["tags"],
+            },
+        ),
+        (
+            "choose_disciplines",
+            {
+                "legend": f"<i class='bi bi-info-lg' aria-hidden='true'></i>&nbsp;\
+            {AI_ENHANCEMENT_FIELDS_HELP_TEXT['discipline']['choose_information_string']}<br>\
+            <div>\
+                <div class='row'>\
+                    <div id='initial-version-disciplines' class='col'>\
+                        <div\
+                            class='border-d rounded-4 p-3 mb-3 mt-3 blockquote'\
+                            title='{AI_ENHANCEMENT_FIELDS_HELP_TEXT['discipline']['initial_information_t']}'>\
+                                {AI_ENHANCEMENT_FIELDS_HELP_TEXT['discipline']['initial_information']}\
+                        </div>\
+                    </div>\
+                    <div id='ai-version-disciplines' class='col'>\
+                        <div\
+                            class='border-d rounded-4 p-3 mb-3 mt-3 blockquote'\
+                            title={AI_ENHANCEMENT_FIELDS_HELP_TEXT['discipline']['ai_information_t']}>\
+                                {AI_ENHANCEMENT_FIELDS_HELP_TEXT['discipline']['ai_information']}\
+                        </div>\
+                    </div>\
+                </div>\
+            </div>",
+                "fields": ["disciplines"],
+            },
+        ),
+    ]
+
+    def __init__(self, *args, **kwargs):
+        """Init method."""
+        super(AIEnhancementChoice, self).__init__(*args, **kwargs)
+        self.fields = add_placeholder_and_asterisk(self.fields)
+
+
+class NotifyUserThirdPartyServicesForm(forms.Form):
+    """Form to notify user about third party services."""
+
+    agree = forms.BooleanField(
+        label=_("I agree to use third-party services"),
+        help_text=_(
+            "Please check this box if you agree to use a third-party service to improve this video."
+        ),
+        widget=forms.CheckboxInput(
+            attrs={
+                "aria-describedby": "id_agreeHelp",
+            },
+        ),
+    )
+
+    def __init__(self, *args, **kwargs) -> None:
+        """Init method."""
+        super(NotifyUserThirdPartyServicesForm, self).__init__(*args, **kwargs)
+        self.fields = add_placeholder_and_asterisk(self.fields)
+        if AI_ENHANCEMENT_CGU_URL != "" and self.fields.get("agree"):
+            to_know_more = '<a href="%s" target="_blank" >' % AI_ENHANCEMENT_CGU_URL
+            to_know_more += (
+                '  <i class="bi bi-box-arrow-up-right" aria-hidden="true"></i>&nbsp;'
+            )
+            to_know_more += _("For more information.")
+            to_know_more += "</a>"
+            self.fields["agree"].help_text += "&nbsp;" + to_know_more
+
+
+class NotifyUserDeleteEnhancementForm(forms.Form):
+    """Form to notify user before delete an enhancement."""
+
+    confirm = forms.BooleanField(
+        label=_("I want to delete this enhancement"),
+        help_text=_("Please check this box if you want to delete this enhance."),
+        widget=forms.CheckboxInput(
+            attrs={
+                "aria-describedby": "id_confirmHelp",
+            },
+        ),
+    )
+
+    def __init__(self, *args, **kwargs) -> None:
+        """Init method."""
+        super(NotifyUserDeleteEnhancementForm, self).__init__(*args, **kwargs)
+        self.fields = add_placeholder_and_asterisk(self.fields)
+        if AI_ENHANCEMENT_CGU_URL != "" and self.fields.get("agree"):
+            to_know_more = '<a href="%s" target="_blank" >' % AI_ENHANCEMENT_CGU_URL
+            to_know_more += (
+                '  <i class="bi bi-box-arrow-up-right" aria-hidden="true"></i>&nbsp;'
+            )
+            to_know_more += _("For more information.")
+            to_know_more += "</a>"
+            self.fields["agree"].help_text += "&nbsp;" + to_know_more
diff --git a/pod/ai_enhancement/migrations/__init__.py b/pod/ai_enhancement/migrations/__init__.py
diff --git a/pod/ai_enhancement/models.py b/pod/ai_enhancement/models.py
@@ -0,0 +1,65 @@
+from django.db import models
+from django.utils.translation import ugettext as _
+from django.dispatch import receiver
+from django.db.models.signals import post_save
+from pod.video.models import Video
+
+
+class AIEnhancement(models.Model):
+    """AIEnhancement model."""
+
+    class Meta:
+        """Metadata class for AIEnhancement model."""
+
+        ordering = ["-created_at"]
+        get_latest_by = "updated_at"
+        verbose_name = _("AI enhancement")
+        verbose_name_plural = _("AI enhancements")
+
+    video = models.OneToOneField(
+        Video,
+        verbose_name=_("Video"),
+        on_delete=models.CASCADE,
+        help_text=_("Select the video to enhance with AI"),
+    )
+    created_at = models.DateTimeField(
+        verbose_name=_("Created at"),
+        auto_now_add=True,
+        help_text=_("The date and time when the enhancement was created"),
+    )
+    updated_at = models.DateTimeField(
+        verbose_name=_("Updated at"),
+        auto_now=True,
+        help_text=_("The date and time when the enhancement was updated"),
+    )
+    is_ready = models.BooleanField(
+        verbose_name=_("Is ready"),
+        default=False,
+        help_text=_("Check if the enhancement is ready"),
+    )
+    ai_enhancement_id_in_aristote = models.TextField(
+        verbose_name=_("AI enhancement ID in Aristote"),
+        help_text=_("Enter the ID of the enhancement in Aristote"),
+    )
+
+    @property
+    def sites(self) -> models.QuerySet:
+        """Return the sites of the video."""
+        return self.video.sites
+
+    def __str__(self) -> str:
+        """Return the string representation of the AI enhancement."""
+        return f"{self.video.title} - {self.ai_enhancement_id_in_aristote}"
+
+
+@receiver(post_save, sender=Video)
+def delete_AIEnhancement(sender, instance, created, **kwargs):
+    """
+    Delete AIEnhancement if launch encoding if requested.
+
+    Args:
+        sender (:class:`pod.video.models.Video`): Video model class.
+        instance (:class:`pod.video.models.Video`): Video object instance.
+    """
+    if hasattr(instance, "launch_encode") and instance.launch_encode is True:
+        AIEnhancement.objects.filter(video=instance).delete()
diff --git a/pod/ai_enhancement/static/ai_enhancement/css/choose-video-element.css b/pod/ai_enhancement/static/ai_enhancement/css/choose-video-element.css
@@ -0,0 +1,8 @@
+.border-d {
+    border: 1px dashed;
+}
+
+.border-d:hover, .enrich-input-selected {
+    border: 1px solid;
+    background-color: var(--background-color);
+}