Changing from a plugin to a standalone CLI application

.github/workflows/build.yml

@@ -7,18 +7,16 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, macOS-latest, windows-latest]
-        jdk: [ 11, 17, 21 ]
+        jdk: [ 21 ]
         experimental: [false]
     steps:
       - uses: actions/checkout@v4
+        with:
+          lfs: true
       - name: Set up JDK ${{ matrix.jdk }}
         uses: actions/setup-java@v4
         with:
-          distribution: temurin
+          distribution: adopt
           java-version: ${{ matrix.jdk }}
-          cache: gradle
-      - name: Assemble target plugin
-        uses: gradle/gradle-build-action@v3
-        with:
-          cache-disabled: true
-          arguments: -Dtests.security.manager=false assemble
+      - name: Build
+        run: mvn --batch-mode test

.gitignore

@@ -29,3 +29,17 @@ gradle-app.setting
 .project
 .classpath
 
+# Ignore Gradle project-specific cache directory
+.gradle
+
+# Ignore Gradle build output directory
+build
+
+# intellij files
+.idea/
+*.iml
+*.ipr
+*.iws
+build-idea/
+out/
+

Dockerfile

@@ -0,0 +1,3 @@
+FROM opensearchproject/opensearch:2.18.0
+
+RUN /usr/share/opensearch/bin/opensearch-plugin install --batch https://github.com/opensearch-project/user-behavior-insights/releases/download/2.18.0.2/opensearch-ubi-2.18.0.2.zip

README.md

@@ -1,64 +1,11 @@
-# Search Evaluation Framework
+# OpenSearch Evaluation Framework
 
-This repository contains the search quality evaluation framework as described in the [RFC](https://github.com/opensearch-project/OpenSearch/issues/15354).
+This is an application to perform search evaluation.
 
-Note: Some of the data files in this repository are tracked by `git lfs`.
+## Building
 
-## Repository Contents
-
-* `data` - The data directory contains scripts for creating random UBI queries and events for purposes of development and testing.
-* `opensearch-search-quality-evaluation-plugin` - An OpenSearch plugin that extends the OpenSearch Scheduler plugin that provides the ability to generate scheduled (and on-demand) implicit judgments from UBI data.
-* `opensearch-search-quality-implicit-judgments` - A standalone Java application to generate implicit judgments from indexed UBI data.
-
-## OpenSearch Search Quality Evaluation Plugin
-
-This is an OpenSearch plugin that extends the OpenSearch Scheduler plugin that provides the ability to generate scheduled (and on-demand) implicit judgments from UBI data.
-
-To use the plugin:
-
-```
-./gradlew build
-cd opensearch-search-quality-evaluation-plugin
-docker compose build
-docker compose up
-```
-
-To create a schedule to generate implicit judgments:
+Build the project from the top-level directory to build all projects.
 
 ```
-curl -s -X POST "http://localhost:9200/_plugins/search_quality_eval/schedule?id=1&click_model=coec&job_name=test&interval=60" | jq
-```
-
-See the created job:
-
-```
-curl -s http://localhost:9200/search_quality_eval_scheduled_jobs/_search | jq
-```
-
-To run an on-demand job without scheduling:
-
+mvn clean install
 ```
-curl -X POST "http://localhost:9200/_plugins/search_quality_eval/judgments?click_model=coec&max_rank=20" | jq
-```
-
-To see the job runs:
-
-```
-curl -X POST "http://localhost:9200/search_quality_eval_completed_jobs/_search" | jq
-```
-
-See the first 10 judgments:
-
-```
-curl -s http://localhost:9200/judgments/_search | jq
-```
-
-## OpenSearch Search Quality Implicit Judgments
-
-This is a standalone Java application to generate implicit judgments from indexed UBI data. It runs outside OpenSearch and queries the UBI indexes to get the data for calculating the implicit judgments.
-
-To run it, run the `org.opensearch.eval.App` class. This will connect to OpenSearch running on `localhost:9200`. It expects the `ubi_events` and `ubi_queries` indexes to exist and be populated.
-
-## License
-
-This code is licensed under the Apache 2.0 License. See [LICENSE.txt](LICENSE.txt).

data/esci/index.sh

@@ -7,4 +7,4 @@ echo "Initializing UBI..."
 curl -s -X POST "http://localhost:9200/_plugins/ubi/initialize"
 
 echo "Indexing queries and events..."
-curl -s -T "http://localhost:9200/_bulk?pretty" -H "Content-Type: application/x-ndjson" --data-binary @ubi_queries_events.ndjson
+curl  -X POST 'http://localhost:9200/index-name/_bulk?pretty' --data-binary @ubi_queries_events.ndjson -H "Content-Type: application/x-ndjson"

docker-compose.yaml

@@ -0,0 +1,47 @@
+services:
+
+  opensearch_sef:
+    build: .
+    container_name: opensearch_sef
+    environment:
+      discovery.type: single-node
+      node.name: opensearch
+      plugins.security.disabled: "true"
+      logger.level: debug
+      OPENSEARCH_INITIAL_ADMIN_PASSWORD: SuperSecretPassword_123
+      http.max_content_length: 500mb
+      OPENSEARCH_JAVA_OPTS: "-Xms16g -Xmx16g"
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+      nofile:
+        soft: 65536
+        hard: 65536
+    ports:
+      - "9200:9200"
+      - "9600:9600"
+    networks:
+      - opensearch-net
+    volumes:
+      - opensearch-data1:/usr/share/opensearch/data
+
+  opensearch_sef_dashboards:
+    image: opensearchproject/opensearch-dashboards:2.18.0
+    container_name: opensearch_sef_dashboards
+    ports:
+      - "5601:5601"
+    environment:
+      OPENSEARCH_HOSTS: '["http://opensearch_sef:9200"]'
+      DISABLE_SECURITY_DASHBOARDS_PLUGIN: "true"
+    depends_on:
+      - opensearch_sef
+    networks:
+      - opensearch-net
+
+volumes:
+  opensearch-data1:
+
+networks:
+  opensearch-net:
+    driver: bridge