v6.6.1

6.6.1 - 2021-08-11

• In testing the Keycloak oAuth integration via the Chorus project, identified that the name of the Client that is passed to Keycloak was tied to the KEYCLOAK_REALM env setting. Changed this to be hardcoded to quepid so it can be different then the Realm name (i.e the Realm is Chorus in the Chorus project, and the Client is quepid). Commmit de9e0e6 by @epugh.

v6.6.0

6.6.0 - 2021-08-09

Features

• OpenID Support! Historically you had to create your own account on Quepid, but that is another barrier to entry, and something else to keep track of. Inspired by the issue Add OAuth Providers by @gregoryduckworth #188, #280 and #389 by @epugh implements the first two providers, Google and Keycloak.

• Explain Query Parsing! We often want to know what the search engine is doing to our query. Now you can click the "Explain Query" popup and see what is happening to your query and the various filter queries. This works with both Solr and Elasticsearch. https://twitter.com/dep4b/status/1420480501710827520. #391 by @epugh.

• Frequently type deftype instead of defType in Solr? I found an old, but broken feature that provided a popup warning on common Solr query syntax errors, which has now been restored. #394 by @epugh fixes #390.

Improvements

• Quepid is running on Rails 6! Rails 6 was released ~26 months ago, and represents the future of Rails. I'm excited that this push initiated by @DmitryKey is going to bring us some great new features like: better developer experience with Webpack for JavaScript, ActionText to handle better text formatting of notes and messages about Cases and Queries, ActionCable which will let us notify all the users of the same case that data has changed when it changes. #381 by @DmitryKey with assist from @epugh.

• Make our ActiveRecord modeling for ownership the same. Teams have an owner, Scorers have an owner, but Cases have a user. Now we have case.owner relationship. #359 by @epugh.

• The nav bar link for Scorers is labeled "Custom Scorers", and takes you to an page called /advanced. Both of the names are rather offputting. Creating Scorers is a normal common thing to do in Quepid, so lets label it Scorers. #392 by @epugh fixes #351.

• We had, as someone put it "dinosaur" age CSS tags for mozilla and webkit browsers that triggered lots of browser warnings in the console. #380 by @epugh cleaned up many of them. It has also prompted some thinking about migrating to Bootstrap 5 for the non core pages. If anyone has some CSS skills, please contact @epugh, to help refine the header menu in Bootstrap 5 CSS.

v6.5.5

6.5.5 - 2021-06-30

Features

• You can now tag a field with translate:, as in translate:content and you will get an icon to pop the text open in a Google Translate in a new browser window. #386 by @epugh.

Improvements

• You can now export fields that have the formatting modifiers thumb and image using the detail format. Also improved the handling of the General and Detail export from the Case Listing page. #383 by @epugh fixes #382. Thanks @DmitryKey for the improvement assist and spotting the Case Listing export issue.

• Admin user can now reset a users password with a new password. #385 by @epugh to fix issue identified by @michaelcizmar. Thanks Michael!

• Trying to communciate about HTTPS better when you set up a case. #384 by @epugh inspired by #279 by @arafalov.

6.5.4

6.5.4 - 2021-06-16

Features

• You can now Clone an existing Scorer!. Super useful when you want to start with one of the communal scorers that only go to 10 deep, like P@10, and you want it to be P@40. Introduced as part of #379 by @epugh.

Improvements

• Enable TLS for Redis 6 on Sidekiq on Heroku. #370 by @michaelcizmar fixes #271 by @michaelcizmar. Thanks Michael for your first PR to Quepid!

• During the Case Wizard setup, allow space delimited list of fields, like title overview to be pasted in, just like comma delimited lists. Thanks @peterdm for the assist on the Regex! #378 by @epugh.

• Revamped the layout of the Scorer creation and editing screens to be visually cleaner. Retired the old Fibonnaci scale, and renamed Default to Detail, as well as Short to Graded scales. Introduced Binary as a new, default scale. @DmitryKey and @epugh paired on this during Quepid Qommunity Qoding hour, resulting in #379.

• Shrink production Docker image of Quepid from 2.19GB to 2.17GB by not installing development and test Gems. Commit 426d267 by @epugh.

Bugs

• Preserve the chosen scorer when cloning a case, we were defaulting to the users default scorer. #372 by @epugh fixes #273. Thanks @binarymax and @nathancday for spotting this!

• Ensure that CSV export always has all the columns expected. Queries that aren't fully rated and have blank values didn't put out complete set of , separators. #371 by @epugh fixes #364 by @sstults. Spotted while working with Quaerite.

• If you don't have a Terms & Conditions url, then your users don't have an agreed_time, so the activity pulse chart in the admin errors out. #373 by @epugh sorts this out.

• In the dawn of Quepid, we had agreed_time but not a boolean agreed. A user let me know he couldn't update his email address (from an account in 2015!) because he didn't have agreed=true set. #374 by @epugh cleans up some old user data.

• Fixed the labeling behavior for a Scorer. @DmitryKey and @epugh paired on this during Quepid Qommunity Qoding hour, resulting in #379, fixing #376.

6.5.3

6.5.3 - 2021-06-01

Bugs

• In 6.5.2 we introduced the image:image_path to show larger images, however there is odd wrapping if you don't have many fields listed for a doc. #365 by @epugh fixes this.

• If you didn't have email set up, then sending invites wouldn't work. #369 by @epugh adds better handling for when email hasn't been set up for Quepid.

6.5.2

6.5.2 - 2021-05-27

Improvements

• We have a new Favicon, but didn't use it everywhere, so #363 by @epugh displays the new favicon everywhere.

Features

• Rating documents where the image carries most of the information? The existing thumbnail:image_path limits you to a 75 pixel wide image. We now have image:image_path which displays a 200 pixel wide image. This is an alternative to the media:image_path approach, as this shows up next to your fields versus inline with your fields. #362 by @epugh fixes #360.

6.5.1

6.5.1 - 2021-05-25

Bugs

• Now that we've had the team invitations feature out for a while, we've found places where we make assumptions that aren't correct. This fixes the Admin panel's user page, and makes on the Teams page the messaging around pending invites. #339 by @epugh.

• We discovered that nDCG@10 score for products with a hitCount below 10 (K) was always 0. #336 by @rudibatt fixes this. Thanks @rudibatt for this contribution!

• Passing in quote delimited parameters to bin/docker r wasn't preserving or quotes or escaping the spaces, causing the thor user:create [email protected] "Eric Pugh" "mysuperstrongpassword" example to fail. #340 by @epugh fixes #338.

• Thanks to Liz Novak at VIN for discovering that a non lowercase email address like [email protected] wouldn't be considered a valid email address, so you couldn't send a invite to join Quepid. #342 fixed by #346 by @epugh with assist from @eribeiro.

Improvements

• Uglifier for JavaScript appears to not work with ES6, and so using Terser instead. #329 by @epugh fixes this.

• Add to the Team page the ability to copy the team invitation url to send via other means then Quepid's own email setup. This is helpful if you don't have email set up, or if you want to share invites with team members via chat tools like Slack. Also fixed the user flow of registering after someone has sent out an invite for someone. #337 by @epugh fixes #335 and #326.

• We've added linting for our Dockerfiles via Deepsource.io. #343 by @epugh fixes #341.

• Our Docker images are getting bloated by including log files from dev and test, leading to almost a 3 GB image. Adding a .dockerignore has got us back to around 2.1 GB image. #358 by @epugh fixes #353.

• Remove the remnants of the query test (unit test style custom scorers) and the default scorer table that are still in the source code. #357 by @epugh fixes #349.

• Sometimes you want to create a Case even when you can't validate the connection. Introduce a option to bypass the validation. #356 by @epugh.

• When we export basic rating data there is an extra LF character at the end of the CSV file. This was causing issues when importing ratings into the Quaerite project (tballison/quaerite#14). #355 by @epugh with assist from @sstults fixes #354 by @sstults.

• Quepid 6.5.0 started failing on Kubernetes clusters due to the need to be able to write to the filesystem. Bootsnap dependency upgrade avoids this. #344 by @epugh. Thanks to @LiuCao0614 and @tonomonic for the assist on this.

• Revamp of how all the permissions for doing actions related to Teams, Scorers, and Cases to use the Permissions data we aleady have for a user. Lots of cleanups on those functions. Finally dealt with the left over need to have a Case per User, or the UI blew up. Now you get a nice message, and the UI doesn't blow up. #347 by @epugh fixes #352, #331, #278.

v6.5.0

6.5.0 - 2021-04-22

I can sum up this release of Quepid as either the Hey friend, come join me in improving search release or the so much technical debt has been paid down release.

For those of you upgrading your Docker based install, there are two new ENV variables you need to add to your docker-compose.yml file:

- RAILS_LOG_TO_STDOUT=true
- RAILS_SERVE_STATIC_FILES=true

Hey friend, come join me in improving search

We have added features that make it easier for you to invite your colleagues to come join your team and start rating documents. We
have reworked the initial case creation wizard to be smarter about popping up only the first time you decide to create your own case, instead of having it jump into the user flow when you join an existing team with existing cases. This should make it easier to bring folks interested only in rating docs into Quepid. As part of this, we've also added support for deleting individual Cases and User accounts. So if you invite someone, and change your mind, you can clean up after yourself.

so much technical debt has been paid down

We are finally off Rails 4.2 and are now on the latest Rails 5 release, and we have a shiny new favicon to go with it! We've updated all of our frontend dependencies to the latest versions possible, congruent with the core app still being based on Angular1. The lift from Rails 4 to Rails 5 was a massive effort, and a huge round of thanks goes out to everyone who reported bugs. We've shrunk our list of dependencies by 10% over the v6.4.1 release, and 29% compared to v6.0.1 release, and we're set up now to look at new front end technologies!

We also did some housecleaning by ripping out some features that hadn't seen adoption by users, including "soft delete" of queries and the "unit test" style of custom scorers.

Features

• Invite your friends to join your team on Quepid! You can now send an email with an invite code to join Quepid and your specific team. #259 by @epugh. #305 by @epugh introduced better support for tracking of if someone had gone through the create case wizard or not, extending this feature.

• Add support for sending emails via SMTP, or use Postmark, or don't send emails. #276 by @gabauer fixes #275.

• Let a user (or an Administrator) delete their account from Quepid, handling their cases, scorers, and team memberships. #315 by @epugh fixes #311.

• You can now Delete a case altogether! Historically we had an Archive function, so that you could restore an old case. However, if you are like me, you create lots and lots of throwaway cases, so this allows you to clean up your Quepid setup. This PR also fixed some data modeling issues, and the problem of sometimes have a Try of zero, instead of the default first try of One! We also always include the Try number even if you have named the Try, since that is a big deal. #288 by @epugh fixes #250. Thanks @DmitryKey for help QA'ing this code.

Improvements

• We now have a favicon for Quepid! #317 by @DmitryKey.

• Upgrade to Rails 5 and Ruby 2.7.2! We have been stuck on Rails 4.2 for years, and this unlocks a lot of new capabilities. #256 by @epugh with inspiration from @worleydl.

• Renamed "Export Case" and "Import Ratings" to just "Export" and "Import". #270 by @abhishekChoudhary93 fixes #195.

• Pay down some tech debt by removing our use of Coffee Script in Quepid. #283 by @epugh.

• Build time on CircleCI dropped from ~135 seconds down to ~105 seconds by removing sass gem in favour of sassc. #285 by @epugh.

• We have simplified our queries that involve cases and users that are part of teams by making the simplifying assumption that all owners of teams are also members, which turns out to be true in practice. We also have audited our use of .includes to load child objects (avoiding the N+1 problem), and used more .preload to try and deal with running out of temp space in our database. #286 by @epugh.

• Turns out we had a ERD diagram all along, but it was hidden. Now you can see it on our Data Mapping page, plus we have how to recreate it documented and integrated. #287 by @epugh.

• Remove obscure quepidIf.js file that doesn't seem to do anything. #293 by @worleydl.

• The Export All Cases feature only supported the old "Detail" format, and none of the other export formats, like LTR, Basic, or RRE. Plus the filtering options of "All, Owned, Shared" was pretty simplistic. It wasn't being used, and it added some complexity and performance issues to List All Cases page. So we removed it. #295 by @epugh fixes #294.

• Unit Test style custom scorers let you run a unit test that asserted specific things about specific docs at specific ranks in your search results. This logic however was always convoluted, and only 15 people since 2019 have used it, and I suspect by accident ;-) We want better ways of handling this type of function, so removing this to pay down some tech debt, simplify the database architecture, and open the door to new approach. #296 by @epugh fixes #290. #313 also dealt with this.

• We have removed the Soft Delete for Queries to simplify how Quepid works. If you delete a query in Quepid it is now fully deleted from the database! This is a nice bit of paying down tech debt. Huge thanks to @DmitryKey for testing this PR. #299 by @epugh fixes #298 by @DmitryKey.

• The /case url is one we frequently end up with, and it generates a 404. Thanks @DmitryKey for suggesting we have it route to the case listing page /cases instead. #304 by @epugh.

• The logic around when we popped open the "Create a Case" wizard for a brand new user was somewhat split between the backend home_controller.rb and the front end. It also made anyone who was invited to a team just for rating purposes go through the Create a Case Wizard on their first login, which was awkward. So, converted the concept of a "first_login" for the wizard to just a boolean "completed_case_wizard", and now it it checked if you have NO cases, and popped up, or if you click "Add a Case". #305 by @epugh fixes #281.

• Upgraded to the latest version of Angular 1, 1.8.2, and many of the other front end dependencies. #308 and #324 by @epugh and #320 by @worleydl deals with this.

• You can now override the from email address from the default [email protected] to your own email address. #322 by @slawmac. Thanks @slawmac for this improvement!

Bugs

• You can export a rating that has no actual rating value chosen! #266 by @epugh fixes #265.

• Fixing typo in defining aria-hidden html tag. #269 by @abhishekChoudhary93 fixes #268.

• Rails 5 Upgrade Hangover: fixed strong parameters validation issues, fixed wonky left outer joins on cases showing duplicate cases (and killed performance), eliminated the user_case_finder.rb class in favour of simpler scopes. #292 by @epugh w/ assist from @worleydl. #314 by @worleydl is related. #321 by @janhoy fixed by #323 by @epugh.

• Thanks to @DmitryKey for spotting that we were not tracking scorers for cases (bug introduced when we added the Show Only Ratings feature). #303 by @epugh and @worleydl fixes both #306 AND #289. A twofer!

• Thanks to @DmitryKey for spotting that we were not properly storing the Elasticsearch choice. #310 by @epugh fixes #309.

• There is a known issue where expanding/collapsing queries on a larger case, the UI can become completely locked in Chrome, forcing the user to close out the tab. Thanks to some next level debugging by @LiuCao0614 we have a workaround that disables the JQuery based drag and drop sorting feature that appears to avoid this bug. This is not a fix! This is a workaround! Set QUERY_LIST_SORTABLE=false in your environment to disable the sorting of queries feature. #272 tracks this ongoing issue.

6.4.1 - 2021-01-14

In the 6.4.0 release, the correct splainer-search NPM package was missed in the production Docker image.

This release resolves that oversight. Thanks @LiuCao0614 for testing the upgrade process and reporting the issue.

Improvements

• Small housekeeping update for the hosted app version...

v6.4.1

6.4.1 - 2021-01-14

In the 6.4.0 release, the correct splainer-search NPM package was missed in the production Docker image.

This release resolves that oversite. Thanks @LiuCao0614 for testing the upgrade process and reporting the issue.

Improvements

• Small housekeeping update for the hosted app version running on Heroku. #267 by @dworley.

• Clean up last scoring run details on the Cases Listing page. Now better iconography to call out cases that have queries that need rating. #261 by @epugh fixes #219.

Bugs

• Don't export ratings that don't have a rating set for RRE and Basic export formats. #266 by @epugh fixes #265.

v6.4.0

6.4.0 - 2020-12-18

Only Rated toggle is a powerful new feature added to Quepid, our first relevancy centric feature in a long while.

In certain relevancy cases, you may only care about the ordering of a subset of documents within the entire result set. One particular use case may be in e-commerce where you want certain products to be boosted and others to be demoted in the ranking algorithm. While this was doable in previous versions of Quepid, it could get difficult to manage the list of rated documents, especially if the list went many pages deep into the results.

To help solve this problem we worked with Kobler to introduce several new features to Quepid. We strived to make it easier to work with your corpus of rated documents in the main case view, inside of Explain Missing Documents and within the scorers themselves.

This feature was developed by @worleydl in #229, with an initial POC by @janhoy in #220, and resolves issue #163. Much thanks to @janhoy and the folks at Kobler for this feature.

Features

• Added "Show Only Rated" toggle to the main searchResult listing
• Explain Missing Documents modal now defaults to showing all of your rated documents when you first open it up
• Added eachRatedDoc function to ScorerFactory to iterate over rated documents in scoring
• Provided refreshRatedDocs(k) in ScorerFactory for loading a custom amount of rated documents up to k

Improvements

• Migrated scoring to an asynchronous pipeline
• Fixed stuck "Updating queries" message when creating a new case