Code

Run SVD for concept similarity#56

Code

Draft

f-hafner wants to merge 37 commits intomainf-hafner/mag_sample:mainfrom svd_similarityf-hafner/mag_sample:svd_similarityCopy head branch name to clipboard

+17,988-126

Commits

Commits on Oct 2, 2024

• start script for fitting svd model

  f-hafner

  committed
  b948338

• refactor script for svd model

  f-hafner

  committed
  9c9edc2

• further refactor, add argparser

  f-hafner

  committed
  5bb11d2

• fix random state variable

  f-hafner

  committed
  dc70263

• add script to pipeline

  f-hafner

  committed
  4af6153

• add docstring

  f-hafner

  committed
  fc94cc0

• update end year

  f-hafner

  committed
  e57d453

• fix typo in pipeline

  f-hafner

  committed
  94299e3

• return ordered fields of study

  f-hafner

  committed
  e0bcd59

• add log

  f-hafner

  committed
  7fc718a

• add model checks

  f-hafner

  committed
  2fe805c

• remove todos

  f-hafner

  committed
  d6fd2fd

Commits on Oct 6, 2024

• feat: Add script to apply SVD model to all papers in the database

  chrished

  committed
  a9d0490

• predicted vectors take too much storage

  chrished

  committed
  30e3b0d

Commits on Oct 14, 2024

• Add script to calculate topic similarity using SVD embeddings, own similarity implemented only

  chrished

  committed
  be62899

• Add similarity calculation for student and faculty SVD embeddings

  chrished

  committed
  eb2dd35

• feat: Add SVD-based similarity calculation for closest collaborators

  chrished

  committed
  7fc32ac

• feat: Update to use SVD-based similarity function for collaborators

  chrished

  committed
  bb1bdca

• save model with ndim

  f-hafner

  committed
  050514b

• run multiple svd models

  f-hafner

  committed
  6930b90

• remove old log file

  f-hafner

  committed
  1af2792

Commits on Oct 15, 2024

• add logs for multiple svd runs

  f-hafner

  committed
  a6bcfb8

• Merge branch 'svd_similarity' of github.com:f-hafner/mag_sample into svd_similarity

  chrished

  committed
  0b10a1b

Commits on Oct 16, 2024

• move function for svd similarity calculation to topic_similarity_functions.py. Adjust to load all fields including level 0 in fit_svd

  chrished

  committed
  000ea90

• style: Format code for consistency and readability in topic_similarity_functions.py

  chrished

  committed
  d84c314

• fix: Validate existence of columns before computing SVD similarity

  chrished

  committed
  d66c6f4

Commits on Oct 18, 2024

• transform_topics function to create a DataFrame with transformed data, fix index accessibility issues, remaining: missing FielfOfStudyId in topics_collaborators_affiliations df

  chrished

  committed
  d86f016

Commits on Oct 19, 2024

• extend model checks, add more dimensions

  f-hafner

  committed
  c036804

• add new logs

  f-hafner

  committed
  9069cbb

• move misplaced docstring

  f-hafner

  committed
  1919d2a

• fix some more docstrings

  f-hafner

  committed
  651cc74

Commits on Oct 20, 2024

• iterate over ndims within python script

  f-hafner

  committed
  853e7e6

• ignore ipynb checkpoints

  f-hafner

  committed
  f5495ad

• add script to start jupyter on remote

  f-hafner

  committed
  a1fe172

• fix start_jupyter

  f-hafner

  committed
  a4eee7c

Commits on Oct 28, 2024

• read svd similarity

  chrished

  committed
  2d05977

Commits on Oct 29, 2024

• add read svd similarity to pipeline

  chrished

  committed
  577d174