Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

More CLI feedback in SQLAlchemy DB migration #5213

Closed
chrisjsewell opened this issue Nov 2, 2021 · 1 comment
Closed

More CLI feedback in SQLAlchemy DB migration #5213

chrisjsewell opened this issue Nov 2, 2021 · 1 comment
Assignees
@chrisjsewell
Labels
topic/database-migrations topic/database-sqlalchemy topic/setup

Comments

@chrisjsewell
Copy link
Member

chrisjsewell commented Nov 2, 2021 •
edited
Loading

Using verdi quicksetup, for django it prints out every migration on a separate line (which is perhaps a bit overkill), whereas for sqlalchemy there is no info printed by default (you have to change the alembic logging level to INFO) .

Something "in the middle" would be nice; at least the to/from versions

django default:

Success: created new profile `to-delete`.
Report: migrating the database.
Operations to perform:
  Apply all migrations: auth, contenttypes, db
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0001_initial... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying db.0001_initial... OK
  Applying db.0002_db_state_change... OK
  Applying db.0003_add_link_type... OK
  Applying db.0004_add_daemon_and_uuid_indices... OK
  Applying db.0005_add_cmtime_indices... OK
  Applying db.0006_delete_dbpath... OK
  Applying db.0007_update_linktypes... OK
  Applying db.0008_code_hidden_to_extra... OK
  Applying db.0009_base_data_plugin_type_string... OK
  Applying db.0010_process_type... OK
  Applying db.0011_delete_kombu_tables... OK
  Applying db.0012_drop_dblock... OK
  Applying db.0013_django_1_8... OK
  Applying db.0014_add_node_uuid_unique_constraint... OK
  Applying db.0015_invalidating_node_hash... OK
  Applying db.0016_code_sub_class_of_data... OK
  Applying db.0017_drop_dbcalcstate... OK
  Applying db.0018_django_1_11... OK
  Applying db.0019_migrate_builtin_calculations... OK
  Applying db.0020_provenance_redesign... OK
  Applying db.0021_dbgroup_name_to_label_type_to_type_string... OK
  Applying db.0022_dbgroup_type_string_change_content... OK
  Applying db.0023_calc_job_option_attribute_keys... OK
  Applying db.0024_dblog_update... OK
  Applying db.0025_move_data_within_node_module... OK
  Applying db.0026_trajectory_symbols_to_attribute... OK
  Applying db.0027_delete_trajectory_symbols_array... OK
  Applying db.0028_remove_node_prefix... OK
  Applying db.0029_rename_parameter_data_to_dict... OK
  Applying db.0030_dbnode_type_to_dbnode_node_type... OK
  Applying db.0031_remove_dbcomputer_enabled... OK
  Applying db.0032_remove_legacy_workflows... OK
  Applying db.0033_replace_text_field_with_json_field... OK
  Applying db.0034_drop_node_columns_nodeversion_public... OK
  Applying db.0035_simplify_user_model... OK
  Applying db.0036_drop_computer_transport_params... OK
Updating settings  [####################################]  1/1
 OK
  Applying db.0038_data_migration_legacy_job_calculations... OK
  Applying db.0039_reset_hash... OK
  Applying db.0040_data_migration_legacy_process_attributes... OK
  Applying db.0041_seal_unsealed_processes... OK
  Applying db.0042_prepare_schema_reset... OK
  Applying db.0043_default_link_label... OK
  Applying db.0044_dbgroup_type_string... OK
  Applying db.0045_dbgroup_extras... OK
  Applying db.0046_add_node_repository_metadata... OK
  Applying db.0047_migrate_repository... OK
  Applying db.0048_computer_name_to_label... OK
  Applying db.0049_entry_point_core_prefix... OK
Success: database migration completed.

sqlalchemy default

Success: created new profile `to-delete2`.
Report: migrating the database.
Success: database migration completed.

sqlachemy alembic logging level INFO

Success: created new profile `to-delete3`.
Report: migrating the database.
11/02/2021 06:06:37 PM <75788> alembic.runtime.migration: [INFO] Context impl PostgresqlImpl.
11/02/2021 06:06:37 PM <75788> alembic.runtime.migration: [INFO] Will assume transactional DDL.
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade  -> e15ef2630a1b, Initial schema
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade e15ef2630a1b -> 70c7d732f1b2, Deleting dbpath table and triggers
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 70c7d732f1b2 -> a6048f0ffca8, Updating link types - This is a copy of the Django migration script
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade a6048f0ffca8 -> 89176227b25, Add indexes to dbworkflowdata table
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 89176227b25 -> 35d4ee9a1b0e, Migrating 'hidden' properties from DbAttribute to DbExtra for code.Code. nodes
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 35d4ee9a1b0e -> 7a6587e16f4c, Unique constraints for the db_dbgroup_dbnodes table
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 7a6587e16f4c -> 0aebbeab274d, Correct the type string for the base data types
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 0aebbeab274d -> 6c629c886f84, Add the process_type column to DbNode
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 6c629c886f84 -> f9a69de76a9a, Delete the kombu tables that were used by the old Celery based daemon and the obsolete related timestamps
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade f9a69de76a9a -> a514d673c163, Drop the DbLock model
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade a514d673c163 -> 59edaf8a8b79, Adding indexes and constraints to the dbnode-dbgroup relationship table
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 59edaf8a8b79 -> 62fe0d36de90, Add a unique constraint on the UUID column of the Node model
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 62fe0d36de90 -> 5d4d844852b6, Invalidating node hash - User should rehash nodes for caching
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 5d4d844852b6 -> a603da2cc809, Correct the type string for the code class
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade a603da2cc809 -> 162b99bca4a2, Drop the DbCalcState table
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 162b99bca4a2 -> 140c971ae0a3, Migration to reflect the name change of the built in calculation entry points in the database.
11/02/2021 06:06:38 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 140c971ae0a3 -> 239cea6d2452, Migration after the provenance redesign
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 239cea6d2452 -> b8b23ddefad4, DbGroup class: Rename name with label and type with type_string
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade b8b23ddefad4 -> e72ad251bcdb, DbGroup class: change type_string values
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade e72ad251bcdb -> 7ca08c391c49, Migration of ProcessNode attributes for metadata options whose key changed.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 7ca08c391c49 -> 041a79fc615f, This migration cleans the log records from non-Node entity records.
It removes from the DbLog table the legacy workflow records and records
that correspond to an unknown entity and places them to corresponding files.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 041a79fc615f -> ea2f50e7f615, This migration creates UUID column and populates it with distinct UUIDs
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade ea2f50e7f615 -> 375c2db70663, This migration adds uniqueness constraint to the UUID column.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 375c2db70663 -> 6a5c2ea1439d, Data migration for `Data` nodes after it was moved in the `aiida.orm.node` module changing the type string.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 6a5c2ea1439d -> 37f3d4882837, Make all uuid columns unique
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 37f3d4882837 -> 12536798d4d3, Move trajectory symbols from repository array to attribute
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 12536798d4d3 -> ce56d84bcc35, Delete trajectory symbols array from the repository and the reference in the attributes
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade ce56d84bcc35 -> 61fc0913fae9, Final data migration for `Nodes` after `aiida.orm.nodes` reorganization was finalized to remove the `node.` prefix
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 61fc0913fae9 -> d254fdfed416, Data migration for after `ParameterData` was renamed to `Dict`.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade d254fdfed416 -> 5ddd24e52864, Renaming `DbNode.type` to `DbNode.node_type`
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 5ddd24e52864 -> 5a49629f0d45, Adding indices on the `input_id`, `output_id` and `type` column of the `DbLink` table
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 5a49629f0d45 -> 3d6190594e19, Remove `DbComputer.enabled`
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 3d6190594e19 -> 1b8ed3425af9, Remove legacy workflows
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 1b8ed3425af9 -> 1830c8430131, Drop the columns `nodeversion` and `public` from the `DbNode` model.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 1830c8430131 -> de2eaf6978b4, Drop various columns from the `DbUser` model.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade de2eaf6978b4 -> 07fac78e6209, Drop the `transport_params` from the `Computer` database model.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 07fac78e6209 -> 26d561acd560, Data migration for legacy `JobCalculations`.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 26d561acd560 -> e797afa09270, Invalidating node hash - User should rehash nodes for caching
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade e797afa09270 -> e734dd5e50d7, Data migration for some legacy process attributes.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade e734dd5e50d7 -> 7b38a9e783e7, Seal any process nodes that have not yet been sealed but should.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 7b38a9e783e7 -> 91b573400be5, Prepare schema reset.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 91b573400be5 -> 118349c10896, Update all link labels with the value `_return` which is the legacy default single link label.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 118349c10896 -> bf591f31dd12, Migration after the `Group` class became pluginnable and so the group `type_string` changed.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade bf591f31dd12 -> 0edcdd5a30f0, Migration to add the `extras` JSONB column to the `DbGroup` model.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 0edcdd5a30f0 -> 7536a82b2cc4, Migration to add the `repository_metadata` JSONB column.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 7536a82b2cc4 -> 1feaea71bd5a, Migrate the file repository to the new disk object store based implementation.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 1feaea71bd5a -> 535039300e4a, Rename the ``name`` column of the ``Computer`` entity to ``label``.
11/02/2021 06:06:39 PM <75788> alembic.runtime.migration: [INFO] Running upgrade 535039300e4a -> 34a831f4286d, Update node types after `core.` prefix was added to entry point names.
Success: database migration completed.
@chrisjsewell chrisjsewell self-assigned this Nov 2, 2021
@chrisjsewell
Copy link
Member Author

closed in #5330

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chrisjsewell chrisjsewell

Labels
topic/database-migrations topic/database-sqlalchemy topic/setup
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@chrisjsewell