DRAFT - Move Orca to a separate Postgres extension #1111
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This patch adds a skeleton for a new extension gpcontrib/gp_orca with empty _PG_init and _PG_fini. The extension will be filled in consequent commits. Plus this patch updates 'shared_preload_libraries' value in the postgresql.conf.sample file if ORCA is enabled.
GPMemoryProtect_TrackStartupMemory() contained compile-time dependency on Orca. It prevented moving of all Orca's code into a shared lib and making its work transparent to the GPDB core. This patch reworks the logic of the GPMemoryProtect_TrackStartupMemory(). Now extension should call GPMemoryProtect_RequestAddinStartupMemory() at its _PG_init(), if the extension affects the per-process startup committed memory. Such call is added to the Orca's _PG_init(). Orca related code is removed from the GPMemoryProtect_TrackStartupMemory().
Problem: Orca's code was strongly coupled with the standard_planner() code. It introduced difficulties for moving Orca's functionality into a shared lib. This patch: 1. moves the invocation of Orca planner out from the standard_planner() into a hook in gp_orca shared lib; 2. moves the init procedure of Orca to the first call of the hook (thus the init is done when the backend has already been initialized including its memory protection means); 3. moves the deinit procedure of Orca into a callback, that is registered at the init (thus the deinit is done right before ShutdownPostgres() is called, where it was before). 4. adds clang-format config file
Problem: Explain component had compile-time dependencies on Orca. It introduced difficulties for moving Orca's functionality into a shared lib. This patch: 1. Adds explain hook implementation to the gp_orca extension. It handles DXL output format of explain command. 2. Removes all Orca related code from explain component in the GPDB core. 3. Adds function standard_ExplainOneQuery() that now contains the default explain code (thus it can be invoked from the hook). 4. Removes 'planGen' field from 'PlannedStmt' structure. Its type was enum, and it had only 2 values: Planner and Optimizer (for Postgres planner and Orca planner respectively). It was not suitable for a general planner plugged in from an extension. 5. Adds 'plannerName' field to 'PlannedStmt' structure. It is now used to output the name of the external planner, if it is used. 6. Updates 'compute_jit_flags()' due to changes above. Now gp_orca extension defines its own implementation for computing JIT parameters. Orca related logic for JIT parameters is removed from the core. 7. Adds new ABI ignore file, as the 'PlannedStmt' structure is updated.
…ce (#1126) Problem: Functions in gp_optimizer_functions.c had a compile-time dependency on Orca. It introduced difficulties for moving Orca's functionality into a shared lib. This patch: 1. Moves implementation of 'DisableXform()', 'EnableXform()', 'LibraryVersion()' into gp_orca shared lib. 2. Updates implementation of wrapper functions 'enable_xform()', 'disable_xform()', 'gp_opt_version()'. Now they try to load the underlying functions from a shared lib. Attempt to load a symbol from a shared lib is wrapped into PG_TRY & PG_CATCH (with all necessary shenanigans), as the lib may be missing. In this case we catch the error, and show an appropriate message. 3. Updates tests due to change of the message if Orca is not enabled. Note: we do not move the wrapper's implementation into gp_orca extension, as they are already a part of the system catalog. And we'd like to avoid changing the system catalog.
Problem: File 'guc_gp.c' contained a compile-time dependency on Orca, related to 'optimizer' GUC. It prevented moving of all Orca's code into a shared lib and making its work transparent to the GPDB core. This patch: 1. Sets the default value for 'optimizer' GUC to 'false'. 2. Adds enabling of 'optimizer' GUC in gp_orca extension shared lib. 3. Updates check in 'check_optimizer()' function. Now it utilizes runtime check of planner_hook presence to distinguish if an external planner is available or not. Note: the semantic of 'optimizer' GUC has changed. Previously it was used solely to enable & disable Orca. Now it is used to enable & disable any general external planner. The name of GUC hasn't been changed, because too many places in code and infrastructure rely on it, and we aim not to blow up the size of the patch.
Decouple ORCA from the pg_hint_plan extension. This change moves the definition of plan_hint_hook from ORCA, which is going to be an extension to the planner. Although this hook is currently used only in ORCA, it makes it possible to use it with any planner that needs to get a hint list from an extension, such as pg_hint_plan. Additionally, we make this hook, plan_hint_hook, typed and return a pointer to HintState, as it actually does. There is no reason to keep it generic and returning void *.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TDB. Currently this PR is for CI only to test accumulated changes in the feature branch.