You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently, archipelago evolution has a bottleneck since selection and migration policies are executed in the main thread. This is barely noticeable for the simple default selection and replacement policies, as they are very fast. More sophisticated strategies with a non-negligible overhead deployed to larger archipelagos (>100 islands) constitute a scalability problem though.
Describe the solution you'd like
To improve scalability, execution of selection and replacement policies should be moved inside the remote process/thread, i.e. handled on island level and not on archipelago level.
Describe alternatives you've considered
Depending on the selection and replacement logic, some of it could be moved as a workaround inside the evolve-method of the UDA, which is parallelized (i.e. flagging certain individuals during evolution so that the selection just needs to pick them up). However, this is not a very clean solution (breaks the semantics of evolve) and might not work in all cases. Using parallelization (like multi-processing) in user defined policies could maybe work as a speed up in some cases, but would not change the fact that multiple policy exeuctions are still executed back-to-back, producing a potentially large overhead for spawning new threads/processes every time. It would be more efficient and clean to use the already established parallelization structure of the archipelago for this task.
Additional context
This feature request came into existence while discussing if there was an "easy" way to re-evaluate migrants upon entering a new population? Current pagmo behavior copies the fitness vector of an individual over, which is fine as long as the udps of source and target island do not differ. Migrating between islands with "inconsistent" udps however require a reevaluation of individuals, which is doable by a user-defined replacement function, as long as it has access to its islands fitness function. More sophisticated migration strategies (i.e. replacement by "merging" individuals and migrants) might be another use-case where a fitness evaluation within a policy could be desirable.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Currently, archipelago evolution has a bottleneck since selection and migration policies are executed in the main thread. This is barely noticeable for the simple default selection and replacement policies, as they are very fast. More sophisticated strategies with a non-negligible overhead deployed to larger archipelagos (>100 islands) constitute a scalability problem though.
Describe the solution you'd like
To improve scalability, execution of selection and replacement policies should be moved inside the remote process/thread, i.e. handled on island level and not on archipelago level.
Describe alternatives you've considered
Depending on the selection and replacement logic, some of it could be moved as a workaround inside the evolve-method of the UDA, which is parallelized (i.e. flagging certain individuals during evolution so that the selection just needs to pick them up). However, this is not a very clean solution (breaks the semantics of evolve) and might not work in all cases. Using parallelization (like multi-processing) in user defined policies could maybe work as a speed up in some cases, but would not change the fact that multiple policy exeuctions are still executed back-to-back, producing a potentially large overhead for spawning new threads/processes every time. It would be more efficient and clean to use the already established parallelization structure of the archipelago for this task.
Additional context
This feature request came into existence while discussing if there was an "easy" way to re-evaluate migrants upon entering a new population? Current pagmo behavior copies the fitness vector of an individual over, which is fine as long as the udps of source and target island do not differ. Migrating between islands with "inconsistent" udps however require a reevaluation of individuals, which is doable by a user-defined replacement function, as long as it has access to its islands fitness function. More sophisticated migration strategies (i.e. replacement by "merging" individuals and migrants) might be another use-case where a fitness evaluation within a policy could be desirable.
The text was updated successfully, but these errors were encountered: