profiling support?

[mwotton]

I'd like to know where the solver is spending its time. Is there an idiomatic way to detect when you've set the solver up in an inefficient way? I tried jfr, but this doesn't seem to tell me anything about my code in particular, just GC events.

 Event Type                              Count  Size (bytes)
=============================================================
 jdk.GCPhaseParallel                     18323        464783
 jdk.ObjectAllocationSample               3372         56420
 jdk.NativeMethodSample                   2309         29146
 jdk.ModuleExport                         1742         19021
 jdk.TenuringDistribution                  645          7495
 jdk.ThreadPark                            558         21462
 jdk.SystemProcess                         549         62376
 jdk.BooleanFlag                           516         15706
 jdk.Checkpoint                            436       1641346
 jdk.ActiveSetting                         343          9904
 jdk.ExecutionSample                       302          3857
 jdk.JavaErrorThrow                        273         55571

[triceo]

For profiling the solver, I typically use a tailored JFR profile, see attached.

However, without deep understanding of how the solver works under the hood, profiling is unlikely to give you any answers. You'd benefit from profiling if you had a piece of custom code somewhere that would be very resource-intensive, because you'd be easily able to see that.

But profiling Constraint Streams, for example, is pointless without understanding the details of tuple propagation through the node network.

[mwotton]

how do you go about debugging performance problems at that Constraint Streams level? Obviously I'd like to understand it at a deeper level, but usually i find it's useful to have some feedback from a running program, rather than just thinking really hard about what the performance should be (especially about a framework I don't understand really deeply).

My only idea at this stage is to put some counters into each stage of my pipeline so i can at least get a feel for how often each piece is getting executed.

[triceo]

especially about a framework I don't understand really deeply

I think this is the biggest problem here - Constraint Streams are similar in operation to RETE, and the performance characteristics can only be understood with understanding of the underlying engine. It becomes even more complicated by the fact that each node (e.g. join or groupBy) has built-in performance improvements for specific cases, or it sometimes suffers in some pathological corner-cases.

For that reason, there is simply no one way how I could describe CS performance. CS performance is always dependent on how the constraint is written and how much data flows through it. (So are CS bugs, by the way.) This makes debugging and profiling CS a highly specialized task which, unfortunately, cannot be easily taught.

One day, I'd like to find the time to write down the basics of it. Until that time, from the user's perspective, I have the following recommendations:

• Only worry about performance of CS if you think speed is your problem; CS is so fast these days that raw engine performance is often not the problem. Instead, score traps or lack of move variety is often the true problem.
• If CS performance is really the problem, some useful tips can be found in the documentation.
• Test constraints in isolation. Use trial and error to isolate the most time-consuming constraint, fix the constraint and repeat with the next. If you're lost, bring the constraint to us, maybe we spot something.
• Avoid any non-trivial computation inside constraints. Avoid collections, avoid iteration. If it's not incremental, it doesn't scale.
• If you can precompute something, do it. Often people compute in constraints what is actually a static, unchanging information which could've been cached on the entities or on the solution.

[triceo]

On StackOverflow, I recently answered a question regarding constraint ordering. Although unrelated to your problem @mwotton, I go a bit into detail on how constraints operate under the hood.

This may help you understand why profiling and debugging constraints is hard.

[ulrich-grossmann]

I very much recommend JProfiler (commercial). The memory view especially the hotspots or flame graph view will show you at a glance which are the most inefficient parts of your code and which constraints are really worth to improve.