Wire Neo4j heuristics into the search fitness function by andyfelder16 · Pull Request #1731 · WebFuzzing/EvoMaster

andyfelder16 · 2026-09-01T13:15:18Z

Last of the PRs splitting #1641, on top of #1580, #1672, #1647, #1697 and #1715, all of which are merged. It closes the chain: the Cypher queries the SUT runs have been captured since #1460, and this is what finally turns them into search guidance.

Neo4jHandler collects the queries intercepted from Session.run, reads the graph once per action rather than once per query, and scores each query against that snapshot. Only MATCH queries are scored; anything that does not parse as one, such as a write, is skipped
Neo4jCommandWithDistance and Neo4jDistanceWithMetrics carry the result to the DTO
EnterpriseFitness.handleNeo4jHeuristics feeds the distances to the search, behind the new heuristicsForNeo4j option, and reports evaluation counts and the average number of nodes inspected through Statistics
Nothing throws back at the SUT: a driver that cannot be queried, a query that does not parse, and a scoring failure each degrade to no heuristic or to a failed evaluation, and the search carries on
5 handler tests plus one in StatisticsTest, covering the scored path, a write being skipped, no driver, heuristics disabled, and a driver that cannot be reached

# Conflicts: # core/src/main/kotlin/org/evomaster/core/search/service/Statistics.kt

…b and cassandra heuristics

… from its result types

…ration

arcuri82 · 2026-09-03T10:06:51Z

+     * @param evaluationFailure whether the evaluation failed
+     */
+    public Neo4jDistanceWithMetrics(double distance, int numberOfEvaluatedNodes, boolean evaluationFailure) {
+        if (distance < 0.0d || distance > 1.0d || Double.isNaN(distance)) {


hi @andyfelder16 .thx ;) but should first request @jgaleotti review, and, then, once that is completed, JP asks for my review. anyway, with a quick look here i can see the naming is confusing. see DistanceHelper for a definition of distance vs heuristic inside EM codebase

Hi @arcuri82 , I am taking over the review. However, this is indeed a distance since it follows the same model as MongoDistanceWithMetrics, RedisDistanceWithMetrics, DynamoDbDistanceWithMetrics and CqlDistanceWithMetrics (i.e., Cassandra)

if this is distance, why are we throwing an exception if greater than 1? this is confusing. if it is a normalized distance, then it should be explicitely called that, eg, double normalizedDistance

jgaleotti · 2026-09-03T16:23:39Z

+     * @param evaluationFailure whether the evaluation failed
+     */
+    public Neo4jDistanceWithMetrics(double distance, int numberOfEvaluatedNodes, boolean evaluationFailure) {
+        if (distance < 0.0d || distance > 1.0d || Double.isNaN(distance)) {


Hi @arcuri82 , I am taking over the review. However, this is indeed a distance since it follows the same model as MongoDistanceWithMetrics, RedisDistanceWithMetrics, DynamoDbDistanceWithMetrics and CqlDistanceWithMetrics (i.e., Cassandra)

jgaleotti · 2026-09-03T22:03:35Z

+                metrics = new Neo4jDistanceWithMetrics(distance, graph.nodeCount(), false);
+            } catch (Exception e) {
+                SimpleLogger.uniqueWarn("Failed to compute Neo4j heuristic for query: " + query);
+                metrics = new Neo4jDistanceWithMetrics(1.0, graph.nodeCount(), true);


replace 1.0 with Neo4HeuristicsCalculator.MAX_NEO4J_DISTANCE = 1.0d

the value of MAX_NEO4J_DISTANCE should be Double.MAX_VALUE instead of 1.0d

jgaleotti · 2026-09-03T22:10:14Z

+                .toList()
+
+            if (toMinimize.isNotEmpty()) {
+                fv.setExtraToMinimize(i, toMinimize)


this code is behind master

jgaleotti · 2026-09-03T22:11:45Z

@andyfelder16 please update branch and fix conflicts

…ration

… to evaluate

arcuri82 · 2026-09-08T11:58:52Z

+     * @param evaluationFailure whether the evaluation failed
+     */
+    public Neo4jDistanceWithMetrics(double distance, int numberOfEvaluatedNodes, boolean evaluationFailure) {
+        if (distance < 0.0d || distance > 1.0d || Double.isNaN(distance)) {


if this is distance, why are we throwing an exception if greater than 1? this is confusing. if it is a normalized distance, then it should be explicitely called that, eg, double normalizedDistance

arcuri82 · 2026-09-08T11:59:47Z

+     */
+    public Neo4jDistanceWithMetrics(double distance, int numberOfEvaluatedNodes, boolean evaluationFailure) {
+        if (distance < 0.0d || distance > 1.0d || Double.isNaN(distance)) {
+            throw new IllegalArgumentException("distance must be between 0 and 1, but was " + distance);


a distance does not need to be within 0 and 1. if it is a normalized distance, then explicitely state so, eg, a normalized distance must be between 0 and 1,

arcuri82 · 2026-09-08T12:00:32Z

+    /**
+     * Creates a Neo4j heuristic result.
+     *
+     * @param distance normalized distance to satisfying the query, 0 meaning satisfied


any specific reason why Neo4j works on normalized distances?

arcuri82 · 2026-09-08T12:01:09Z

+    /**
+     * @return normalized distance to satisfying the query, 0 meaning satisfied
+     */
+    public double getDistance() {


if normalized, then the name getDistance is confusing.

arcuri82 · 2026-09-08T12:02:46Z

+                    try {
+                        neo4jHandler.handle(it);
+                    } catch (Exception e){
+                        SimpleLogger.error("FAILED TO HANDLE NEO4J COMMAND");


could add the e.getMessage to the this error log

arcuri82 · 2026-09-08T12:08:15Z

+     * cannot compute a distance (for example when the evaluation throws) reports this value, so a failure
+     * never looks closer to satisfied than a genuine miss.
+     */
+    public static final double MAX_NEO4J_DISTANCE = 1.0d;


why does Neo4J require this bound, where all other systems do not? note that normalizing distances do lose information. that is why in DistanceHelper there are a lot of different helper functions to deal then a maxed distance is Double.MAX_VALUE

arcuri82

approved by mistake, there are comments to address first

jgaleotti · 2026-09-08T12:31:45Z

hi @arcuri82 , the term normalized came from the fact that the Neo4JHeuristicsCalculator returns a Truthness that is in the [0..1] range. The distance is then calculated as (1.0d - truthness.ofTrue), which returns a value between [0..1].
Should we simply drop the normalized term and checks for Neo4jDistanceWithMetrics?

arcuri82 · 2026-09-08T12:59:43Z

@jgaleotti what do you mean with checks for Neo4jDistanceWithMetrics??

arcuri82 · 2026-09-08T13:02:05Z

@jgaleotti if the distance is computed from (1.0d - truthness.ofTrue), then it is fine. but still should remove any check on >1, or, alternative, explicitely state in the variable names that those are normalized within 1. otherwise it is quite confusing with the rest of the codebase

jgaleotti · 2026-09-08T13:14:22Z

please @andyfelder16 remove the checks that constrain the Neo4J distance to be <1. Also, replace 1.0d with Double.MAX_VALUE as the MAX_NEO4J_DISTANCE

…1 check on the distance

andyfelder16 added 8 commits July 21, 2026 09:48

wire neo4j heuristics into the search fitness function

8f5e81c

Merge branch 'neo4j-graph-reader' into neo4j-fitness-integration

5e69c02

Merge branch 'neo4j-graph-reader' into neo4j-fitness-integration

ce64642

Merge branch 'neo4j-graph-reader' into neo4j-fitness-integration

edce4cc

# Conflicts: # core/src/main/kotlin/org/evomaster/core/search/service/Statistics.kt

Merge branch 'neo4j-graph-reader' into neo4j-fitness-integration

8929d2d

merge master into the neo4j fitness integration alongside the dynamod…

1fb8dc6

…b and cassandra heuristics

document the neo4j heuristics handler and drop the redundant prefixes…

f67a9f1

… from its result types

Merge remote-tracking branch 'origin/master' into neo4j-fitness-integ…

ce624d2

…ration

andyfelder16 marked this pull request as ready for review September 1, 2026 14:59

andyfelder16 requested review from arcuri82 and jgaleotti September 1, 2026 14:59

arcuri82 requested changes Sep 3, 2026

View reviewed changes

jgaleotti requested changes Sep 3, 2026

View reviewed changes

andyfelder16 added 2 commits September 4, 2026 18:07

Merge remote-tracking branch 'origin/master' into neo4j-fitness-integ…

9cb092f

…ration

report the calculator's maximum distance when a neo4j heuristic fails…

15974b1

… to evaluate

andyfelder16 requested a review from jgaleotti September 5, 2026 22:05

jgaleotti approved these changes Sep 7, 2026

View reviewed changes

jgaleotti requested a review from arcuri82 September 7, 2026 16:11

arcuri82 approved these changes Sep 8, 2026

View reviewed changes

arcuri82 requested changes Sep 8, 2026

View reviewed changes

a failed neo4j heuristic now reports Double.MAX_VALUE, dropping the >…

1bfc0e3

…1 check on the distance

andyfelder16 requested a review from jgaleotti September 8, 2026 13:29

jgaleotti approved these changes Sep 8, 2026

View reviewed changes

jgaleotti requested a review from arcuri82 September 8, 2026 13:48

arcuri82 approved these changes Sep 9, 2026

View reviewed changes

arcuri82 merged commit 7022032 into master Sep 9, 2026
53 of 54 checks passed

arcuri82 deleted the neo4j-fitness-integration branch September 9, 2026 10:51

Uh oh!

Conversation

andyfelder16 commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jgaleotti Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jgaleotti commented Sep 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arcuri82 left a comment

Choose a reason for hiding this comment

Uh oh!

jgaleotti commented Sep 8, 2026

Uh oh!

arcuri82 commented Sep 8, 2026

Uh oh!

arcuri82 commented Sep 8, 2026

Uh oh!

jgaleotti commented Sep 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

andyfelder16 commented Sep 1, 2026 •

edited

Loading

jgaleotti Sep 8, 2026 •

edited

Loading