Arazzo Sampler V1 by daniellopera15 · Pull Request #1682 · WebFuzzing/EvoMaster

daniellopera15 · 2026-08-15T02:00:09Z

No description provided.

jgaleotti · 2026-08-17T20:10:17Z

+import org.evomaster.core.search.service.mutator.StandardMutator
+import org.evomaster.core.search.service.mutator.StructureMutator
+
+class ArazzoRestModule(private val bindRemote: Boolean = true) : RestBaseModule() {


Add @JavaDoc of this class

After a more careful consideration, this class should be deleted.

jgaleotti · 2026-08-17T21:07:03Z

+import org.evomaster.core.search.service.mutator.MutatedGeneSpecification
+import org.evomaster.core.sql.SqlInsertBuilder
+
+class ArazzoStructureMutator : ApiWsStructureMutator() {


I am not sure if we need a specific mutator for Arazzo? I think we need to discuss this with Andrea

We can discuss it. However, the MIO algorithm needs to use the mutator. We could use an existing mutator, but that might break the workflow logic.

jgaleotti · 2026-08-17T21:12:32Z


 import com.google.inject.Inject
+import com.webfuzzing.arazzo.access.ArazzoAccess
+import com.webfuzzing.arazzo.models.domain.Workflow


in a future PR, rename Workflow to ArazzoWorkflow

add as a //TODO to rename it to ArazzoWorkflow

jgaleotti · 2026-08-17T21:13:15Z

    lateinit var skippedEndpoints : List<Endpoint>
        private set

+    var workflowsArazzo = mutableListOf<Workflow>()


rename field to arazzoWorkflows

jgaleotti · 2026-08-17T21:13:30Z

+    var workflowsArazzo = mutableListOf<Workflow>()
+        private set
+
+    lateinit var workflowsArazzoById: Map<String, Workflow>


rename field to to arazzoWorkflowsById

jgaleotti · 2026-08-17T21:32:04Z

+            return listOfNotNull(findActionForOperation(step.operationId))
+        }
+        if (!step.workflowId.isNullOrBlank()) {
+            val nested = workflowsArazzoById[step.workflowId] ?: return emptyList()


but it is expected for the workflowId to be missing? Shouldn't this signal an exception?

Yes, it is better for it to throw an exception. However, the ideal approach would be for the ArazzoParser to perform the validations and guarantee a valid Arazzo document.

jgaleotti · 2026-08-17T21:32:42Z

+    /**
+     * Every Step Arazzo has its corresponding RestCallAction in the actionCluster
+     */
+    private fun findActionForOperation(operationId: String): RestCallAction? {


can operationId be null? If not, please add a Objects.requireNonNull()

I am adding an exception.

jgaleotti · 2026-08-17T23:17:06Z

+            val nested = workflowsArazzoById[step.workflowId] ?: return emptyList()
+            return nested.steps.flatMap { resolveStep(it) }
+        }
+        return emptyList()


I am not sure about this recursive approach, could you change it to a iterative approach to build the RestCallActions?

Done. Refactored to an iterative solution using ArrayDeque.

jgaleotti · 2026-08-17T23:17:39Z

+        val template = actionCluster.values
+            .filterIsInstance<RestCallAction>()
+            .find { it.operationId == operationId }
+        return template?.copy() as? RestCallAction


what happen if no operationId is found? Is it expected?

I am adding an exception.

jgaleotti · 2026-08-17T23:19:19Z

+
+    enum class ArazzoStrategy {
+        NONE,
+        ENABLED


ENABLED is not declarative enough. Please replace with a more declarative name and add a Javadoc to describe this strategy

How about calling it WORKFLOW?

@daniellopera15 was it modified to WORKFLOW? i still see ENABLED here

jgaleotti · 2026-08-25T16:45:41Z

 import javax.annotation.PostConstruct
-import kotlin.sequences.forEach
-



Please revert changes to this file if they are not needed (this seems refactoring such as formatting and unused imports)

jgaleotti · 2026-08-25T16:46:23Z

+
+    enum class ArazzoStrategy {
+        NONE,
+        ENABLED


jgaleotti · 2026-09-02T10:08:49Z

+        <dependency>
+            <groupId>org.evomaster</groupId>
+            <artifactId>arazzo-parser</artifactId>
+            <scope>compile</scope>


why the compiler scope? The parser will be required in the distribution to be able to parse Arazzo specs within the evomaster.jar, right?

The default scope for dependencies is compile. I'm going to remove it.

jgaleotti · 2026-09-02T10:12:35Z

                EMConfig.ProblemType.REST -> {
-                    if (config.blackBox) {
+                    if (config.isEnabledArazzoStrategy()) {
+                        ArazzoRestModule()


I am not sure that an ArazzoRestModule is required. I think Arazzo sampling should be possible to enable in BlackBoxRestModule, ResourceRestModule and RestModule. The Arazzo sampling looks more like a sampling strategy.

That’s a possibility. I’m going to look into making a commit that includes the Sampler but not the Module. That way, we’d avoid the Mutator issue.

jgaleotti · 2026-09-03T23:59:12Z

+import org.evomaster.core.search.service.mutator.StandardMutator
+import org.evomaster.core.search.service.mutator.StructureMutator
+
+class ArazzoRestModule(private val bindRemote: Boolean = true) : RestBaseModule() {


After a more careful consideration, this class should be deleted.

…mpler

…g/EvoMaster into feature/arazzo-sampler

arcuri82 · 2026-09-08T09:54:49Z

+
+    enum class ArazzoStrategy {
+        NONE,
+        ENABLED


@daniellopera15 was it modified to WORKFLOW? i still see ENABLED here

arcuri82 · 2026-09-08T09:55:22Z

+    /**
+     * Enum that enables and disables the generation of individuals based on Arazzo Workflows.
+     */
+    enum class ArazzoStrategy {


if it is only two option on/off, why is this an enum and not a boolean?

or are there already plans to add more in future? in that case, leave a TODO comment

No. At the moment, there are no plans to add more in the future. We're going to switch it to a boolean.

arcuri82 · 2026-09-08T09:57:47Z

+
+    @Experimental
+    @Cfg("Probability of controlling the creation of Arazzo individuals.")
+    @Probability


i guess that should add activating=false, as controlled by arazzoStrategy.

arcuri82 · 2026-09-08T09:58:21Z



+    @Experimental
+    @Cfg("arazzo location in disk")


Arazzo path location on disk

arcuri82 · 2026-09-08T10:00:14Z


+    @Experimental
+    @Cfg("arazzo location in disk")
+    var arazzoLocation: String? = null


can be ="", with ExistingPath(val canBeBlank: Boolean, val shouldBeWritable: Boolean) having true,false

same as overlay option

arcuri82 · 2026-09-08T10:21:04Z

+     */
+    fun load(openAPI: OpenAPI, location: String?) {
+        if (location.isNullOrBlank()) {
+            throw ConfigProblemException("arazzoLocation must not be null when Arazzo strategy is enabled")


Arazzon strategy will be on by default in future, with location empty by default

arcuri82 · 2026-09-08T10:22:02Z

+        }
+        val workflows = readArazzoWorkflows(openAPI, location)
+        if (workflows.isEmpty()) {
+            throw ConfigProblemException("There must be at least one Arazzo workflow.")


in this error message, add info on the location

arcuri82 · 2026-09-08T10:23:12Z

+
+    private fun readArazzoWorkflows(openAPI: OpenAPI, location: String): List<Workflow> {
+        val arazzoText = ArazzoAccess.readFromDisk(location)
+        return ArazzoParser.parse(arazzoText, openAPI).workflows


maybe the parse should be in a try/catch, throwing a ConfigProblemException if any error? eg, what happens if user specify wrong non-Arazzo file by mistake?

arcuri82 · 2026-09-08T10:26:59Z

+     * Choose a random workflow
+     */
+    fun sampleAtRandom(
+        actionCluster: Map<String, Action>,


why is this given as input, and not read directly from an injected field holding ref to the singleton sampler?

Are you talking about doing something like this?

@Inject
private lateinit var sampler: AbstractRestSampler

yes. by the time sampleAtRandom is called, the singletons should all be already initialized

arcuri82 · 2026-09-08T10:29:22Z

+        actionCluster: Map<String, Action>,
+        createIndividual: (SampleType, MutableList<RestCallAction>) -> RestIndividual,
+    ): RestIndividual {
+        val actions = buildArazzoRestCallActions(workflow.steps, actionCluster)


should clarify here in a comment that, for the time being, we are only considering one single OpenAPI schema fro the workflow, and ignoring cases of arazzo spanning different APIs

daniellopera15 added 8 commits August 13, 2026 22:05

ArazzoSampler v1 - Only Sampler

ed80b98

Sampler with test - Not Working

0a157cb

Arazzo Verifier Sampler Test

8c8dfbe

Add new test in ArazzoVerifierSampletTest

ef00cfe

Fix pom version and imports

c55a13b

Add comments in tests ArazzoSamplerVerifierTest

f474d74

Fix merge conflicts

097b64c

Generate Markdown documentacion - Arazzo

6b50305

daniellopera15 requested a review from jgaleotti August 15, 2026 02:00

jgaleotti requested changes Aug 17, 2026

View reviewed changes

daniellopera15 added 3 commits August 18, 2026 22:59

Arazzo Sampler: Review refactor v1

1b269ad

Arazzo Sampler: Review refactor v2

48c6e94

ArazzoSampler: Review refactor v3

e779e61

daniellopera15 requested a review from jgaleotti August 23, 2026 22:32

jgaleotti requested changes Aug 25, 2026

View reviewed changes

daniellopera15 and others added 2 commits September 1, 2026 14:15

ArazzoSampler: Review refactor v4

3b7a033

Merge branch 'master' into feature/arazzo-sampler

c889646

daniellopera15 requested a review from jgaleotti September 1, 2026 21:16

jgaleotti requested changes Sep 4, 2026

View reviewed changes

daniellopera15 added 3 commits September 5, 2026 00:05

Merge remote-tracking branch 'upstream/master' into feature/arazzo-sa…

2fc861f

…mpler

ArazzoSampler: Review refactor v5

24d3d0e

Merge branch 'feature/arazzo-sampler' of https://github.com/WebFuzzin…

11e9db0

…g/EvoMaster into feature/arazzo-sampler

daniellopera15 requested a review from jgaleotti September 5, 2026 04:26

jgaleotti requested changes Sep 5, 2026

View reviewed changes

ArazzoSampler: Review refacor v6

6daa628

daniellopera15 requested a review from jgaleotti September 6, 2026 22:36

jgaleotti approved these changes Sep 7, 2026

View reviewed changes

jgaleotti requested a review from arcuri82 September 7, 2026 14:59

arcuri82 requested changes Sep 8, 2026

View reviewed changes

ArazzoSampler: Review refactor v7

d708715

ArazzoSampler: Review refactor v8

2525dc1

daniellopera15 requested a review from arcuri82 September 10, 2026 03:51

		import javax.annotation.PostConstruct
		import kotlin.sequences.forEach

Uh oh!

Conversation

daniellopera15 commented Aug 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment