BMv2-PSA: Header stack turns into single headers in the pipeline config JSON

[kirazero17]

I have a header stack defined this way:

header msg_t {
    chunk_t msg;
}

struct hdr_structures_t {
[...]
    msg_t[12] Msg;
[...]
}

In my control block, I called the pop_front() method:

control sw_ingress_control (
    inout hdr_structures_t hdr, inout metadata_t meta, 
    in    psa_ingress_input_metadata_t  std_ingress_input_meta,
    inout psa_ingress_output_metadata_t std_ingress_output_meta
)
{
[...]
    hdr.Msg.pop_front(1);
[...]
}

However, when I load the compiled P4 config JSON into the PSA BMv2 switch, the logs shows these two line, which I believe to be related:

Invalid reference to object of type 'header stack' with name 'Msg'
psa_switch_grpc: /home/p4/src/behavioral-model/include/bm/bm_sim/data.h:185: unsigned int bm::Data::get_uint() const: Assertion `arith' failed.

With the Invalid reference to object line, I have checked the compiled JSON file for the pipeline config. This is what I found:

1. There was no header stack at all:

[...]
"header_stacks" : [],
[...]

2. The stack turned out to be converted to single headers:

{
[...]
      "name" : "Msg[0]",
      "id" : 7,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[1]",
      "id" : 8,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[2]",
      "id" : 9,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[3]",
      "id" : 10,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[4]",
      "id" : 11,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[5]",
      "id" : 12,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[6]",
      "id" : 13,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[7]",
      "id" : 14,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[8]",
      "id" : 15,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[9]",
      "id" : 16,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[10]",
      "id" : 17,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
    {
      "name" : "Msg[11]",
      "id" : 18,
      "header_type" : "msg_t",
      "metadata" : false,
      "pi_omit" : true
    },
[...]

Since there is no header stack, it can't be referred to. Is this a compiler bug ?

Thanks.

[kirazero17]

To be more specific, I'm using version 1.2.5.16

[jafingerhut]

I have not done any investigation to determine whether this is a bug, but I would not be surprised at all if it was. Note that the PSA support in p4c for bmv2/behavioral-model target is not nearly complete, unfortunately, so bugs are not hard to find there. The PSA support for the DPDK and/or EBPF back ends might be better, but I have not used them enough personally to know how complete they are.

If you are willing to try adapting your program for the v1model architecture, p4c's support for that is much more complete and well tested.

[aeron-gh]

Yes, this is a compiler bug, and it is not specific to your program.

The BMv2 PSA and PNA back ends never emit header_stacks. The code that would do it is still commented out:

p4c/backends/bmv2/portable_common/portable.cpp

Lines 164 to 167 in 4f5713b

	/* TODO */
	// for (auto kv : header_stacks) {
	// json->add_header_stack(stack_type, stack_name, stack_size, ids);
	// }

Only v1model implements it, in HeaderConverter::addHeaderStacks:

p4c/backends/bmv2/common/header.cpp

Lines 128 to 165 in 4f5713b

	void HeaderConverter::addHeaderStacks(const IR::Type_Struct *headersStruct) {
	LOG2("Creating stack " << headersStruct);
	for (auto f : headersStruct->fields) {
	auto ft = ctxt->typeMap->getType(f, true);
	auto stack = ft->to<IR::Type_Array>();
	if (stack == nullptr) continue;
	auto stack_name = f->controlPlaneName();
	auto stack_size = stack->getSize();
	auto type = ctxt->typeMap->getTypeType(stack->elementType, true);
	bool isUnion = type->is<IR::Type_HeaderUnion>();
	if (type->is<IR::Type_Header>() || isUnion) {
	auto ht = type->to<IR::Type_StructLike>();
	addHeaderType(ht);
	cstring stack_type = ht->controlPlaneName();
	std::vector<unsigned> ids;
	for (unsigned i = 0; i < stack_size; i++) {
	cstring hdrName = f->controlPlaneName() + "[" + Util::toString(i) + "]";
	int id;
	if (isUnion) {
	// We have to add separately a header instance for all
	// headers in the union. Each instance will be named with
	// a prefix including the union name, e.g., "u.h"
	auto fields = addHeaderUnionFields(hdrName, type->to<IR::Type_HeaderUnion>());
	id = ctxt->json->add_union(stack_type, fields, hdrName);
	} else {
	id = ctxt->json->add_header(stack_type, hdrName);
	}
	ids.push_back(id);
	}
	if (isUnion)
	ctxt->json->add_header_union_stack(stack_type, stack_name, stack_size, ids);
	else
	ctxt->json->add_header_stack(stack_type, stack_name, stack_size, ids);
	} else {
	BUG("%1%: unexpected type in stack", type);
	}
	}
	}

and HeaderConverter is constructed only by simple_switch. psa_switch and pna_nic both call PortableCodeGenerator::createHeaders instead, so both are affected.

Reproduced without your program, on a file already in the tree. testdata/p4_16_samples/psa-header-stack.p4 declares vlan_tag_h[2] vlan_tag:

$ p4c-bm2-psa testdata/p4_16_samples/psa-header-stack.p4 -o out.json
"header_stacks" : []
headers: ... "vlan_tag[0]", "vlan_tag[1]"

Same shape you are getting.

The two log lines are connected. The declaration side is missing, but the use side still emits stack references, so the JSON contradicts itself. Adding pop_front(1) to that same program gives:

"op" : "pop",
"parameters" : [
  { "type" : "header_stack", "value" : "vlan_tag" },
  { "type" : "hexstr", "value" : "0x1" }
]

That is a reference to a header_stack the file never declares, which is exactly the Invalid reference to object of type 'header stack' line. The get_uint assert right after looks downstream of it.

Worth flagging that this is wider than pop_front. The parser has the same problem, because extract(h.vlan_tag.next) in that sample emits:

"parameters" : [ { "type" : "stack", "value" : "vlan_tag" } ],
"op" : "extract"

So any PSA program that parses into a header stack produces JSON BMv2 cannot load, whether or not it ever calls push or pop. The push/pop and extract conversions live in the shared code, which is why the use side works while the declaration side does not:

p4c/backends/bmv2/common/action.cpp

Lines 195 to 204 in 4f5713b

	} else if (builtin->name == IR::Type_Array::push_front) {
	BUG_CHECK(mc->arguments->size() == 1, "Expected 1 argument for %1%", mc);
	auto arg = ctxt->conv->convert(mc->arguments->at(0)->expression);
	prim = "push"_cs;
	parameters->append(arg);
	} else if (builtin->name == IR::Type_Array::pop_front) {
	BUG_CHECK(mc->arguments->size() == 1, "Expected 1 argument for %1%", mc);
	auto arg = ctxt->conv->convert(mc->arguments->at(0)->expression);
	prim = "pop"_cs;
	parameters->append(arg);

The TODO goes back to the original PSA support in #1298 (Jun 2018), and pna_nic inherited it from the same code in #4729. There is also no committed BMv2 JSON reference for any sample, so nothing in the test suite reads this output, which is probably why it sat this long.

No workaround I can offer other than avoiding header stacks on PSA for now. Constant indexing like h.vlan_tag[0].f does work, since the elements are present as flat headers, but anything that needs the stack object itself does not.

[fruffy]

I have not done any investigation to determine whether this is a bug, but I would not be surprised at all if it was. Note that the PSA support in p4c for bmv2/behavioral-model target is not nearly complete, unfortunately, so bugs are not hard to find there. The PSA support for the DPDK and/or EBPF back ends might be better, but I have not used them enough personally to know how complete they are.

There is a question here whether we should continue the work on PSA support in BMv2 or drop it. I think BMv2 is a good target to explore, both for PNA/PSA. DPDK is much harder to set up.

[kirazero17]

I have not done any investigation to determine whether this is a bug, but I would not be surprised at all if it was. Note that the PSA support in p4c for bmv2/behavioral-model target is not nearly complete, unfortunately, so bugs are not hard to find there. The PSA support for the DPDK and/or EBPF back ends might be better, but I have not used them enough personally to know how complete they are.

There is a question here whether we should continue the work on PSA support in BMv2 or drop it. I think BMv2 is a good target to explore, both for PNA/PSA. DPDK is much harder to set up.

I actually agree with @fruffy . BMv2 can be used to test the functionality and behaviours before moving to hardware targets. Also, I've made a pull request (#5773 ) as a proposal for what can be implemented for the header stacks on PSA-BMV2. I haven't done the same for header union stacks, tho.