Skip to content

Modularization: avro module #115

Open

Description

@fabiojose

This module implements the Avro Event Format.

  • avro

This module depend on api one.

When someone wants the avro format to create or read events, they add this module and they concern about potential conflicts with pre-existing dependencies in their projects.

Activity

  1. subanasif commented on Dec 11, 2020

    @subanasif

    @slinkydeveloper: I do have implementation of EventFormat interface for avro. However, the implementation class has a setter method for Avro schema since this not defined in EventFormat interface. Hence, following code would be used to get hold of format and serialize.

    AvroFormat avroFormat = (AvroFormat) EventFormatProvider
            .getInstance()
            .resolveFormat(AvroFormat.CONTENT_TYPE);
    //Set avro schema
    avroFormat.setSchema(schema)
    

    Let me know your thoughts.

  2. slinkydeveloper commented on Dec 11, 2020

    @slinkydeveloper
    Member

    avroFormat.setSchema(schema)

    Which schema?

  3. subanasif commented on Dec 11, 2020

    @subanasif

    This is avro schema. That is org.apache.avro.Schema.
    This could be coming from filesystem or remote repository like schema registry.

  4. slinkydeveloper commented on Dec 11, 2020

    @slinkydeveloper
    Member

    avro schema of what? of Cloudevents? The one from the spec https://github.com/cloudevents/spec/blob/v1.0/spec.avsc?

  5. subanasif commented on Dec 11, 2020

    @subanasif

    Thats correct. It *.avsc schema

  6. slinkydeveloper commented on Dec 11, 2020

    @slinkydeveloper
    Member

    Shouldn't this be loaded statically? Can we ship this schema as a resource in the package and then load it statically?

  7. subanasif commented on Dec 11, 2020

    @subanasif

    The schema at https://github.com/cloudevents/spec/blob/v1.0/spec.avsc is just a spec, whereas real schema would have much more details (in data part, with specific field definitions).
    Hence loading statically as resource won't work. More often, these spec compliant schemas are stored at third-party registries.

  8. slinkydeveloper commented on Dec 11, 2020

    @slinkydeveloper
    Member

    TBH I'm not an expert of avro, so I can't tell. What I can say is that it sounds weird to me to mutate the internal state of EventFormat in order to provide a custom schema... My expectation is that EventFormat doesn't parse the body, but it just collects it in a raw form (like in json, where it collects the data as json node) and then the user process this raw data into the pojo she/he wants to.
    I think you should check out this PR where they're working on improving the schema cloudevents/spec#613

  9. yuce commented on Dec 11, 2020

    @yuce

    @subanasif https://github.com/cloudevents/spec/blob/v1.0/spec.avsc is compiled into Java before shipping, so dynamically loading it is not necessary (although possible). That Avro spec is used by other CE clients, so I don't think it can be changed as long as compatibility with CE 1.0 is desired.

  10. pdebuitlear commented on Jan 24, 2021

    @pdebuitlear

    @subanasif https://github.com/cloudevents/spec/blob/v1.0/spec.avsc is compiled into Java before shipping, so dynamically loading it is not necessary (although possible). That Avro spec is used by other CE clients, so I don't think it can be changed as long as compatibility with CE 1.0 is desired.

    I'm not sure that's what is being suggested.
    @subanasif you mentioned you had an implementation, any chance you could share it for some more context?

  11. pdebuitlear commented on Apr 18, 2021

    @pdebuitlear

    any update on this issue?

  12. alsonlu commented on Jun 28, 2021

    @alsonlu

    Is there any update on if/when this will be released?

  13. slinkydeveloper commented on Jun 29, 2021

    @slinkydeveloper
    Member

    This is just waiting for a contribution to come 😄

  14. sunng87 commented on Jul 16, 2021

    @sunng87

    Just let you know I have been working on an avro serde impl these days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions