go-esi

go-esi is the implementation of the non-standard ESI (Edge-Side-Include) specification from the w3. With that you'll be able to use the ESI tags and process them in your favorite golang servers.

What are the ESI tags

The ESI tags were introduced by Akamai to add some dynamic tags and only re-render these parts on the server-side. The goal of that is to render only specific parts. For example, we want to render a full e-commerce webpage but only the cart is user-dependent. So we could render the "static" parts and store with a predefined TTL (e.g. 60 minutes), and only the cart would be requested to render the block.

There are multiple esi tags that we can use but the most used is the esi:include because that's the one to request another resource.

We can have many esi:include tags in a single response, and each esi:include tags can itself have one or more esi:include tags.

We can have multiple esi:include tags in the page to request another resource and add its content to the main page.

References

https://www.w3.org/TR/esi-lang/

Install

go get -u github.com/darkweak/go-esi

Usage

Parse a whole document

import (
    // ...
    "github.com/darkweak/go-esi/esi"
)

//...

func functionToParseESITags(b []byte, r *http.Request) []byte {
    // Returns the parsed response. The include tags are fetched in parallel.
    res := esi.Parse(b, r)

    //...
    return res
}

Stream a document

esi.Stream processes the ESI tags while the document is written to it. The static content is forwarded as soon as it is read, the include tags are fetched concurrently, and their content is written in the document order. The destination is flushed (if it implements http.Flusher) whenever the output has to wait for an include or for more input.

func streamESI(dst io.Writer, src io.Reader, r *http.Request) error {
    s := esi.NewStream(dst, r)
    if _, err := io.Copy(s, src); err != nil {
        _ = s.Close()

        return err
    }

    // Close processes the remaining content and waits for the pending includes.
    return s.Close()
}

Wrap an http.ResponseWriter

import "github.com/darkweak/go-esi/writer"

func esiMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
        cw := writer.NewWriter(rw, r)
        next.ServeHTTP(cw, r)
        _ = cw.Close()
    })
}

Resolve relative includes against another host

By default, a relative include (e.g. <esi:include src="/fragment"/>) is fetched from the host of the incoming request. writer.WithBaseURL resolves them against another origin instead, e.g. an internal service. The path of the incoming request is kept, and the Cookie and Authorization headers are forwarded to that origin as it becomes the same-origin target. Absolute includes are not affected.

// The scheme is optional and defaults to http.
base, err := writer.ParseBaseURL("static-webserver.my-app.svc.cluster.local")
if err != nil {
    return err
}

cw := writer.NewWriter(rw, r, writer.WithBaseURL(base))

Every middleware exposes it as a default host setting, see below.

Available as middleware

• Caddy
• Træfik
• Roadrunner

Caddy middleware

xcaddy build --with github.com/darkweak/go-esi/middleware/caddy

Refer to the sample Caddyfile to know how to use that.

To resolve the relative includes against another host:

route /* {
    esi {
        default_host backend:8080
    }
    reverse_proxy backend:8080
}

Roadrunner middleware

To use the go-esi processor as Roadrunner middleware, you just have to follow the steps below.
You have to build your rr binary with the go-esi dependency.

[velox]
build_args = ['-trimpath', '-ldflags', '-s -X github.com/roadrunner-server/roadrunner/v2/internal/meta.version=v2.12.0 -X github.com/roadrunner-server/roadrunner/v2/internal/meta.buildTime=10:00:00']

[roadrunner]
ref = "v2.12.3"

[github]
    [github.token]
    token = "GH_TOKEN"

    [github.plugins]
    logger = { ref = "v3.2.0", owner = "roadrunner-server", repository = "logger" }
    esi = { ref = "master", owner = "darkweak", repository = "go-esi", folder = "middleware/roadrunner", replace = "/opt/middleware/roadrunner" }
    server = { ref = "v3.2.0", owner = "roadrunner-server", repository = "server" }
    gzip = { ref = "v3.2.0", owner = "roadrunner-server", repository = "gzip" }
    http = { ref = "v3.2.0", owner = "roadrunner-server", repository = "http" }

[log]
level = "debug"
mode = "development"

After that, you'll be able to set enable and add the esi processor to the middleware chain.

# .rr.yaml
http:
  # Other http sub keys
  esi: {}
  # Or, to resolve the relative includes against another host:
  # esi:
  #   default_host: backend:8080
  middleware:
    - headers
    - gzip
    - esi

Træfik middleware

# anywhere/traefik.yml
experimental:
  plugins:
    souin:
      moduleName: github.com/darkweak/go-esi
      version: v0.0.7

# anywhere/dynamic-configuration
http:
  routers:
    whoami:
      middlewares:
        - esi
      service: whoami
      rule: Host(`domain.com`)
  middlewares:
    esi:
      plugin:
        esi: {}

To resolve the relative includes against another host (e.g. a Kubernetes service), set defaultHost:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: esi
  namespace: my-app
spec:
  plugin:
    esi:
      defaultHost: static-webserver.my-app.svc.cluster.local # or http(s)://host:port

Refer to the sample traefik file to know how to use that.

TODO

• choose tag
• comment tag
• escape tag
• include tag
• remove tag
• otherwise tag
• try tag
• vars tag
• when tag