CloudEvents Hands-On: What's the latest CNCF Graduation all about?

24.04.2024•Tom Trapp
Cloud Cloud Native Computing Foundation cloudevents Cloud Computing Event-Driven open-source Microservices

Banner

In the second TechUp on CloudEvents, we want to take a closer and hands-on look at the latest CNCF graduation. Of course, you can find the code used on GitHub.

We want to answer the following questions in this TechUp:

  • 🌩️ What is CloudEvents?
  • 🏆 Why is the graduation of CloudEvents a milestone?
  • 🛠️ How can CloudEvents be used in practice?
  • 🔍 How can CloudEvents be tested?
  • 💡 Is it worth using CloudEvents?

CloudEvents?

CloudEvents is an open-source initiative launched by the Serverless Working Group of the CNCF. Work on CloudEvents began in 2017, and since then the initiative has evolved into a significant standard for exchanging events between cloud applications and services.

More on this in Ricky’s CloudEvents TechUp.

Graduation

In January 2024, the Cloud Native Computing Foundation (CNCF) announced that CloudEvents had reached graduation status. This status is awarded to projects that have reached the maturity and adoption to be considered full members of the CNCF community. The graduation of CloudEvents marks an important milestone in the development of the initiative and underscores its widespread acceptance and maturity. Thus, CloudEvents has received the highest seal of approval from the CNCF, and is thus on par with other well-known projects such as Kubernetes, Argo, Cilium, Prometheus and Envoy.

Hands-On

In this TechUp, we want to take a completely hands-on look at CloudEvents and build a small but fully functional application consisting of several microservices that use CloudEvents to communicate with each other.

Let’s first take a look at what SDKs are available:

Quite extensive, isn’t it? We want to focus on Go, Java and JavaScript in this TechUp.

The Goal

We have a simple front end with a button. This button should suggest an activity to us when we are bored.

Technically, the frontend should call a Go service, which makes a request to another Java microservice. This Java microservice then queries the activity API of BoredAPI and returns an activity to us.

To begin with, however, we first want to look at the communication between the services without responses.

Big Picture

img_1.png

Here it is nice to see that the communication between the different microServices takes place via CloudEvents. Of course, the communication between browser and Node.js server could also take place via CloudEvents, but for the sake of simplicity we use “plain” HTTP here.

We have a purely synchronous use case here, we send an event and wait for a response.

One Way Communication

The communication between the services should take place via HTTP and CloudEvents. To do this, we need to include a CloudEvents SDK in each service and process the events accordingly.

First, we want to look at how we can send data in one direction. This will also allow us to build the complete architecture of our application and then implement our use case.

Frontend

In the repository cloud-events-example-frontend you will find a Node.js project that uses Express to start a simple web server. The web server delivers a simple HTML page that contains two buttons. When one of the buttons is pressed, an event should be sent to the Go service.

img_4.png

The frontend project is accessible via http://localhost:3000.

We don’t want to go into the general structure of the frontend project at this point, but let’s take a look directly at the CloudEvents part:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

import {CloudEvent, emitterFor, httpTransport} from "cloudevents";

const emit = emitterFor(httpTransport("http://localhost:8080"));

interface ButtonEvent {
    clicked: boolean;
}
//...

app.post('/button-clicked', (req, res) => {
    console.log('Button was clicked!');
    const ce = new CloudEvent<ButtonEvent>({
        type: 'com.bnova.techhub.button.clicked',
        source: 'cloud-events-example-frontend',
        data: {clicked: true},
    });
    emit(ce);

    res.json({message: 'Button click handled by server!'});
});
  • On line 1 we see that we need to import different parts of cloudevents
  • On the next line we create an emitter that sends the events via HTTP localhost:8080 to the Go service
  • Now we define an endpoint that sends a CloudEvent to the Go service
  • In it we create a new CloudEvent. This has a type, a source and data. In this case the type is com.example.button.clicked, the source is /button-clicked and the data is { clicked: true } of type ButtonEvent
  • And last but not least we send the event with emit(ce)

It’s nice to see that the CloudEvent is completely defined in TypeScript and we can therefore use the type safety of TypeScript.

Quite simple, isn’t it? So now we have sent a CloudEvent to our Go service. Now let’s look at how we can process this event in our Go service.

The FE is built and started with the following command:

1

npm run build && npm start

Go Service

The repository cloud-events-example-go contains our Go project, which receives and processes the CloudEvent. For this we use the Go SDK from CloudEvents.

Let’s also take a closer look at the CloudEvent specific code here. We use CloudEvent directly as an HTTP handler so that we don’t have to parse HTTP requests first.

Our main function looks like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

func main() {
    ctx := context.Background()
    p, err := cloudevents.NewHTTP()
    if err != nil {
        log.Fatalf("failed to create protocol: %s", err.Error())
    }
    
    c, err := cloudevents.NewClient(p)
    if err != nil {
        log.Fatalf("failed to create client, %v", err)
    }
    
    log.Printf("will listen on :8080\n")
    log.Fatalf("failed to start receiver: %s", c.StartReceiver(ctx, receive))
}

We define an HTTP client and then a CloudEvent client on it. Using StartReceiver we start listening for events and call the receive function when an event is received.

Quite practical, since we don’t have to worry about HTTP processing. All CloudEvents are passed to us directly as cloudevents.Event, there is only one endpoint in this setup.

The receive method takes the CloudEvent context and the actual event.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

func receive(ctx context.Context, event cloudevents.Event) (*event.Event, protocol.Result) {
    if event.Type() == "com.bnova.techhub.button.clicked" {
        log.Printf("Received event, %s", event)
        data := &ButtonEvent{}
        err := event.DataAs(data)
        if err != nil {
            log.Printf("failed to get data as ButtonEvent: %s", err)
        }
        log.Printf("Button clicked: %t", data.Clicked)
    
        sendCloudEvent(event)
	} else {
		log.Printf("Unknown type, %s", event)

		return nil, cloudevents.NewHTTPResult(500, "Blöd gelaufen")
	}
	return nil, nil
}

We see that we first check if it is the expected type, if so we cast our data into a ButtonEvent and log the value of the Clicked field.

If it is not a known type, we return a 500.

The sendCloudEvent function sends a new CloudEvent to the Java service.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

func sendCloudEvent(event cloudevents.Event) *event.Event {
    c, err := cloudevents.NewClientHTTP()
    if err != nil {
        log.Fatalf("failed to create client, %v", err)
    }

	event.SetSource("cloud-events-example-go")

	ctx := cloudevents.ContextWithTarget(context.Background(), "http://localhost:8081/")

	resp, result := c.Request(ctx, event)
	if cloudevents.IsUndelivered(result) {
		log.Printf("Failed to deliver request: %v", result)
	} else {
		log.Printf("Event delivered at %s, Acknowledged==%t ", time.Now(), cloudevents.IsACK(result))
		var httpResult *cehttp.Result
		if cloudevents.ResultAs(result, &httpResult) {
			log.Printf("Response status code %d", httpResult.StatusCode)
		}

		if resp != nil {
			log.Printf("Response, %s", resp)
			return resp
		} else {
			log.Printf("No response")
		}
	}
	return nil
}

It’s nice to see that we’re building a new HTTP client, which we’re using to forward the event. We set the source to cloud-events-example-go and send the event to http://localhost:8081/. Technically, we don’t define a new event, but enrich the existing event with a new source.

If the event cannot be delivered, we log it and output the error. Otherwise, we log the response and return it.

Now our Go service is also ready to receive and process CloudEvents.

The Go service is built and started with the following command:

1

go run main.go

Java Service

The repository cloud-events-example-java contains our Java project, which receives and processes the CloudEvent. For this we use the Java SDK from CloudEvents. I had a few difficulties here, the Git repo contains the final state of course.

Fortunately, the Java SDK offers us a, unfortunately quite outdated, Quarkus example. Since I want to build on the latest Quarkus version, I initialize a new project and add the CloudEvent dependencies.

Let’s look at the relevant part of the code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

@Path("/")
@Consumes({ JsonFormat.CONTENT_TYPE })
@Produces({ JsonFormat.CONTENT_TYPE })
public class ActivityResource
{
	private static final Logger LOGGER = LoggerFactory.getLogger(ActivityResource.class);
	@Inject
	ObjectMapper mapper;
	@Inject
	@RestClient
	BoredApiService boredApiService;

	@SneakyThrows @POST
	public Response create(CloudEvent event)
	{
		LOGGER.info("Received event: {}", event);
		if (event == null || event.getData() == null)
		{
			throw new BadRequestException("Invalid data received. Null or empty event");
		}

		switch (event.getType())
		{
			case "com.bnova.techhub.button.clicked" ->
			{
				var buttenEvent = PojoCloudEventDataMapper
						.from(new ObjectMapper(), ButtonEvent.class)
						.map(event.getData())
						.getValue();
				LOGGER.info("Received ButtonEvent: {}", buttenEvent);
				Thread.sleep(Duration.ofSeconds(2).toMillis());

				return Response
						.ok()
						.build();
			}
			default ->
			{
				LOGGER.info("Received event: {}", event);
				return Response
						.status(Response.Status.BAD_REQUEST)
						.entity("Invalid event type")
						.build();
			}
		}
	}
}

In the ActivityResource we register a new endpoint that receives CloudEvents. We check if it is the expected type and log the data. Unfortunately, CloudEvents does not use generics, so we would have to map our data ourselves into the desired object, here our ButtonEvent model, using the PojoCloudEventDataMapper.

Unfortunately, after numerous attempts and different approaches, I was unable to get the Quarkus to run cleanly, either it returned 415 Unsupported Media Type or an exception was thrown when deserializing CloudEvent.

1
2

2024-04-02 13:36:10,754 ERROR [io.qua.res.rea.jac.run.map.NativeInvalidDefinitionExceptionMapper] (executor-thread-1) com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `io.cloudevents.CloudEvent` (no Creators, like default constructor, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1]

Ok, I managed to get it working after all, apparently certain dependencies are not compatible with each other and you need a custom config for the ObjectMapper.

1
2
3
4
5
6
7

@Singleton
public class ObjectMapperConfig implements ObjectMapperCustomizer {
	@Override
	public void customize(ObjectMapper objectMapper) {
		objectMapper.registerModule(JsonFormat.getCloudEventJacksonModule());
	}
}

Unfortunately, this was not immediately apparent, I had to fight my way through numerous, partly outdated, documentation and examples. I then found the actual solution in a Spring Boot example.

Now our Java service is also ready to receive and process CloudEvents.

The Java service is built and started with the following command:

1

mvn quarkus:dev

Testing & Troubleshooting

I happily click the button and see that the Go service has received the event and is logging. Unfortunately, the forwarding does not work, an error is thrown in the Java service.

With curl I manage to send a CloudEvent in Structured Mode to my Java service.

1
2
3
4
5
6
7

curl -X POST http://localhost:8081 -H "Content-Type: application/json" -d '{
    "specversion" : "1.0",
    "type" : "com.bnova.techhub.button.clicked",
    "source" : "/mycontext",
    "id" : "1234-1234-1234",
    "data" : {"clicked": true}
  }'

Our Go service still returns a 400, why?

Unfortunately, the Quarkus does not log any information as to why it is rejecting the request. So we have to take care of the troubleshooting ourselves. Let’s first set the LogLevel via quarkus.log.level=DEBUG and look at the logs.

1
2
3
4
5

2024-04-02 14:05:15,660 DEBUG [org.jbo.res.rea.ser.han.RequestDeserializeHandler] (executor-thread-1) Error occurred during deserialization of input: com.fasterxml.jackson.databind.exc.MismatchedInputException: Missing mandatory specversion attribute
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 16]
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at io.cloudevents.jackson.CloudEventDeserializer$JsonMessage.getStringNode(CloudEventDeserializer.java:211)
	at io.cloudevents.jackson.CloudEventDeserializer$JsonMessage.read(CloudEventDeserializer.java:87)

That sounds helpful, apparently the specversion field is missing in our Go service. In the payloads of the faulty requests we see that a SpecVersion of 1.0 is sent along. Explicit setting also did not help.

Do we have a problem here because of the different ways CloudEvents can be used to transfer data? With the Structured Mode we have a JSON object that contains the data as well as the CloudEvents metadata. With the Binary Mode we still have the data in the body as JSON, but the CloudEvents metadata in the HTTP headers.

Using the debugger, we quickly see that the body is searched in the CloudEventDeserializer class, but there is no CloudEvents metadata there, since we are using Binary Mode.

Let’s try to make the curl in Binary Mode.

1

curl -X POST http://localhost:8081 -H "ce-specversion: 1.0" -H "ce-type: com.bnova.techhub.button.clicked" -H "ce-source: /mycontext" -H "ce-id: 1234-1234-1234" -H "Content-Type: application/json" -d '{"clicked": true}'

At least something, we have the same error. 🤣 Means, either we have to teach Quarkus the Binary Mode, or Quarkus does not support it. In this case we would have to switch our Go service to Structured Mode.

Let’s try to use the Structured Mode in our Go service. For this we only have to change one line, after digging deep into the documentation and the SDK samples.

In the sendCloudEvent method we have to adjust the context. The following patch shows how we adjust the context to use the Structured Mode.

1
2

-	ctx := cloudevents.ContextWithTarget(context.Background(), "http://localhost:8081/")
+	ctx := cloudevents.ContextWithTarget(cloudevents.WithEncodingStructured(context.Background()), "http://localhost:8081/")

And Tada, it works. We can now send events from our front end to the Go service and then to our Java service.

We see that under Frontend logs that the button was clicked. In the middle we then see that our Go service has received the CloudEvent and forwards it accordingly. On the right we then see that the Java service has received the event and outputs the data.

img_2.png

What a cool terminal that is, you can find out in Stefan’s TechUp on Warp.

So we have built our architecture and can send data in one direction and log it. 🚀

Two Way Communication

Now to the actual use case, we want to return the activity accordingly. For this we want to create our own type.

Frontend

Now comes the second, Generate Activity button into play!

In our TypeScript code we duplicate the method, bind it to a new endpoint and adjust the event accordingly. We also implement a response handling so that we can display the data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

app.post('/get-activity', (req, res) => {
    console.log('Get Activity Button was clicked!');
    const ce = new CloudEvent<ButtonEvent>({
        type: 'com.bnova.techhub.get.activity',
        source: 'cloud-events-example-frontend',
        data: {clicked: true},
    });

    emit(ce).then((result) => {
        console.log('Result:', result);
        let ceResultStr = (result as { body: string }).body;
        let ceResult = JSON.parse(ceResultStr) as CloudEvent<Activity>;
        res.json(ceResult.data);
    }).catch(console.error);
});

The method looks very similar, we again create a CloudEvent with the content ButtonEvent (within data) and send it to the Go service. Now we wait for the response, which is of type Activity, and return it to the HTML for display.

Go Service

In our Go service we define a new if-else condition for the new CloudEvent type.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

    else if event.Type() == "com.bnova.techhub.get.activity" {
		log.Printf("Received event, %s", event)
		data := &ButtonEvent{}
		err := event.DataAs(data)
		if err != nil {
			log.Printf("failed to get data as ButtonEvent: %s", err)
		}

		if data.Clicked {
			log.Printf("Querying activity")
			result := sendCloudEvent(event)
			log.Printf("Result: %s", result)

			event.SetSource("cloud-events-example-go")
			if err := event.SetData(cloudevents.ApplicationJSON, result); err != nil {
				log.Fatalf("failed to set data, %v", err)
			}
			return &event, nil
		}
    }

Same as in the first example, we check if it is the expected type and then work further with the ButtonEvent. If now data.Clicked is true, we send the CloudEvent to the Java service and wait for the response. If a response comes back, we set this data in the event and return it.

Here it is nice to see that we explicitly set the source for the return, so our frontend knows that the data comes from the Go service.

Java Service

The extension in the Java service is conceptually the same, we define a new block that handles our CloudEvent.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

case "com.bnova.techhub.get.activity" ->
{
    var buttenEvent = PojoCloudEventDataMapper
            .from(mapper, ButtonEvent.class)
            .map(event.getData())
            .getValue();
    LOGGER.info("Received ButtonEvent: {}", buttenEvent);

    Activity activity = boredApiService.getActivity();

    CloudEvent cloudEvent = CloudEventBuilder.v1(event)
            .withSource(URI.create("cloud-events-example-java"))
            .withData(PojoCloudEventData.wrap(activity, mapper::writeValueAsBytes))
            .build();

    LOGGER.info("Prepare Sending activity: {}", cloudEvent);
    Thread.sleep(Duration.ofSeconds(2).toMillis());
    LOGGER.info("Now Sending activity: {}", cloudEvent);

    return Response.ok(cloudEvent).build();
}

Here it is again nice to see that we have to cast our Pojo by hand first.

Then we get the actual data of type Activity from the BoredAPI and add it to the current CloudEvent. For demo purposes we wait another 2 seconds before sending the event back. This gives us enough time to see that the data really comes from the Java service.

Then we return the cloudEvent back to the Go service.

It should be noted here that this communication is synchronous.

If you now have problems that the Go service cannot receive the data, pay attention to your Consumes & Produces annotations in your Java service. I had problems at first because the normal JSON MediaType was still in Produces, the Go service could not parse the data then. You have to explicitly set the in-house CloudEvents MediaType.

And now our complete implementation for synchronous, bidirectional communication is ready. 💡

Testing

Now let’s test our complete application and press the Generate Activity button.

We see in our Warp terminal that all services give corresponding logs and after a short wait the data is also displayed in the front end.

img.png

  • In the left terminal, in the front end, we see that

    • the button was clicked
    • an event was received
    • the event contains the activity data from the BordeAPI

  • In the middle terminal, in the Go service, we see that

    • the event was received
    • the event contains the ButtonEvent information
    • the event was forwarded
    • a response was received, with status code 200
    • the activity data was set in the event

  • In the right terminal, in the Java service, we see that

    • an event of type com.bnova.techhub.get.activity was received
    • it is a ButtonEvent with clicked: true
    • a result with activity data from the BoredAPI was returned

So we have built and tested our complete architecture. 🎉

This data is then also displayed in the front end. After a few tries, we found a suitable, summery activity and our boredom is defeated.

img_3.png

So our CloudEvents Hands-On project is finished for now! 🍦

Conclusion

We have now looked at CloudEvents together, with a hands-on, HTTP use case. I clearly see the advantages of CloudEvents, it offers a standardized way to define and use common schema types for events.

I wonder if HTTP is the right use case here. A communication like in the example could have been implemented super, I would even say easier with normal HTTP requests. The use of OpenAPI Specs would certainly make sense here.

From my point of view, CloudEvents could play to its full potential in a microservices architecture, where events are triggered and processed by different services. Provided that these are also technically different transport types, such as Protobuf or JSON, Kafka or RabbitMQ.

What do you think? Should we do a third, deep-dive TechUp on CloudEvents? We are thinking of a use case with Rest & Kafka, for example, to connect the synchronous and asynchronous worlds. It would also be exciting to see if and how to use a schema registry to manage the schemas.

Furthermore, I would be very interested to know how easy it is to adapt the transport layer. How much effort is involved in switching from HTTP to Kafka?

Would that be interesting? Let us know!

This techup has been translated automatically by Gemini

Tom Trapp

Tom Trapp – Problemlöser, Innovator, Sportler. Am liebsten feilt Tom den ganzen Tag an der moderner Software und legt viel Wert auf objektiv sauberen, leanen Code.