Monitoring And Orchestrating Your Microservices Landscape using Workflow Automation (Part 3 of 7)

On Wednesday, March 11, 2020, I conducted the webinar titled “Monitoring & Orchestrating Your Microservices Landscape using Workflow Automation.” Not only was I overwhelmed by the number of attendees, but we also got a huge list of interesting questions before and especially during the webinar. Some of them were answered, but a lot of them were not. So I want to answer all open questions in this series of seven blog posts covering.

• Part 1: BPMN & modeling related questions (6 answers)
• Part 2: Architecture related questions (12)
• Part 3: Stack & technology questions (6)
• Part 4: Camunda product-related questions (5)
• Part 5: Camunda Optimize specific questions (3)
• Part 6: Questions about best practices (5)
• Part 7: Questions around project layout, journey and value proposition (3)

Note that we also started to experiment with the Camunda’s question corner and discuss to make this more frequent, so keep an eye to our community for more opportunities to ask anything (especially as in-person events are canceled for some time).You can find the recording of the webinar online, as well as the slides.

Stack & technology questions

Q: How could Camunda be used in conjunction with Apache streaming and queuing platform?

The downside of questions via chat is that you simply can’t ask counter questions to understand the exact meaning. So I am not sure what Apache streaming and queuing platform relates to. But if my consulting past teaches me just one thing then that this does not need to stop me from answering ;-)

Let’s assume this platform is what I know as Apache Kafka. Then there are two interesting aspects to look at.

First: There is an overlap of the use case of streaming and workflow, especially when streaming is used to build event-chains between microservices, as I described with the choreography in an earlier answer.

But, there are also good examples of how to use streaming and workflow together. My ah-ha moment was when saw one use case around vehicle maintenance, which I describe in detail in Zeebe loves Kafka: Act on Insights of your Streams:

Assume that you have a huge amount of sensors that constantly send measurements (oil pressure is 80 psi) via Kafka. Now you have some clever logic generating insights based on these measures (oil pressure is critically high). All these insights are also sent via Kafka. But now you want to act on this information, e.g. to alert an operations person to organize some maintenance.

In order to do so you have to get from the world of stateless event streams into the world of stateful workflows, from a world of a massive amount of information (the measures might be send every second) to a world with lower numbers, especially as you want to start a workflow

The second interesting aspect is actually about the integration of both tools, which is exactly the next question :-)

Q: Integration with Apache Kafka, best practices?

Integrating a workflow engine with Apache Kafka is technically speaking relatively easy. You need to:

Ingest records from Kafka into the workflow engine, which is done via message receive events/tasks in BPMN.Publish records onto Kafka from the workflow, which is done via message send events/tasks in BPMN (or service tasks if you prefer).

You can find some pieces of code doing this with Java and Camunda BPM e.g. in the flowing-retail example using Spring Boot and Spring Cloud Streams. Or if you use Camunda Cloud (Zeebe) you could also leverage the Kafka connector for Zeebe, as you can see in this example on GitHub.

Q: Integrating with Spring Cloud?

Yes, this is possible. Camunda BPM offers a Spring Boot starter, so it is easy to hook it into the Spring universe including Spring Cloud. There are some examples using e.g. Spring Cloud Streams or also Spring Cloud to run Camunda on Cloud Foundry/PCF.

Q: Do you have any hands-on experience with a heterogeneous Camunda architecture? If yes, do you have any suggestions or best practices?

This question would need to be revised with more details. Almost every customer uses Camunda in a heterogeneous context. I could imagine this question is related to the next two: What if I don’t develop in Java?

Q: You mentioned the benefits of embedded Camunda. In this scenario, what languages or tech stack is Camunda compatible with? We like our microservices written in Go and Node.JS.

The Camunda engine is written in Java. So it is not possible to run it embedded as a library in any other language. Sorry.

BUT: You can use Camunda from other languages. I wrote about that in 2017: Use Camunda as an easy-to-use REST-based orchestration and workflow engine (without touching Java). The main idea is: Camunda provides a REST API which allows you to code in whatever language you like and just talk REST with Camunda:

This works very well for a lot of customers in different worlds, like for example Go, Node.JS, C# and Ruby.

With the upcoming 7.13 release we further improve the experience by introducing two features:

Camunda Run: A distribution of Camunda BPM that is highly configurable and doesn’t require you to have any Java know-how.OpenAPI Support (aka Swagger): This allows you to use the REST API from the programming language of your choice easier, for example by generating a client for it.

Q: Is it OK to use Camunda with a full .NET solution?

Yes, of course. See my answer to the last question, which also applies to .NET. We still do not provide a ready-to-be-used client library as a NuGet package, but we are not that far away anymore. And we plan to improve support for other languages over time anyway.

Previously published at https://blog.bernd-ruecker.com/microservices-webinar-faq-1a9741f4481c#06ac