Transcript
David Brown
Welcome to episode 80 of coding over cocktails. My name is David Brown, our guest for today is a consultant, author, and instructor. He is the founder of microapis.io and the author of “Microservice APIs” which is currently available under Manning's early access program. He is recognized as a thought leader in the fields of cloud computing, devops and automation. He speaks regularly at international conferences and frequently organizes public workshops and seminars, joining us for a round of cocktails today is José Haro Peralta. Hello, Jose. How you doing?
José Haro Peralta
Hi, David. Thank you for having me. I'm very excited.
David Brown
Good! It was a pleasure to have you. Congratulations on the book, it's on the early access program so when does that mean it will be published as a completed product?
José Haro Peralta
We're expecting the book to launch in January next year, January 2023.
David Brown
Yeah. I mean it's already pretty comprehensive, so it must be pretty close. So let's get started, we'll dive into some of the topics of the book. You mentioned in the introduction of your book that what inspired you to write it was years of experimentation and often through frustration of building microservices platforms and driving integrations through web APIs. So run me through what these frustrations were and share some of the challenges you faced.
José Haro Peralta
Sure. So I have a lot of stories about this, but I think the most recurring theme about around these stories is, you know, working with a team, we're trying to build an API integration and obviously we're trying to build this API and we obviously the client development team in the backend development team, we sit together, we discuss the requirements of the API, we agree on something, then we part ways, we build a client and the server, we tried to get them talking to each other and it's like we're talking different languages. And so it's about- what we need is a number of processes and tools that help us deliver these API integrations more reliably. And what I try to do in the book is to explain some of these processes and tooling that would help people avoid these situations.
David Brown
Okay. Well, let's dive into some of those processes and tooling. So you talk about documentation driven development, let's get started with that. What is documentation driven development?
José Haro Peralta
Sure. So, you know, when you talk to API practitioners, traditionally we've had this idea that to build an API the right approach is to do API first. Now API first is a little bit ambiguous. And the, you know, more recently we realized we have to be a little bit more precise is what we mean by that. And so we come up with the concept of design first and that's a great concept. It means before we build API, we should design it because at the end of the day, you know, when we're building an API we're building it for our consumers. It’s that similar to building a UI. We are building a UI for users to use it. We're building an API for a client to consume it. So, ideally, we designed the API to meet the requirements of this client and there is a design process to get that done. But when it comes to the reliability of the integration, what I want to make sure is that is that we have something more specific than that. Something we can use to validate our implementation and that's what would be the documentation or the API specification. So I want to be more specific that what we need is this API specification in place.
David Brown
Okay. So the documentation project is part of the design process, it's still encompassed in API first development, it's just it's the initial part of that process is sitting down, designing it, documenting, getting everyone on board before you start writing any code. Is there, that collaboration process between stakeholders and developers and who's in the room?
José Haro Peralta
Everybody as if it's like when you're building a UI really when you're building this yard because it's gonna have implications for the back end is obviously the front end application and for the stakeholders, designers, product manager, everybody who has a stake in the API should be in the room because we're trying to serve the needs of the client, of the stakeholders who are going to use this client and it's gonna have implications for the back end as well. So it's gonna have to be a conversation between everybody who has taken the API.
David Brown
How comprehensive does the documentation become before you start mocking the services of the API.
José Haro Peralta
What I would say is the best way to start is small. So like the approach that I always try to evangelize is to have ideally a very comprehensive API design, very detailed and then you go to to build it. Obviously this is not realistic in a practical scenario, because the way you're going to discover what the API needs to look like is when you start tinkering with it. So the best way to get started is to have a conversation about what the API should look like, then do some experimentation, you know, build some in the backend put some endpoints, some payloads, see what it implies for the process of fetching data from your database, transforming that data, computing certain properties do the same in the frontend or your client could be a CLI could be an IOT device, whatever is consuming that API see what you need to get from the API at every stage of your communication and see if this makes sense also for the stakeholders. So this is a dynamic process at the beginning but what you want to make sure you have at some point is this consolidated API specification and then building against that.
David Brown
Okay, Now in the start of the book you launch into REST APIs and GraphQL APIs. Run us through the use case of each. And are they the only options for microservice APIs?
José Haro Peralta
So the last question first - absolutely not the only option. There is also a popular choice is also gRPC and there are all synchronous API you can also have a synchronous API as well. So like if you have a queue or a stream between your services. In terms of comparing REST with GraphQL, you know what I would say the best use cases of REST is when it's straightforward to fetch a resource representation from the server and it kind of matches exactly what you need in your client and GraphQL when you need flexibility using those resources from the survey because that's the whole point of GraphQL. Right. So you have this very complex and very big representation of resources in your server. You don't need all of those properties in your client. So you need to be very selective with the information you get so that you can get everything you need. And that's a good case for GraphQL.
David Brown
And I'm guessing they're in a microservices integration or in a larger one. You could have use cases where you're using both. Would that be reasonable?
José Haro Peralta
Exactly, So the very popular pattern these days is to use graphQL as a kind of front end for REST APIs as a kind of gateway. So to simplify things for your consuming site for your API clients. Very popular pattern is to put a GraphQL interface between the client and services and this GraphQL API knows how to fetch data from different services and the client only needs to know how to query data using the GraphQL query language.
David Brown
Okay, why is that becoming popular?
José Haro Peralta
I guess it's because it offers a simplified and unified interface for the client. The client doesn't need to know about different endpoints for different services. Everything is just one single endpoint for querying data and it's the same query language for all services. You don't need to worry about different endpoints and domains and so on. All you have is queries and mutations and these queries and mutations because they are the couple from the services the same query maybe can combine where it's from three different services perhaps. And also this middle layer can maybe leverage caching and some things that might not be so easy to work straight together with separate services.
David Brown
Makes a lot of sense. In your book you discuss the adaptation of a three tier architecture to structure microservices into modular layers. Can you run us through this concept of layers and these three tiers?
José Haro Peralta
It's one of the earliest architectural concepts I guess when we get familiar with right when we are building web services. So the idea of three tier architecture or multi architecture as we also know it. The ideas we structure the application in three or more layers. The idea is that every part of the application goes into its own space. So typically we will have a presentation layer, business layer and a data access layer or presentation tier, business tier and data tier. So the idea is these different domains of the application live in different spaces. And visually or architecturally the idea of three tier architecture helps us to decide where to put it's part of the application. Then obviously we will have each layer completely coupled as well.
David Brown
Now this leads us towards something else you talked about which is a hexagon architecture. So it sounds a lot more complex. The idea is loosely coupled services, but how is it different from the three tier architecture?
José Haro Peralta
So in multi tier architecture the proposition is very simple. We structure our application into tiers. The idea of hexagon architecture built into this tradition of how we can build more decoupled services with clearly defined boundaries and so on. So the idea is visually we think of this hexagon and the center of the hexagon represents our business layer and that's the part of the application we want to isolate and maintain free of dependencies from external components, then the external components becomes the edges of the hexagon. So we know hexagonal architecture also as architecture of ports and adapters. Then the idea is these external dependencies of the app, like a database, like an API interface, a CLI interface, a cache. These external dependencies which are not necessarily to implement the business layer they just help us operate this business layer. So they become the adapters of the application and then we use ports which are technology agnostic interfaces towards these external dependencies. So the idea is our business layer, when it's talking to the database, it doesn't know it's talking to a database, it's just using something to persist data in a persistent storage and the ports takes care of translating, businesses commands into database specific transactions. So the hexagon architecture has a much more clear proposition of how we decouple the business layer from our external dependencies. That's what I love about it.
David Brown
So there must be some benefits to using the hexagonal architecture. Is the benefits in terms of testing or development. Where the benefits coming from?
José Haro Peralta
Exactly. So because it helps you decouple the business layer from everything else, you're able to test the business layer in isolation. So it means when you're testing your business layer, you don't need to have a database. Not necessarily. And in some cases you may want to have it, but in most cases really you want to test that your classes and functions are doing what they're supposed to do regardless of whether there is a database or an interface or something like that. So you can test the business layer in isolation and you can replace, because the ports encapsulates the complexity behind the adapter, you can put behind a test adapter. That's the idea. So instead of having a real database, you have something that looks like a database. But the business layer doesn't care because the port is handling this communication with the external dependency. It also, you know, because everything is decoupled if it's properly done your business layer doesn't depend on the database or the API. It means you can, if you have situations in which you have changes to the schema of your database which are not directly relevant for the business layer, those changes are handled by your ports and adapters. The business layer is not affected. It shouldn't. It should prevent these external dependencies from leaking into the business layer. So very common situation for example renaming a column in your database is the same data. Everything is the same. Just change the name. If you have this typical pattern in which you're basically streamlining. We're gonna break the API immediately because you changed one name in the column property name changes but if you have this translation between layers then translation middle handles these changes and protects and isolates your application from those changes. So in terms of development it's a much better experience for the team because as small changes to the external dependencies don't impact the application.
David Brown
Does it change in any way? The concept of domain driven design for microservices?
José Haro Peralta
I think it goes well with it. So we would apply to domain driven design in different levels when we're working with microservices. So we can use them as domain driven design. To take down the concept of sub domains right? So we may have a payment subdomain, users subdomain, claims subdomain. They become individual services. But then within the service we can still leverage domain driven design and its patterns to make our implementation better. So we will still use domain object, domain entities, data objects and so on to make our implementation better.
David Brown
It sounds like it's a more complex scenario to develop this way. They use cases for the hexagonal architecture. Their use cases where it's not worth the effort. When do you recommend to use it?
José Haro Peralta
Let me start with when I wouldn't use it. I would say I wouldn't use it if the application is going to be like really simple, then perhaps I wouldn't use it. But if there is any chance that the application is going to grow complex, it really is not very difficult to to lay the foundation of a good structure and good architecture for your application by, you know, there's nothing necessarily complicated about hexagonal architecture and although the pattern encourage you to have fully decoupled business layer from the, from the adapters, this is something that can go also progressively, maybe in the first iteration, you may have database logic leaking into your business layer and that's fine at the beginning. But as you move on and the data becomes more complex and the interface becomes more complex, you want to make sure that you're fully decoupling each layer so that the evolving and development of each layer can happen more easily without interfering with each other. Then that's when we introduce all the parents, to enforce this decoupling, there are some patterns I introduced in the book as well, like for example, repository pattern which is very commonly used in domain drive design as well. So to decouple the data access layer from the business layer, it's a good pattern to do that.
David Brown
Okay. And I'm guessing that under this architecture, it's agnostic in terms of the style of API you're using as well, whether it's rest, GraphQL, event driven or anything else.
José Haro Peralta
Exactly, because the basic idea is so simple. You have a co-business layer and external dependencies. APIs are external dependencies. Because it's what allows you to expose the capabilities of your service to the world. It doesn't matter if it's a rest API as a matter of fact any kind of architecture would allow you to do that anyways but it fits very nicely in a hexagonal architecture. Thinking that you can have different interfaces to the service. So it can be a REST interface, it can be a GraphQL interface, it can be a CLI interface to the service. And all these interfaces sit together outside of the business layer. They're just consuming the capabilities of the business layer and the business layer is not bound to any of them because we want to keep it isolated. It's perfectly possible and it fits nicely with this concept
David Brown
The concept was introduced in, as I understand it in 2005 by Alistair Cockburn. So it's basically an adaptation of that concept to microservices. Right? So is it something which is just recently been adapted and is taking off or is it just slowly evolving. What's initiated this?
José Haro Peralta
I would imagine. Just to clarify, it’s called the hexagon architecture applies to individual services, not to the whole architecture. So when you're building an individual services, when you want to apply this architecture inside the service. I would think that hexagonal architecture has been quite popular since the beginning, but perhaps I've seen more recently a lot of emphasis on hexagon architecture. Perhaps what we're seeing is a lot of developers coming to face with the fact that growing applications, growing complexity need better structure. And hexagonal architecture is a good solution for that. And it's true also when you see diagrams of microservices, every service is always a hexagon. So we come to think naturally for services. Excellent. And it's very embedded in our developer culture, I think now.
David Brown
Interesting. Tell me there's the container orchestration engine which you're using to power your microservices, does that affect in any way the architecture or API you are using? So it does matter whether you're choosing Kubernetes versus Docker swarm or a proprietary solution like AWS elastic container service for example.
José Haro Peralta
It shouldn't have any effect. So your infrastructure should be completely decoupled from the implementation of your services. There are obviously technologies that these cloud services provide that can support your implementation. So AWS API gateway for example, is a great support for your REST interfaces, but otherwise I would be very, very careful not to cut too tight couple your service implementation to infrastructure. So whether you use kubernetes or and whether it is in GCP or in Azure or in AWS or whether it is eleastic container service or whether it is something like Heroku or Render your implementation should run just the same. And infrastructure should help you just with things like reliability, operations, availability, redundancy and things like that.
David Brown
Your book is incredibly comprehensive, covers a lot of ground. Who is the intended audience for it? Is it the stakeholders involved in the API which in particular like the architect or the designer. Who is it?
José Haro Peralta
During the book review, a lot of reviewers highlighted that the book is useful for a lot of stakeholders in the system from developers to CTOs and going through managers, business stakeholders and the piece of engineering and so on. And I think to an extent is true because by giving all these approaches and tooling it can help business leaders to make better decisions or technology leaders to make better decisions about the implementation strategy. But at the end of the day, what I had in mind was the developer who's working day to day with microservices and APIs. And really the kind of setting I had in mind, is a lot of companies that I work with mostly says growing startups that suddenly they need to work with microservices and APIs and because everything is going so fast and everything is growing so fast, there is really no much time to teach how to do things and how to put in place appropriate strategy because it needs to happen very fast. So what I see is a lot of, often these developers are lost and they have to figure out a lot of things by themselves. So thinking of these people, how can I make their life easier in this situation, which is so common and really what I wanted, it covers a lot of ground, but what I wanted to give is just kind of like the basic stuff, the basic stuff you need to know in each step of the way of building microservices platform and the APIs. So in terms of documentation, design, implementation, authorization and deployments. Just so that you have a road map in your mind because otherwise I know the feeling when I work with these developers is being, feeling so lost with so many choices out there and in every step of the process is how did you document it, how do you test it? How do you authorize? What about security? What is JWT? What is Oauth? What is OpenAPI? So just bringing together all these things for these people who are working on these things every day.
David Brown
And because you do give some examples for developers in python, but you the concepts that apply to any language or even a low code platform.
José Haro Peralta
Absolutely. So the original title of the book was Microservice APIs in python. But during the review process, like I said, some review is why highlighting that the book is actually useful for more than just developers. And a lot of reviewers also highlighted that what I teach is not just for python developers, there is a lot of content which is useful for people working with different stacks. So we decided to leave with microservices API and we just happen to have the examples in python, but the principles and the patterns and the strategies that are teaching the book applied regardless of the stack.
David Brown
I think it's a very easy read and there's a lot of valuable information in there, Jose, congratulations on the book. And how can our listeners follow you and make sure their up to date with the announcement of that book. What are the best social media platforms to follow you on?
José Haro Peralta
The best one? I would say Linkedin. So follow me or connected with me on Linkedin. I like to connect with people there and I think every day we have some interesting conversation on Linkedin. It's becoming really a great forum to talk about anything related to tech and for announcements I have a newsletter in Superstack is my creepy eyes dot super stack. Registering with that newsletter is also a good way to stay up to date with my book workshops that I release every so often or new content that I'm publishing related to the book.
David Brown
And you handle on linkedin is Jose hyphen haro hyphen Peralta?
José Haro Peralta
Like you said. Jose hyphen haro hyphen Peralta. Yeah.
David Brown
Alright look Jose, thank you so much for your time today. Congratulations on the book. It's a very easy to consume book with so much useful information. Introducing some really good concepts like the hexagonal architecture, which is really really useful information. Perhaps not as widely known as some of the other concepts like rest and graphQL So I would highly recommend it to our listeners. Thank you having coming on our program today.
José Haro Peralta
Thank you so much. David.
David Brown
It's a pleasure.