While talking with a client recently about their performance challenges, I was relieved to find that the database wasn't the problem. Instead, their API server was overloaded by the number of calls taking place in their application. While the database did provide the backing for the API calls, there was a fair amount of caching. However, as they'd moved to microservices, more and more of the interaction between modules was taking place as a network call to a single server, which became overloaded.

Certainly, they could add more API servers, or break out different types of API servers, but each of these changes potentially has downsides. Data can become out of sync across two systems, which is why we often want a single database storing all data. Distributed systems are hard to architect and implement, and I suspect as more developers try to build them, we'll find more issues like this. Not that developers are doing anything wrong, but they often don't realize the challenges of building a distributed system when the workload scales.

They're relearning the knowledge that others have gained through their own experiments and failures.

One of the things we try to do here at SQL Server Central is share our knowledge with others. We hope to help each of you learn from others, and hopefully not need to make the same mistakes in your applications.

As I listened to the client talk, I was reminded of this post from Troy Hunt. In it he talks about the HaveIBeenPwned site, which he built a scaled to meet demands. The site doesn't cost a lot, but there is a great quote about the API server he set up. He wrote: "I’ve tested it with 380k queries a minute! Infrastructure will scale out and magic will happen, so long as you’re not maxing it out for perpetuity, just go for it."

However, in your company, you might be scaling in perpetuity in an app, especially as a workload grows and more parts of your application need to make queries against an API. Good design, good coding, and adhering to good distributed system design, will help ensure you can handle the load. This is especially true in the cloud, where scaling up is just provisioning another resource, but there is a cost.

You have to know what to do, so I urge anyone looking to build a distributed system to read about the experiences of others. Learn from their knowledge, rather than make your own mistakes and try to correct them. There is a lot of great information out there, but you need to be looking for resources that provide technical details, like Troy's posts on how he built his site or Jeff Moden's posts on writing high performing queries. That's the type of knowledge that is invaluable if you want to be an efficient and effective developer.