Observability! Cloud Functions! APIs! What could go wrong?!
While researching the performance of object storage APIs there appeared to be custom run time magic happening, leading to significant performance differences. Further research showed that it was *not magic*, but lead to even more questions.
Working with modern systems means network connections, many of them. Understanding how those connections impact your customer’s experience can be difficult. Distributed tracing helps isolate what parts of the system are failing, but when only implemented at the RPC level the reasons for and scope of network induced issues can be lost. See how network level insights can be integrated into distributed traces and hear how to effectively practice iterative observability from the specific case of this research to a general framework for investigation.