monitoring & metrics are key for having many microservices tbh
03/05/2020, 8:23 PM
I have logging, monitoring and metrics pretty extensively. It's still super annoying to troubleshoot. For instance one of the micros couldn't communicate to the other in certain situations, I had three instances running with a loadbalancer in between right (kubernetes). First to troubleshoot you need to figure which service is having the issue, then which instance, then when and how, then replicating is nearly impossible as you can't control which instance is hit, etc.
I find it massively easier to troubleshoot a more consolidated system. Also things seem to be a lot faster and more stable since I consolidated. I guess I'm not bought into the microservice thing entirely....
03/05/2020, 9:28 PM
do you have tracing?
I couldn’t live without jaeger tracing tbh
and I have over 50 microservices heh
may be something to look into!
03/06/2020, 1:28 AM
Not using anything like Istio, but I do plan to get Istio installed at some point so we can see flow
What transport mechanism are you using? I am using TCP and it seems very unreliable. I was considering GRPC, or some sort of Queue System like RabbitMq