8

So currently where I work we try to mimic our production environment as close as possible to our production environment. Our micro-services (APIs, listeners, data stores) are organized in the fashion that if it performs a responsibility (down to the data model) different than something else it deserves its own container.

Currently for our local development environment we have a mesh of Docker Compose files and NodeJS code that intertwines all this together, however it has proven to be unsustainable as it only works 20% of the time. Each API microservice (per model) I have been able to have my input accepted to condense the datastores into 1 database, as I believe the database systems such as MongoDB/MySQL have proven to have solved these issues at a lower layer.

Right now our containers come out to about 48 instances, which for me is crazy but this is described as the new way. These containers will randomly crash (some have memory limits so perhaps this is the cause). Is there a way that we can make our development environment not conform to our complicated production environment but still maintain productivity?

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
Darren
  • 191
  • 5
  • 1
    You already pointed to the problem. I would suggest to review why the APIs consume that much of memory. If to solve it is cheaper than to buy more hardaware then go for it. Otherwise scale the hardaware. If no scale solves the problem, then its critical to look for those conflictive apps that crashes their Dockers. From my experience , I would do all the possible to grant a test env closer to the production, instead of wasting time on palliative solutions. Anyways It's going to be hard to state a valid answer to your question because we would need to know In deep your whole context. – Laiv Oct 26 '16 at 06:11

2 Answers2

7

Microservices should be relatively independent of each other. During developer testing, you should be able to only start the one you are working on and perhaps two or three others, usually related to storage, messaging, or authentication. If you can't, you should work on improving your architecture.

If your full application won't fit properly on a developer machine, then provide a separate cluster for developer integration testing. A lot of organizations simply budget extra time on their production cloud provider, or you can build an in-house cluster using something like OpenStack.

This cluster can be over-scheduled because every developer doesn't need it 100% of the time. I estimate I do about 85% of my microservice development just with unit tests, about 10% just with the single-microservice environment on my own machine, and the remaining 5% on our shared cluster.

You probably also want to invest in better monitoring and orchestration, so you know exactly why your microservices are crashing and can have them automatically restart when they do. Perhaps something like Prometheus and Kubernetes. Kubernetes also refuses to schedule microservices in the first place if they will require too many resources than the available nodes can handle.

Karl Bielefeldt
  • 146,727
  • 38
  • 279
  • 479
-1

If you're running microservices on Kubernetes, we're working on solution that involves proxying local code to a remote cluster. Means you get fast feedback loop of local development, but full realistic environment for running the service. Check it out: http://telepresence.io

Itamar
  • 1