Add-ons fill the enterprise gaps in Docker's ecosystem

Nick Hardiman
Updated

 Image: Docker

Ten years of IT transformation, from the launch of AWS to the growth of the container ecosystem today, has put new tools, new APIs, new techniques, and new algorithms in the hands of developers. The latest big shift to a developer-centric world is the rise of the Docker ecosystem.

Developers heart Docker

If there's one thing developers like, it's coding. All those infrastructure headaches -- and that includes everything from defining networks to dealing with infrastructure operators -- are annoying overhead. Docker, a Linux container system, is the pill that makes infrastructure pain go away (if you work in ops, you may now have steam coming out your ears).

Developers should love Docker. It simplifies and speeds up IT. For instance, Docker lets developers treat infrastructure as code. Their application, neatly packaged up in a Docker container, is easily copied around the infrastructure, from dev to test and test to production. With the addition of third-party management tools like Kubernetes, Docker infrastructure can be described as a YAML file -- home turf for devs. Nice and tidy.

Docker is not all that for distributed applications

Docker is sold as the platform to help developers create distributed applications, but Docker has not been so hot in some areas.

Container Solutions is an Amsterdam-based consultancy based around Docker. Adrian Mouat is chief scientist at Container Solutions, and author of "Using Docker." I took the opportunity to talk to Mouat at QCon London 2015, a developer conference. Mouat gave an example of a problem. "You can have multiple containers on a host, but moving a container from one host to another is an issue -- especially getting its data to travel with it. Getting Docker, out of the box, to communicate with different hosts is a lot more difficult."

The Docker ecosystem has grown up to connect the distributed application dots in areas like these.

  • Motion: Stateless containers can be destroyed, but stateful containers need to be moved around hosts.

  • Networking: The ability to talk to other containers elsewhere in the network is essential for a distributed application.

  • Orchestration: Getting many containers to work together requires a management system.

  • Extension management: How to add many optional extras to Docker.

  • Service discovery: A new containerized application needs an authoritative source to tell it where to find services.

The Docker ecosystem fills in the gaps

Docker was a good fit with the cloud dev, but it was a single box solution that didn't come with the large-scale components that an enterprise needed. Organizations have been rolling their own solutions to fill in the gaps, creating a whole Docker ecosystem in the process.

The Docker people decided to fill in these gaps themselves to create a more appealing platform. Jessica Frazelle, software engineer at Docker, described the Docker approach to solving these problems as "batteries included, but replaceable." That means Docker comes with a full set of default components, but they can be easily replaced with third-party products.

Networking

"Docker out of the box will communicate with different containers on the same host. Containers on separate hosts, it becomes a lot more difficult," Mouat explained. Docker bought SocketPlane to provide its "batteries included" solution to this problem.

Mouat explained how Weave is one of the "replaceable batteries" solutions. "Weave is an SDN (Software Defined Network). That lets you set up a network - it's all done in software -- which makes it much easier to talk across hosts, without doing any of the hard wiring work."

Movement

Mouat said, "You can have multiple containers on a host, but moving a container from one host to another is an issue -- especially if you want the data to travel with it. Flocker is a solution to that." Flocker, from ClusterHQ, aims to solve the problem of managing stateful containers. For instance, an organization may run a database in a container, and they may not be happy to just destroy that container and start again.

Orchestration

Mouat talked about the challenges of managing a large fleet of containers. "If you've got a large network of containers, how do you manage them all? How do you decide which job goes in which container, [and] how many containers you need? What do you do about monitoring, job control, and scaling?"

Mouat described how you can swap out the default Docker orchestration "for a different one, like Mesosphere. There's Kubernetes from Google, and CoreOS are also involved in things like Fleet."

Extension management

Simply swapping one component for another is limited. Docker does not come with an extension model for adding many extra components. A developer could not use both Weave and Flocker because each one wants to wrap itself around the Docker binary.

Mouat described how ClusterHQ and others have come up with a solution called Powerstrip. "Powerstrip is a way to plug extensions into Docker. It's essentially a hack that relies on the fact that the Docker daemon talks over HTTP. You can intercept that call, do your own bit, and pass the connection onto Docker." Mouat continued, "With Powerstrip between the Docker client and daemon, you call Powerstrip first, which will then call Weave and then Flocker, and then it's passed onto the Docker daemon."

Service discovery

A new application, fired up in a new container, needs to know where to find services. Containers require some kind of central authority with configuration answers. "The etcd solution is you go to this distributed key-value store, and you ask it -- where can I find my database," said Mouat. "ZooKeeper is similar -- that's the one that is used by Hadoop."

These gaps have been filled before

These problems have been solved before, and some of these solutions can be extended to include Docker. JFrog Artifactory is a binary repository manager that was recently extended to manage Docker binaries. Cloudsoft added Clocker to Apache Brooklyn to make a Docker cloud system. Ansible and Puppet can manage Docker.

Final thoughts

The Docker ecosystem is young, and it is incomplete. But it's surprising how much whitespace has already been filled in.

Also read

Disclaimer: TechRepublic, ZDNet, and Tech Pro Research are CBS Interactive properties.