Library Knowledge Depth

There are no small libraries, not anymore. Actually, come to think of it, there are small libraries, they just aren't used by the application developer these days because the tools employed tend to be all-encompassing. That's the attempt anyway, and with valid reason too — I'm not going to like using toolkit X because I can't do Y. To do Y, I need toolkit Z. The less variability in the dependencies of my application, the happier I am as a developer. This over-arching attitude of mammoth libraries, however, creates a knowledge problem. The library tries to solve so many problems, that its size grows beyond what can fit in any one developer's head.

That's not to say that there are fenced-off areas of, say, a UI framework that are completely foreign to a developer trying to use and understand the library. It's just that they cannot apply it with the same degree of confidence as other areas of the library. I might have a conceptual idea of what the particular aspect of the library does, at a conceptual level. I might even understand some of the code examples at first glance. But unlike the parts of the library I'm more familiar with, these ones I'm not putting to use on a daily basis and making new discoveries about the code. Knowledge of library intricacies come only from applying the library in a diverse environment. Repetitively. There just isn't enough time for us developers to understand these libraries in their entirety.

Rarely, are libraries created by one developer in a silo. Instead, libraries are the result of a collaborative effort in which many developers contribute. Just like the applications that use these libraries — there is a team involved. Does this mean that we're safe from the emanating shadow of huge software libraries? Can we just stick each developer in their area of expertise within the library? We do that to a degree in any software development effort anyway. If it isn't a hobby project, involving just one developer, there are many developers, all of whom have skill sets in different areas. Not necessarily from the same toolkit either. And this is essential, this level of diversity, because of the communication levels required between software components, because of the wide variety of hardware platforms and interoperability requirements. Perhaps this is part of the end goal of big, monolithic libraries — to cover all the requirements of the software development team using that library. Which just begs the question — are huge libraries trying to be everything to everyone?

Maybe it isn't quite that bad. Maybe it's more like trying to be a lot of things to a lot of people. Which is a commendable undertaking and no doubt solves a lot of the world's problems. But why? Why so many bells and whistles? Why twenty different ways to do the same thing? Is it really necessary? Perhaps there are cases where a library does everything your application needs, in which case, that is a big win as dependency management goes. But being smart in choosing dependencies that involve many libraries isn't so bad either. For one thing, you're likely going to end up working with a more diverse crowd — which is rewarding in its own right. For another thing, you're going to see the barrier to entry lowered in using smaller libraries that don't require such knowledge lock-in.