I will be a bit cynical for a bit.

There isn't anything in C++ as a language preventing us from having a good PDM.
Of course, the preprocessor completely undermines that observation, but the preprocessor is not part of C++. It's a bad templating engine awkwardly sitting on top.
And people have expectations that various output of that templating engine with various set of parameters will lead to compatible sources. It's madness.
With modules, the committee had a unique opportunity to sandbox that madness a bit. They are apparently shying away from that.
And that' s why we may fail to come up with a good dependency manager.

Not just because of the preprocessor, but because having a universal tool will require strong constraints on what a project can do in its build system.
We need to take away freedom. 
We need to limit the number of compiler flags a library can use and impose to others, we need to limit the number of ways a given source file can be expressed.
It's the only way to have a system of manageable complexity with a manageable numbers of unsolvable dependencies.

And we will shy away from that.

The only way to have a large number of libraries that one can include in their projects from the same place, with the same straight forward workflow is to have those libraries play by the same rules.
Other tools in other language impose those rules, they have limited abilities. Yet they are powerful enough that everybody use those tool with great effects.
But those tools were there from the start, so, the code was made with these tools in mind.

There is something to be said for monolithic ecosystems.

From a user perspective, to install the package `foo` maybe they need to type a command or maybe the build system will do that for them. 
If the command does not exist on the system, we failed. If the correct command is not identical to the one indicated in the documentation of the project, we failed. If that documentation is longer than exactly one line in most cases, we failed, if the build system does not support it, we failed.
(imagine : vcpkg get bar/foo / xcode-cpp-fetch  baz.com/bar:foo / iso-get --from baz.com --org bar --import=export --package foo@version:<auto last> ) 

And, if there is anyway `foo` can resolve to different projects, we failed.
Monolithic systems also exist because there is a need for "dependency foo" to have exactly one meaning.

Doing a dependency manager is hard but as you said, it's a solved problem. Doing one able to handle the existing code may not be. It's harder. Maybe not solvable. Is it worth trying ? I don't know.
Like Titus said, people need too learn what a well behaved dependency is.

But if we decide to have an idealistic approach, will people adopt that new tool/protocol/standard ? Nobody like dealing with build systems, and whatever crappy CMake scripts they have today "works" - well, they think they do.

Rust, for example did not endure 30 years of complete wild west jungle mess before offering those tools, that's the difference.
Likewise, I'm doubtful that anyone in those communities would be successful in offering an alternative solution to their idiomatic tools.

So, really C++ is not the major issue.
The major issue is the amount of C++ libraries out there all with overly complex, non portable build scripts, the unwillingness of their authors to support something new or their lack of resources to do so.
And a package manager without libraries is a bit pointless, isn't it ? Chicken egg problem and all. (I also tend to think that the publication of a given project in any kind of public repository is best left to their author, rather than a third party packager as that weakens the trust in the dependencies)

Corentin

Le ven. 16 mars 2018 à 04:46, Rene Rivera <grafikrobot@gmail.com> a écrit :
We have a rather serious problem in our hands. On one hand we have a clamoring of people who want a PDM driven ecosystem for C++. On the other we have those who think this is an impossible task and we should give up now. To both groups I can say that C++ PDMs exist and are growing in popularity and breadth and making something standard from the existing practice is possible. We just have to decide how much should be standard. And the same goes for build systems.

With that in mind I've been looking at how we get from where are today and the future many of us want. Here are some guidelines for approaching the problem:

1. This is actually a solved problem. And as such we need to identify what works and doesn't from existing solutions.

2. We should strive for an open implementation model in line with how C++ is currently specified.

3. It is impossible to specify everything ahead of time as the C++ tool and library landscape changes faster than we can possible write standards for them.

4. This is the ecosystem we are trying to improve. Hence we need to think of how the solutions fit the overall domain.

5. This is a placeholder item. This is not a complete list and should grow over time. Please suggest more items :-)


That's the TLDR version of the guidelines. Here's the long version..

It's been done before (1)

We have a variety of existing solution for both PDMs for C++, and other languages (for instance conan and cget). Build systems are numerous, and varied. There are even some systems that combine both.

Open implementation model (2)

When we look at other ecosystems we see mostly a "monolithic" model. We see one compiler, one runtime, one PDM, one build system, all integrated to work together. Although this is an attractive model to present users it doesn't fit the C++ heterogeneous model:

* We already have a multitude of compilers.
* That multitude increases the rate of improvement of those compilers, and related tools.
* We can tailor those compilers to specific use cases while presenting the same language to users.

We achieved that model by not specifying the implementation of the language. But by instead specifying the "surface" of the language. This same heterogeneous model is what we should follow when consider PDMs and build systems. We should define the "surface" of how PDM and build system implementations operate and communicate within the ecosystem. This is the achievable part.

Open specification (3)

The possibilities of the specifics of what a PDM and build system can accomplish is open ended. And as we see from current compiler solutions, and technological innovations, it's a quickly changing landscape. We need specifications that are flexible enough to grow and adjust faster than the core C++ Language standard has evolved so far. This implies that we can't follow the current model of the standards process to accommodate this flexibility. We need a standards model that allows for new specifications "outside" of the core. There is one group of standards that follows an open model we can borrow from, OpenGL. OpenGL's core specifications with added extension specifications allows for quick adaptation to the quickly changing GPU advancements. We can use such a structure to model the changing C++ tool advancements without waiting for the three year cycle.

Fit what we have and know (4)

If we have some hope of improving the C++ ecosystem this generation we need to work within the current set of tools we have. Our ecosystem may be unwieldy and downright treacherous at times, but it's what has worked for us. We need to conceive of specifications that fit CLIs, IDEs and their related checkers, debuggers, build systems, PDMs, distribution mechanism, and all that without closing doors to different and better technologies. It will be a golden era when we can pick and choose our preferred set C++ tools and have them all work together seamlessly (within obvious constraints). For PDMs and build systems this means thinking about not just CLI environments. But IDE, GUI installers, GUI wizards, CI automation, and so on when defining how PDMs and build systems should operate.


--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net

_______________________________________________
Tooling mailing list
Tooling@isocpp.open-std.org
http://www.open-std.org/mailman/listinfo/tooling