C/C++ development with package managers

Nenad Mikša,

Head of TSI @ Microblink

@dodo at cpplang.slack.com

meetup@microblink.com

Package manager

  • from Wikipedia:

    a collection of software tools that automates the process of installing, upgrading, configuring, and removing computer programs for a computer's operating system in a consistent manner

  • types:

    • system package managers (e.g. apt-get, yum, brew, chocolate)
    • application package managers (e.g. VSCode Extension Manager)
    • development package managers

Development package managers

  • provide a simple way to obtain dependencies in a reproducible manner
    • support handling multiple versions of the same package
  • provide a facility to update dependencies
  • support resolving conflicts between dependencies
  • integrate well with the build system
  • reduce build times

Other languages

  • Python: pip, easy_install
  • JavaScript: npm
  • Java: maven, gradle
  • Ruby: gem
  • Rust: cargo
  • C#: nuget

C/C++

  • main problem:
    • build system integration
    • no standard build system for C/C++

C/C++ - current solutions

Buckaroo

  • similar idea to nix, but focused on build
  • tightly integrated with buck build system
  • very difficult to integrate packages not using buck build
  • package is simply a link to repository commit

Hunter

  • tightly integrated with CMake
  • supports sharing binary cache
    • not binary packages
  • still difficult to work with (but very promising project)

Vcpkg

Conan

  • backed by JFrog
  • supports reproducible builds
  • supports both binary and source packages
    • friendly for distribution of proprietary libraries
  • not tied to any specific build system
    • but has integrations to most popular ones (CMake, Autotools, Visual Studio, Xcode, Scons, ...)
  • written in Python (installable via pip)

Conan (continued)

  • all dependencies specified in conanfile.txt
  • example:
[requires]
opencv/4.1.1@conan/stable
boost/1.71.0@conan/stable

[options]
boost:header_only=True

[imports]
res, haarcascades/haarcascade_frontalface_alt.xml -> .

Conan - package installation

conan install /path/to/conanfile.txt [-g generator] [-s settings] [-o options]

or

conan install /path/to/conanfile.txt -pr profile
  • generates an import file for specific build system using specified settings and options

Conan demo https://github.com/zagreb-cpp-user-group/meetup-2019-11-28

Dependency overriding

  • possible to override dependencies' dependencies
    • some rebuilds may be necessary, but conan handles this automatically
    • by default, this depends on Semantic Versioning
      • but custom logic can be specified per package
  • if conflicts occur, conan installation fails
    • user needs to resolve it

Creating conan packages

  • create a conanfile.py file with:
conan new package-name/version
  • (optionally) implement the source and build methods
  • implement the package and package_info methods
  • create the package with:
conan create /path/to/conanfile.py user/channel [-pr profile] [-s settings] [-o options]

Template for conan package

from conans import ConanFile, CMake, tools


class FacecameraConan(ConanFile):
    name = "face-camera"
    version = "0.1.0"
    license = "<Put the package license here>"
    author = "<Put your name here> <And your email here>"
    url = "<Package recipe repository url here, for issues about the package>"
    description = "<Description of Facecamera here>"
    topics = ("<Put some tag here>", "<here>", "<and here>")
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False]}
    default_options = {"shared": False}
    generators = "cmake"

    def source(self):
        self.run("git clone https://github.com/conan-io/hello.git")
        # This small hack might be useful to guarantee proper /MT /MD linkage
        # in MSVC if the packaged project doesn't have variables to set it
        # properly
        tools.replace_in_file("hello/CMakeLists.txt", "PROJECT(HelloWorld)",
                              '''PROJECT(HelloWorld)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()''')

    def build(self):
        cmake = CMake(self)
        cmake.configure(source_folder="hello")
        cmake.build()

        # Explicit way:
        # self.run('cmake %s/hello %s'
        #          % (self.source_folder, cmake.command_line))
        # self.run("cmake --build . %s" % cmake.build_config)

    def package(self):
        self.copy("*.h", dst="include", src="hello")
        self.copy("*hello.lib", dst="lib", keep_path=False)
        self.copy("*.dll", dst="bin", keep_path=False)
        self.copy("*.so", dst="lib", keep_path=False)
        self.copy("*.dylib", dst="lib", keep_path=False)
        self.copy("*.a", dst="lib", keep_path=False)

    def package_info(self):
        self.cpp_info.libs = ["hello"]

from conans import ConanFile, CMake, tools


class FacecameraConan(ConanFile):
    name = "face-camera"
    version = "0.1.0"
    license = "<Put the package license here>"
    author = "<Put your name here> <And your email here>"
    url = "<Package recipe repository url here, for issues about the package>"
    description = "<Description of Facecamera here>"
    topics = ("<Put some tag here>", "<here>", "<and here>")
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False]}
    default_options = {"shared": False}
    generators = "cmake"

    def source(self):
        self.run("git clone https://github.com/conan-io/hello.git")
        # This small hack might be useful to guarantee proper /MT /MD linkage
        # in MSVC if the packaged project doesn't have variables to set it
        # properly
        tools.replace_in_file("hello/CMakeLists.txt", "PROJECT(HelloWorld)",
                              '''PROJECT(HelloWorld)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()''')

    def build(self):
        cmake = CMake(self)
        cmake.configure(source_folder="hello")
        cmake.build()

        # Explicit way:
        # self.run('cmake %s/hello %s'
        #          % (self.source_folder, cmake.command_line))
        # self.run("cmake --build . %s" % cmake.build_config)

    def package(self):
        self.copy("*.h", dst="include", src="hello")
        self.copy("*hello.lib", dst="lib", keep_path=False)
        self.copy("*.dll", dst="bin", keep_path=False)
        self.copy("*.so", dst="lib", keep_path=False)
        self.copy("*.dylib", dst="lib", keep_path=False)
        self.copy("*.a", dst="lib", keep_path=False)

    def package_info(self):
        self.cpp_info.libs = ["hello"]

Demo

Uploading packages

conan upload -r <remote> package/version@user/channel [--all]

Custom conan servers

Conclusion

  • development with package managers is easier
    • focus on developing your code, not compiling 3rd party packages
  • multiple solutions for C and C++
  • conan currently has most advantages
    • cross-platform
    • orthogonal to build system
    • decentralized
    • custom/private server instances

Thank you