Multiple Repository support


Cliff Brake <cliff.brake@...>
 

Hello,

I've started collecting ideas from various emails on multiple
repository support.

http://wiki.openembedded.org/index.php/MultipleRepositoryMethods

Please feel free to update the above page.

In my mind, this is a key problem we need to solve, not just for
Yocto/OE, but also for anyone doing product development.

I've personally been using git submodules for most projects, and repo
for Android based projects.

Appreciate any ideas, experiences, or insights into how we solve this problem.

Thanks,
Cliff

--
=================
http://bec-systems.com


Bruce Ashfield <bruce.ashfield@...>
 

On 10-12-22 11:09 AM, Cliff Brake wrote:
Hello,

I've started collecting ideas from various emails on multiple
repository support.

http://wiki.openembedded.org/index.php/MultipleRepositoryMethods

Please feel free to update the above page.

In my mind, this is a key problem we need to solve, not just for
Yocto/OE, but also for anyone doing product development.

I've personally been using git submodules for most projects, and repo
for Android based projects.

Appreciate any ideas, experiences, or insights into how we solve this problem.
We've used (and walked away) from git submodules in the
past for some really large projects. Our experiences closely
match "the bad" on the wiki link you sent. I've met very
few people who can effectively manage active development
in a git submodule based environment. My blunt opinion is
that if I never see another one in my life, I'd probably
be happy. I haven't looked at a submodule in about a year,
so maybe they have been fixed for the better .. but I'm
skeptical.

The solution to the submodule chaos that was used
sucessfully was to create something we called
'subgits'. They use git, a wrapper script and standard
git configs. A subgit points to a remote repo, specifies
where it should be cloned, and has a place for special
properties. You can recursively pull from a top level
and have all subgits updated in a uniform fashion. The
model can be extended for multi repository development
as well. The big win here is that you can independently
update repos, use normal git workflows, or wrap it as
you want. Nothing is forced.

Note: all the functionality of submodules isn't present
here (global tracking, bisect, etc), but that was functionality
that wasn't really crucial.

Just some comments and opinions to throw into the
discussion.

Cheers,

Bruce




Thanks,
Cliff


Esben Haabendal <eha@...>
 

On Wed, 2010-12-22 at 11:09 -0500, Cliff Brake wrote:
Hello,

I've started collecting ideas from various emails on multiple
repository support.

http://wiki.openembedded.org/index.php/MultipleRepositoryMethods

Please feel free to update the above page.

In my mind, this is a key problem we need to solve, not just for
Yocto/OE, but also for anyone doing product development.

I've personally been using git submodules for most projects, and repo
for Android based projects.

Appreciate any ideas, experiences, or insights into how we solve this problem.
We are using git submodules for just this task in OE-lite.
It works pretty well, but I fear that this will not be the case if a
push model is used for the top repository.

We have partly wrapped the git submodule configuration into a bitbake
parsed configuration file, looking something like:

# OE-lite/core metadata
OE_MODULES += "core"
OE_MODULE_PATH_core = "meta/core"
OE_MODULE_URL_core =
"git://git.doredevelopment.dk/oe-lite/core.git"
OE_MODULE_PUSHURL_core =
"ssh://dev.doredevelopment.dk/srv/public/git/oe-lite/core.git"
OE_MODULE_BRANCH_core = "master"
OE_MODULE_REMOTES_core += "gitorious"
OE_MODULE_REMOTE_core_gitorious = "git@...:oe-lite/core.git"

So developers get a more complete and consistent submodule setup.

We considered repo, but the KISS principle ruled in favor of git
submodules. I believe the learning curve is steep enough for any
newcomers to OE, so having to figure out how to master something like
repo also does not seem so attractive.

/Esben

/Esben