Bit-baking with soda

This post is special, in the sense that it marks an important event in my life as a software developer. But let's start from the beginning... oelint-adv When you use bitbake/YOCTO on a frequent basis, you might have are checked out my linting tool for these file called  oelint-adv . This tool parses bitbake file and checks for several rules and best practices when it comes to working within this ecosystem. Initially I thought about using the original bitbake parser , which is available as open source licensed under GPL-2.0 - Unfortunately it requires more or less a whole workspace setup, including all layers a.s.o. As this was meant as a quick check tool to be used in CI or git pre-commit hooks, I was pretty sure that this wouldn't work, so I wrote my own parser for bitbake files, till then exclusively used with the linting tool. Scancode-toolkit All of a sudden, I got a few weeks back a ticket on my GitHub repo that somebody else had more or less the same issue - they wanted t...

When building a YOCTO based build in CI, normally you would get something like the following in your build log 1 2 3 4 5 6 7 Sstate summary: Wanted 0 Found 0 Missed 0 Current 60 (0% match, 100% complete) NOTE: Executing Tasks NOTE: Setscene tasks completed NOTE: Running task 272 of 272 (/build/my-recipes/python3-systemdlint-native/python3-systemdlint-native_1.1.12.bb:do_addto_recipe_sysroot) NOTE: recipe python3-systemdlint-native-1.1.12-r0: task do_addto_recipe_sysroot: Started NOTE: recipe python3-systemdlint-native-1.1.12-r0: task do_addto_recipe_sysroot: Succeeded NOTE: Tasks Summary: Attempted 272 tasks of which 271 didn't need to be rerun and all succeeded. depending on your build this could go on for hundreds or thousands of lines. If there is an error or a warning, typically one would use some sort of regular expression to filter out these items and present them somehow to the user. Unfortunately sometimes warnings and errors are multi-line and coding regex for ...

The YOCTO-project can do amazing things, but requires a very decent build machine, as by nature when you build everything from scratch it does require a lot of compilation. So the ultimate goal has to be to perform only the necessary steps in each run. Understanding task hashing The thing is that bitbake uses a task hashing to determine, which tasks (such as compilation, packaging, a.s.o.) are actually required to be performed. As tasks depend on each other, this information is also embedded into a hash, so the last task for a recipe is ultimately depending on the variable that are used for this specific task and every task before. You could visualize this by using a utility called bitbake-dumpsig , which produces output like this basewhitelist: {'SOURCE_DATE_EPOCH', 'FILESEXTRAPATHS', 'PRSERV_HOST', 'THISDIR', 'TMPDIR', 'WORKDIR', 'EXTERNAL_TOOLCHAIN', 'FILE', 'BB_TASKHASH', 'USER', 'BBSERVER...

In this post I'm telling my journey which started with my blog post about using Github Action as my main CI provider - in case you missed it see here  - on how to do a full yocto/poky build with just ~14GB of disk space Typically disk space is cheap nowadays and can be used without thinking too much about it - which leads to disk usage of 50GB and more for a yocto/poky build. So what are the options, when disk space becomes precious? Constraints, constraints, constraints While Github Actions is free for open source project (like mine) it is highly limited in regards of resources. Currently (2020/04/11) you get ( source )  2-core CPU 7 GB of RAM memory 14 GB of SSD disk space maximum of 6 hours per pipeline that's not very much when you think about doing a poky/yocto build on that, so every byte actually counts. Constant pain As the involved layer are constantly growing one has to care about every byte that could be saved, without giving up the overall a...

The 'a' in compliance stands for abstract things, that most developer try to avoid artificial created rules (as it's the nature of laws to be sometimes illogical;-)) an absolute must-have if you're working with open-source software. for me it additionally stand for automation potential Introducing the issue Just think of the following situation: a random guy, you never worked with before (and you don't know personally) tries to contribute a bitbake recipe for a new component. As we all know each recipe has a LICENSE entry, which offers a SPDX compatible representation of a LICENSE (or multiple licenses) applied to the source code, that the recipe offers build information for. So how can you be sure, that this setting is correct? Well you can scan the source code all by yourself and try to figure out under what license the source code is provided. For a single source repository, I guess that's  fairly easy, but time consuming. B...

What's the story? I guess most of you are familiar with the concept of merge or pull requests, so I won't go into details on these - but in case you missed it,  here is a brief description what GitLab thinks it is. There are basically two side involved into a merge request (or MR as the cool kids would say;-)) the one who actually provides the code change, lets call them devs the one who's maintaining the repository, where the change should be applied Both sides might have different objectives. The dev wants the code, she/he already wrote (hopefully tested) and streamlined, to be part of the repo upstream - as otherwise she/he could skipped all that work and enjoyed life (something we should all do from time to time). The maintainer usually lacks some implementation detail and is more keen on having the contributed code to be written according to the project style, the code being regression tested, covered by unit tests, so she/he can be sure that the fresh con...

When you're working with bitbake, especially when you're maintaining recipes, you might have asked yourself how do I know which recipes I need to update As we all read the YOCTO manual from beginning till the very end (as all good participants;-), you might be aware that you can check the `upstream` status of a package/recipe be running devtool check-upgrade-status myrecipe which returns (if there is an update available) INFO:myrecipe            0.12.8          0.12.9          None 297cb2458a96ea96d5e9d6ef38f1b7305c071f32  that means, currently you're running on version 0.12.8 and there is an update to 0.12.9 available at the defined source you're pulling your sources from. So far so easy - but wait Do I have to do that all manually, each and everytime? Automating things No, of course not. I'm using GitHub for hosting my sources, so I thought it would be quite convenient to let some ...