Bit-baking with soda

In one of my posts last year I mentioned that one can make automated comments to GitLab very easily with the right tooling - especially if they are coming from linting tools. So any author, reviewer and maintainer gets easy feedback as fast as possible on any proposed changes. This is super easy and very convenient when you're always doing a full build and every possible source in your project is actually checked. Why do we need something new, if that's working so well...? In the bitbake world that is different, we have powerful tools like sstate cache along other mechanisms to avoid exactly building everything from scratch all the time. This makes it tricky to map findings from the meta-sca layer (which fully supports sstate caching) to a pull or merge request, as we never can be sure to have the full picture. Moving from the outside, right into it So it was very clear that the commenting part of a CI pipeline needed to be done with the help of bitbake too... et voila scabot ...

 Let that title settle... and now we're getting back to a more serious issue :-). The issue When you're using bitbake layers you usually clone them on the fly when working in a cloud based setup, meaning a full clone of a repo, that could be highly expensive (just look at the size of the linux git repo for instance). As cloud based setups mostly don't supply a good way to sync those resources, unless you invent something yourself or pay for it, every bit counts. Not only as a matter of time but also as a matter of resulting cost. The layer meta-sca I maintain, has grown over the time quite much, so it became very very huge. Also because I made the mistake in the past to put large blobs (in this case tarballs) into the repository. I learned that lesson but I cannot undo it, as we all know each published git revision should stay untouched for all eternity. Mainly this is because of the linked-list nature of git - if I change one commit at the bottom I will alter any commit t...

What's the story? I guess most of you are familiar with the concept of merge or pull requests, so I won't go into details on these - but in case you missed it,  here is a brief description what GitLab thinks it is. There are basically two side involved into a merge request (or MR as the cool kids would say;-)) the one who actually provides the code change, lets call them devs the one who's maintaining the repository, where the change should be applied Both sides might have different objectives. The dev wants the code, she/he already wrote (hopefully tested) and streamlined, to be part of the repo upstream - as otherwise she/he could skipped all that work and enjoyed life (something we should all do from time to time). The maintainer usually lacks some implementation detail and is more keen on having the contributed code to be written according to the project style, the code being regression tested, covered by unit tests, so she/he can be sure that the fresh con...

This time I want to write something about an issue everybody maintaining a git repo might faced already - CI. In theory every push and every pull request should have been build with all supported layer-versions... well in theory. The issue If you have a local setup it's sometimes hard to switch layer-version - I agree the usage of repo  is highly recommended here, as it simplifies such work heavily. Nevertheless you might need multiple work spaces, which all need a lot of disk space. Roughly calculated you can expect ~50GB of data per architecture/distro-combination without the usage of " rm_work "-bbclass, round about 15GB if you're using it. So if you decided to support >3 layer versions of YOCTO, it's a lot of space blocked for a lot of redundant data. Not to mention that you need to build everything now and then to get results, if your code is still working or not. Solution: CI This is where you pick a CI provider - Jenkins immediately comes in...

When using YOCTO for building your software there always comes the point where everything is woven together to something bigger - usually all your recipes are put together to an image, which can be used directly for booting an embedded device. As this is the final step, extra care should be taken when it comes to checking on security. On an recipe level you can make your component as fortified as you could imagine, if another component doesn't care about security at all, your whole concept will fall into pieces. So how do I check on security when an image is created?  What do I need check for? Are there any helpful tools around, which can support me? To answer the last question first - YES there are some very useful tools around, for instance ansible . Ansible is a very flexible, extendable batch processor, which can automate most of the administrative tasks quite convenient. The result of a YOCTO build is mostly an more or less fully featured Linux system, you can...

It's been a while since I posted something in this blog - Been busy enhancing my layer for static-code-analysis over at GitHub - But I'm back to tell you something about the baddest error of them all: the typo. Over all projects I experienced in my professional life, there were always at least one time when a bug flew in about a typo in an error message or in some documentation file. Mostly release date was near, I had spend days and hours on testing the functionality behind it and all that it was keeping it to become part of the master/product was a single typo?!?! Why didn't tell some one me that earlier? Mostly a code review was involved in the process - Had nobody seen this? The short answer: No! Everybody is going blind after looking at code for a while. So how to become aware of a typo or a badly written document as early as possible? If you have been to this blog before, you may the answer: static code analysis! In the IDE I'm using for most of my ...

In my last post I described a way how to ensure a safe and know configuration for KConfig-based system . But there is a lot more that could be altered when you include additional layer into your workspace. Just think of configure options build instructions additional patches being applied license changes (a very bad one) files remove which should have been there a.s.o. I though about this for a longer time... after a very deep dive into the way bitbake is handling the parsing of any file (bb, bbappend, inc, conf) I created a small helper bbclass which you can find here . This class can monitor changes done by bbappend file (no matter where they are actually located in your workspace) and diff them to the plain recipe. You can now define a set of variables which you don't want to be changed at all (like. SRC_URI, or LICENSE). The helper class does then the magic for you and diffs them against each other. If a variable (and all there machine/arch-specific var...

As you may know KConfig is a well-known system for configuring software modules, as used by busybox , linux kernel , u-boot and many more. If you are unfamiliar you might want to read this nice tutorial . So as KConfig is well established it is used at different locations throughout YOCTO. Basically YOCTO is doing it the following way. You have a base configuration file called "defconfig". This is placed somewhere in the recipe-tree. It describes the absolute minimum of configuration needed. Depending on the feature you want to add, there are *.cfg-file around the recipe-tree. The configuration is done with all the *.cfg-files being merged into the defconfig. This is passed to "make oldconfig" to sanity check and put all the not defined symbols into the configuration. And voila you have a fresh valid configuration to build your software. It's pretty obvious that this is one stage where things can go very wrong...! Mostly the basis (KConfig) is chan...

Have you ever considered to use python as a scripting language in an embedded system? I've been using this on recent projects although it wasn't my first choice. If I had to choose a scripting language to be used in embedded I always had a strong preference for shell/bash or lua, because they are either builtin or designed to have a significant lower footprint compared to others. Nevertheless the choice was python3 (was out of my hands to decide). When putting together the first builds using YOCTO I realized that there are two sides to python. the starting phase, where the app is initializing the execution phase, where the app just processes new data In the 2nd phase python3 has good tradeoffs between maintainability of code vs. execution speed, so there is nothing to moan about. Startup is the worst But the 1st phase where the python3-interpreter is starting is really bad. So I did some research where is might be coming from. Just to give a comparison of ...

This post shall be the start of a loose series on static code analysis tools that I've been using when professionally coding. The tools should be compared against the same code base, so that one could get an impression what the benefits and flaws of a certain tool are. Please put your questions/suggestions into the comments. Today's tool should be cppcheck . Supported Languages C C++ Dependencies (besides gcc) libprce Special features custom rules for checking can be applied as prce-compatible regular expressions Testing Under test shall be the unmodified code of busybox version 1.29.2 as to be found as yocto-poky layer at gitrev 3541f019a505d18263fad0b46b88d470e3fd9d62. As configuration for yocto the standard configuration is used. cppcheck is used in version 1.87 Number of total findings in code : 308 Number of different finding types : 26 Findings in details class vs. count nullPointer 3 memleakOnRealloc ...